2023-10-18T08:48:01.870 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-18T08:48:01.871 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} --name pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726 --verbose --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/orig.config.yaml 2023-10-18T08:48:01.934 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T08:48:02.035 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726 branch: wip-batrick-testing-20231016.203825 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7430726' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-batrick-testing-20231016.203825 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231016.203825 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7703 sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb sleep_before_teardown: 0 subset: 153/256 suite: fs suite_branch: wip-batrick-testing-20231016.203825 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi061.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_23:59:58 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-18T08:48:02.035 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa; will attempt to use it 2023-10-18T08:48:02.036 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks 2023-10-18T08:48:02.037 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-18T08:48:02.039 INFO:teuthology.task.internal:Checking packages... 2023-10-18T08:48:02.068 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f7420d7f8e55ceea786dcbf536c338ce920c21eb' 2023-10-18T08:48:02.069 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-18T08:48:02.069 INFO:teuthology.packaging:ref: None 2023-10-18T08:48:02.070 INFO:teuthology.packaging:tag: None 2023-10-18T08:48:02.070 INFO:teuthology.packaging:branch: wip-batrick-testing-20231016.203825 2023-10-18T08:48:02.070 INFO:teuthology.packaging:sha1: f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T08:48:02.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-batrick-testing-20231016.203825 2023-10-18T08:48:02.305 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6786.gf7420d7f 2023-10-18T08:48:02.306 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-18T08:48:02.313 INFO:teuthology.task.internal:no buildpackages task found 2023-10-18T08:48:02.313 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-18T08:48:02.360 INFO:teuthology.task.internal:Saving configuration 2023-10-18T08:48:02.384 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-18T08:48:02.428 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T08:48:02.455 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.407029', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T08:48:02.480 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.408790', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T08:48:02.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.405045', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T08:48:02.503 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-18T08:48:02.510 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-18T08:48:02.511 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-18T08:48:02.511 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-18T08:48:02.511 INFO:teuthology.run_tasks:Running task console_log... 2023-10-18T08:48:02.621 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f84d1a84ca0>, signals=[15]) 2023-10-18T08:48:02.622 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-18T08:48:02.629 INFO:teuthology.task.internal:Opening connections... 2023-10-18T08:48:02.629 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T08:48:02.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:48:02.700 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2023-10-18T08:48:02.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:48:02.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-10-18T08:48:02.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:48:02.842 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-18T08:48:02.854 DEBUG:teuthology.orchestra.run.smithi061:> uname -m 2023-10-18T08:48:02.876 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2023-10-18T08:48:02.877 DEBUG:teuthology.orchestra.run.smithi061:> cat /etc/os-release 2023-10-18T08:48:02.936 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T08:48:02.937 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T08:48:02.937 INFO:teuthology.orchestra.run.smithi061.stdout:ID="rhel" 2023-10-18T08:48:02.937 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE="fedora" 2023-10-18T08:48:02.937 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="8.6" 2023-10-18T08:48:02.938 INFO:teuthology.orchestra.run.smithi061.stdout:PLATFORM_ID="platform:el8" 2023-10-18T08:48:02.938 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T08:48:02.938 INFO:teuthology.orchestra.run.smithi061.stdout:ANSI_COLOR="0;31" 2023-10-18T08:48:02.938 INFO:teuthology.orchestra.run.smithi061.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T08:48:02.939 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T08:48:02.939 INFO:teuthology.orchestra.run.smithi061.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T08:48:02.939 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T08:48:02.939 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T08:48:02.939 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T08:48:02.940 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T08:48:02.940 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T08:48:02.940 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T08:48:02.941 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2023-10-18T08:48:02.962 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2023-10-18T08:48:02.983 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2023-10-18T08:48:02.983 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2023-10-18T08:48:03.043 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T08:48:03.043 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T08:48:03.043 INFO:teuthology.orchestra.run.smithi110.stdout:ID="rhel" 2023-10-18T08:48:03.043 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="fedora" 2023-10-18T08:48:03.044 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="8.6" 2023-10-18T08:48:03.044 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el8" 2023-10-18T08:48:03.044 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T08:48:03.044 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2023-10-18T08:48:03.044 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T08:48:03.045 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T08:48:03.045 INFO:teuthology.orchestra.run.smithi110.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T08:48:03.045 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T08:48:03.045 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T08:48:03.045 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T08:48:03.046 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T08:48:03.046 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T08:48:03.046 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T08:48:03.047 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2023-10-18T08:48:03.070 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-10-18T08:48:03.089 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-10-18T08:48:03.090 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-10-18T08:48:03.149 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2023-10-18T08:48:03.150 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.6 (Ootpa)" 2023-10-18T08:48:03.150 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2023-10-18T08:48:03.150 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2023-10-18T08:48:03.150 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.6" 2023-10-18T08:48:03.151 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2023-10-18T08:48:03.151 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-18T08:48:03.151 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-10-18T08:48:03.151 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-18T08:48:03.151 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2023-10-18T08:48:03.152 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-18T08:48:03.152 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-18T08:48:03.152 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T08:48:03.152 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-18T08:48:03.152 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-18T08:48:03.153 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-18T08:48:03.153 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-18T08:48:03.153 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-10-18T08:48:03.178 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-18T08:48:03.189 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-18T08:48:03.267 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-18T08:48:03.268 DEBUG:teuthology.orchestra.run.smithi061:> test '!' -e /home/ubuntu/cephtest 2023-10-18T08:48:03.271 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2023-10-18T08:48:03.273 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-10-18T08:48:03.291 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-18T08:48:03.298 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-18T08:48:03.298 DEBUG:teuthology.orchestra.run.smithi061:> test -z $(ls -A /var/lib/ceph) 2023-10-18T08:48:03.330 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2023-10-18T08:48:03.332 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-10-18T08:48:03.377 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-18T08:48:03.448 INFO:teuthology.run_tasks:Running task kernel... 2023-10-18T08:48:03.462 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-18T08:48:03.462 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-18T08:48:03.463 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-18T08:48:03.463 DEBUG:teuthology.orchestra.run.smithi061:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T08:48:03.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:48:03.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-18T08:48:03.725 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:48:03.726 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-18T08:48:03.726 DEBUG:teuthology.orchestra.run.smithi061:> uname -r 2023-10-18T08:48:03.746 INFO:teuthology.orchestra.run.smithi061.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-18T08:48:03.747 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:48:03.747 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-18T08:48:03.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum upgrade -y linux-firmware 2023-10-18T08:48:04.939 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-10-18T08:48:04.940 INFO:teuthology.orchestra.run.smithi061.stdout:Unable to read consumer identity 2023-10-18T08:48:06.299 INFO:teuthology.orchestra.run.smithi061.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-10-18T08:48:06.991 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-10-18T08:48:10.683 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 427 kB/s | 24 kB 00:00 2023-10-18T08:48:11.500 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-10-18T08:48:11.501 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-10-18T08:48:11.501 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-10-18T08:48:11.626 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T08:48:11.626 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-18T08:48:11.839 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ 2023-10-18T08:48:11.840 DEBUG:teuthology.orchestra.run.smithi061:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-18T08:48:11.842 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi061.front.sepia.ceph.com to complete... 2023-10-18T08:48:27.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18 08:48:27 URL:https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [383161052/383161052] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-18T08:48:27.720 INFO:teuthology.orchestra.run.smithi061.stderr:FINISHED --2023-10-18 08:48:27-- 2023-10-18T08:48:27.720 INFO:teuthology.orchestra.run.smithi061.stderr:Total wall clock time: 16s 2023-10-18T08:48:27.720 INFO:teuthology.orchestra.run.smithi061.stderr:Downloaded: 1 files, 365M in 16s (23.4 MB/s) 2023-10-18T08:48:27.722 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-18T08:48:27.722 DEBUG:teuthology.orchestra.run.smithi061:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-18T08:48:29.376 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying... ######################################## 2023-10-18T08:48:29.434 INFO:teuthology.orchestra.run.smithi061.stdout:Preparing... ######################################## 2023-10-18T08:48:29.520 INFO:teuthology.orchestra.run.smithi061.stdout:Updating / installing... 2023-10-18T08:49:10.464 INFO:teuthology.orchestra.run.smithi061.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-18T08:49:14.987 INFO:teuthology.orchestra.run.smithi061.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-18T08:49:35.003 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi061.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-18T08:49:35.004 DEBUG:teuthology.orchestra.run.smithi061:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-18T08:49:35.061 INFO:teuthology.orchestra.run.smithi061.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.062 INFO:teuthology.orchestra.run.smithi061.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.062 INFO:teuthology.orchestra.run.smithi061.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.062 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.063 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-18T08:49:35.063 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-18T08:49:35.063 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-18T08:49:35.063 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-18T08:49:35.063 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-18T08:49:35.064 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-18T08:49:35.064 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-18T08:49:35.064 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-18T08:49:35.064 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-18T08:49:35.065 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-18T08:49:35.065 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-18T08:49:35.065 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-18T08:49:35.065 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-18T08:49:35.066 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-18T08:49:35.066 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-18T08:49:35.066 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-18T08:49:35.066 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-18T08:49:35.067 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-18T08:49:35.067 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-18T08:49:35.067 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-18T08:49:35.067 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-18T08:49:35.067 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-18T08:49:35.068 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-18T08:49:35.068 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-18T08:49:35.068 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-18T08:49:35.068 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-18T08:49:35.069 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-18T08:49:35.069 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-18T08:49:35.069 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-18T08:49:35.069 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-18T08:49:35.069 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-18T08:49:35.070 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-18T08:49:35.070 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-18T08:49:35.070 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-18T08:49:35.070 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-18T08:49:35.070 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-18T08:49:35.071 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-18T08:49:35.071 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-18T08:49:35.071 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-18T08:49:35.071 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-18T08:49:35.071 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-18T08:49:35.072 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-18T08:49:35.072 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-18T08:49:35.072 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-18T08:49:35.072 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-18T08:49:35.072 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-18T08:49:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-18T08:49:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-18T08:49:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-18T08:49:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-18T08:49:35.073 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-18T08:49:35.074 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-18T08:49:35.074 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-18T08:49:35.074 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-18T08:49:35.074 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-18T08:49:35.074 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-18T08:49:35.075 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-18T08:49:35.075 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-18T08:49:35.075 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-18T08:49:35.075 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-18T08:49:35.075 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-18T08:49:35.076 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-18T08:49:35.076 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-18T08:49:35.076 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-18T08:49:35.076 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-18T08:49:35.077 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-18T08:49:35.078 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-18T08:49:35.078 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-18T08:49:35.078 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-18T08:49:35.078 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-18T08:49:35.079 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-18T08:49:35.080 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-18T08:49:35.080 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-18T08:49:35.080 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-18T08:49:35.080 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-18T08:49:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-18T08:49:35.082 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-18T08:49:35.082 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-18T08:49:35.082 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-18T08:49:35.082 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-18T08:49:35.082 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-18T08:49:35.083 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-18T08:49:35.084 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-18T08:49:35.084 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-18T08:49:35.084 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-18T08:49:35.084 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-18T08:49:35.084 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-18T08:49:35.085 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-18T08:49:35.085 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-18T08:49:35.085 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-18T08:49:35.085 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-18T08:49:35.085 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-18T08:49:35.086 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-18T08:49:35.086 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-18T08:49:35.086 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-18T08:49:35.086 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-18T08:49:35.086 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-18T08:49:35.087 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-18T08:49:35.087 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-18T08:49:35.087 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-18T08:49:35.087 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-18T08:49:35.087 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-18T08:49:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-18T08:49:35.089 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-18T08:49:35.089 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-18T08:49:35.089 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-18T08:49:35.089 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-18T08:49:35.089 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-18T08:49:35.090 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-18T08:49:35.090 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-18T08:49:35.090 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-18T08:49:35.090 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-18T08:49:35.091 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-18T08:49:35.091 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-18T08:49:35.091 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-18T08:49:35.091 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-18T08:49:35.091 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-18T08:49:35.092 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-18T08:49:35.092 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-18T08:49:35.092 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-18T08:49:35.092 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-18T08:49:35.092 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-18T08:49:35.093 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-18T08:49:35.094 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-18T08:49:35.094 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-18T08:49:35.094 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-18T08:49:35.094 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-18T08:49:35.095 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-18T08:49:35.096 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-18T08:49:35.096 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-18T08:49:35.096 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-18T08:49:35.096 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-18T08:49:35.096 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-18T08:49:35.097 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-18T08:49:35.097 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-18T08:49:35.097 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-18T08:49:35.097 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-18T08:49:35.097 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-18T08:49:35.098 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-18T08:49:35.098 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-18T08:49:35.098 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-18T08:49:35.098 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-18T08:49:35.098 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-18T08:49:35.099 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-18T08:49:35.099 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-18T08:49:35.099 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-18T08:49:35.099 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-18T08:49:35.099 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-18T08:49:35.100 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-18T08:49:35.101 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-18T08:49:35.101 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-18T08:49:35.101 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-18T08:49:35.101 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-18T08:49:35.101 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-18T08:49:35.102 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-18T08:49:35.102 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-18T08:49:35.102 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-18T08:49:35.102 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-18T08:49:35.102 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-18T08:49:35.103 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-18T08:49:35.103 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-18T08:49:35.103 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-18T08:49:35.103 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-18T08:49:35.103 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-18T08:49:35.104 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-18T08:49:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-18T08:49:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-18T08:49:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-18T08:49:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-18T08:49:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-18T08:49:35.106 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-18T08:49:35.106 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-18T08:49:35.106 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-18T08:49:35.106 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-18T08:49:35.106 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-18T08:49:35.107 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-18T08:49:35.107 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-18T08:49:35.107 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-18T08:49:35.107 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-18T08:49:35.107 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-18T08:49:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-18T08:49:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-18T08:49:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-18T08:49:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-18T08:49:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-18T08:49:35.109 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-18T08:49:35.109 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-18T08:49:35.109 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-18T08:49:35.109 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-18T08:49:35.109 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-18T08:49:35.110 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-18T08:49:35.110 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-18T08:49:35.110 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-18T08:49:35.110 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-18T08:49:35.110 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-18T08:49:35.111 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-18T08:49:35.112 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-18T08:49:35.113 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-18T08:49:35.114 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-18T08:49:35.114 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-18T08:49:35.114 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-18T08:49:35.114 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-18T08:49:35.114 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-18T08:49:35.115 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-18T08:49:35.115 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-18T08:49:35.115 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-18T08:49:35.115 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-18T08:49:35.115 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-18T08:49:35.117 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-18T08:49:35.117 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-18T08:49:35.117 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-18T08:49:35.117 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-18T08:49:35.117 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-18T08:49:35.118 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-18T08:49:35.118 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-18T08:49:35.118 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-18T08:49:35.118 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-18T08:49:35.119 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-18T08:49:35.120 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-18T08:49:35.121 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-18T08:49:35.122 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-18T08:49:35.122 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-18T08:49:35.122 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-18T08:49:35.122 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-18T08:49:35.122 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-18T08:49:35.123 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-18T08:49:35.124 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-18T08:49:35.124 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-18T08:49:35.124 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-18T08:49:35.124 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-18T08:49:35.124 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-18T08:49:35.125 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-18T08:49:35.125 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-18T08:49:35.125 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-18T08:49:35.125 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-18T08:49:35.125 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-18T08:49:35.126 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-18T08:49:35.126 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-18T08:49:35.126 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-18T08:49:35.126 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-18T08:49:35.126 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-18T08:49:35.127 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-18T08:49:35.127 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-18T08:49:35.127 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-18T08:49:35.127 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-18T08:49:35.127 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-18T08:49:35.128 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-18T08:49:35.128 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-18T08:49:35.128 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-18T08:49:35.128 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-18T08:49:35.128 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-18T08:49:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-18T08:49:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-18T08:49:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-18T08:49:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-18T08:49:35.129 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-18T08:49:35.130 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-18T08:49:35.130 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-18T08:49:35.130 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-18T08:49:35.130 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-18T08:49:35.130 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-18T08:49:35.131 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-18T08:49:35.132 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-18T08:49:35.132 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-18T08:49:35.132 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-18T08:49:35.132 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-18T08:49:35.132 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-18T08:49:35.133 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-18T08:49:35.133 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-18T08:49:35.133 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-18T08:49:35.133 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-18T08:49:35.134 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-18T08:49:35.134 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-18T08:49:35.134 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-18T08:49:35.134 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-18T08:49:35.134 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-18T08:49:35.135 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-18T08:49:35.136 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-18T08:49:35.137 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-18T08:49:35.137 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-18T08:49:35.137 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-18T08:49:35.137 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-18T08:49:35.137 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-18T08:49:35.138 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-18T08:49:35.139 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-18T08:49:35.139 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-18T08:49:35.139 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-18T08:49:35.139 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-18T08:49:35.139 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-18T08:49:35.140 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-18T08:49:35.141 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-18T08:49:35.142 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-18T08:49:35.142 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-18T08:49:35.142 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-18T08:49:35.142 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-18T08:49:35.142 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-18T08:49:35.143 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-18T08:49:35.143 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-18T08:49:35.143 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-18T08:49:35.143 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-18T08:49:35.144 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-18T08:49:35.145 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-18T08:49:35.145 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-18T08:49:35.145 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-18T08:49:35.145 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-18T08:49:35.145 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-18T08:49:35.146 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-18T08:49:35.146 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-18T08:49:35.146 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-18T08:49:35.146 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-18T08:49:35.146 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-18T08:49:35.147 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-18T08:49:35.147 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-18T08:49:35.147 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-18T08:49:35.147 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-18T08:49:35.147 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-18T08:49:35.148 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-18T08:49:35.148 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-18T08:49:35.148 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-18T08:49:35.148 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-18T08:49:35.148 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-18T08:49:35.149 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-18T08:49:35.149 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-18T08:49:35.149 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-18T08:49:35.149 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-18T08:49:35.149 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-18T08:49:35.150 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-18T08:49:35.150 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-18T08:49:35.150 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-18T08:49:35.150 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-18T08:49:35.150 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-18T08:49:35.151 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-18T08:49:35.151 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-18T08:49:35.151 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-18T08:49:35.151 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-18T08:49:35.151 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-18T08:49:35.152 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-18T08:49:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-18T08:49:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-18T08:49:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-18T08:49:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-18T08:49:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-18T08:49:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-18T08:49:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-18T08:49:35.156 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-18T08:49:35.156 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-18T08:49:35.156 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-18T08:49:35.156 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-18T08:49:35.156 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-18T08:49:35.157 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-18T08:49:35.158 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-18T08:49:35.158 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-18T08:49:35.158 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-18T08:49:35.158 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-18T08:49:35.159 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-18T08:49:35.159 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-18T08:49:35.159 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-18T08:49:35.159 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-18T08:49:35.159 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-18T08:49:35.160 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-18T08:49:35.161 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-18T08:49:35.161 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-18T08:49:35.161 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-18T08:49:35.161 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-18T08:49:35.161 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-18T08:49:35.162 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-18T08:49:35.162 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-18T08:49:35.162 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-18T08:49:35.162 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-18T08:49:35.162 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-18T08:49:35.163 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-18T08:49:35.163 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-18T08:49:35.163 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-18T08:49:35.163 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-18T08:49:35.163 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-18T08:49:35.164 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-18T08:49:35.164 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-18T08:49:35.164 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-18T08:49:35.164 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-18T08:49:35.164 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-18T08:49:35.165 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-18T08:49:35.166 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-18T08:49:35.166 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-18T08:49:35.166 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-18T08:49:35.166 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-18T08:49:35.166 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-18T08:49:35.167 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-18T08:49:35.168 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-18T08:49:35.168 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-18T08:49:35.168 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-18T08:49:35.168 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-18T08:49:35.168 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-18T08:49:35.169 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-18T08:49:35.169 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-18T08:49:35.169 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-18T08:49:35.169 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-18T08:49:35.169 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-18T08:49:35.170 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-18T08:49:35.170 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-18T08:49:35.170 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-18T08:49:35.170 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-18T08:49:35.170 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-18T08:49:35.171 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-18T08:49:35.172 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-18T08:49:35.172 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-18T08:49:35.172 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-18T08:49:35.172 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-18T08:49:35.173 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-18T08:49:35.173 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-18T08:49:35.173 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-18T08:49:35.173 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-18T08:49:35.173 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-18T08:49:35.174 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-18T08:49:35.175 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-18T08:49:35.175 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-18T08:49:35.175 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-18T08:49:35.175 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-18T08:49:35.175 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-18T08:49:35.176 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-18T08:49:35.177 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-18T08:49:35.178 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-18T08:49:35.178 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-18T08:49:35.178 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-18T08:49:35.178 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-18T08:49:35.178 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-18T08:49:35.179 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-18T08:49:35.179 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-18T08:49:35.179 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-18T08:49:35.179 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-18T08:49:35.179 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-18T08:49:35.180 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-18T08:49:35.180 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-18T08:49:35.180 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-18T08:49:35.180 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-18T08:49:35.180 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-18T08:49:35.181 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-18T08:49:35.182 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-18T08:49:35.182 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-18T08:49:35.182 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-18T08:49:35.182 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-18T08:49:35.183 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-18T08:49:35.184 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-18T08:49:35.184 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-18T08:49:35.184 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-18T08:49:35.184 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-18T08:49:35.184 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-18T08:49:35.185 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-18T08:49:35.185 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-18T08:49:35.185 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-18T08:49:35.185 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-18T08:49:35.185 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-18T08:49:35.186 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-18T08:49:35.186 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-18T08:49:35.186 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-18T08:49:35.186 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-18T08:49:35.186 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-18T08:49:35.187 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-18T08:49:35.188 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-18T08:49:35.188 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-18T08:49:35.188 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-18T08:49:35.188 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-18T08:49:35.188 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-18T08:49:35.189 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-18T08:49:35.189 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-18T08:49:35.189 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-18T08:49:35.189 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-18T08:49:35.189 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-18T08:49:35.190 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-18T08:49:35.190 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-18T08:49:35.190 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-18T08:49:35.190 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-18T08:49:35.190 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-18T08:49:35.191 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-18T08:49:35.191 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-18T08:49:35.191 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-18T08:49:35.191 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-18T08:49:35.191 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-18T08:49:35.192 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-18T08:49:35.193 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-18T08:49:35.194 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-18T08:49:35.195 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-18T08:49:35.195 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-18T08:49:35.195 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-18T08:49:35.195 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-18T08:49:35.196 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-18T08:49:35.196 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-18T08:49:35.196 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-18T08:49:35.196 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-18T08:49:35.196 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-18T08:49:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-18T08:49:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-18T08:49:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-18T08:49:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-18T08:49:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-18T08:49:35.198 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-18T08:49:35.198 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-18T08:49:35.198 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-18T08:49:35.198 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-18T08:49:35.198 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-18T08:49:35.199 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-18T08:49:35.199 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-18T08:49:35.199 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-18T08:49:35.199 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-18T08:49:35.199 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-18T08:49:35.200 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-18T08:49:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-18T08:49:35.202 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-18T08:49:35.202 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-18T08:49:35.202 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-18T08:49:35.202 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-18T08:49:35.202 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-18T08:49:35.203 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-18T08:49:35.204 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-18T08:49:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-18T08:49:35.206 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-18T08:49:35.206 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-18T08:49:35.206 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-18T08:49:35.206 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-18T08:49:35.206 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-18T08:49:35.207 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-18T08:49:35.207 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-18T08:49:35.207 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-18T08:49:35.207 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-18T08:49:35.207 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-18T08:49:35.208 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-18T08:49:35.208 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-18T08:49:35.208 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-18T08:49:35.208 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-18T08:49:35.208 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-18T08:49:35.209 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-18T08:49:35.209 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-18T08:49:35.209 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-18T08:49:35.209 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-18T08:49:35.209 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-18T08:49:35.210 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-18T08:49:35.210 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-18T08:49:35.210 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-18T08:49:35.210 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-18T08:49:35.210 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-18T08:49:35.211 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-18T08:49:35.212 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-18T08:49:35.213 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-18T08:49:35.214 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-18T08:49:35.214 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-18T08:49:35.214 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-18T08:49:35.214 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-18T08:49:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-18T08:49:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-18T08:49:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-18T08:49:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-18T08:49:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-18T08:49:35.216 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-18T08:49:35.217 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-18T08:49:35.217 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-18T08:49:35.217 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-18T08:49:35.217 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-18T08:49:35.217 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-18T08:49:35.218 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-18T08:49:35.218 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-18T08:49:35.218 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-18T08:49:35.218 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-18T08:49:35.218 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-18T08:49:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-18T08:49:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-18T08:49:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-18T08:49:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-18T08:49:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-18T08:49:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-18T08:49:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-18T08:49:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-18T08:49:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-18T08:49:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-18T08:49:35.221 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-18T08:49:35.221 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-18T08:49:35.221 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-18T08:49:35.221 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-18T08:49:35.221 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-18T08:49:35.222 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-18T08:49:35.223 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-18T08:49:35.224 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-18T08:49:35.224 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-18T08:49:35.224 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-18T08:49:35.224 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-18T08:49:35.224 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-18T08:49:35.225 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-18T08:49:35.226 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-18T08:49:35.226 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-18T08:49:35.226 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-18T08:49:35.226 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-18T08:49:35.226 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-18T08:49:35.227 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-18T08:49:35.227 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-18T08:49:35.227 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-18T08:49:35.227 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-18T08:49:35.227 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-18T08:49:35.228 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-18T08:49:35.229 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-18T08:49:35.229 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-18T08:49:35.229 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-18T08:49:35.229 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-18T08:49:35.229 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-18T08:49:35.230 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-18T08:49:35.230 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-18T08:49:35.230 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-18T08:49:35.230 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-18T08:49:35.230 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-18T08:49:35.231 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-18T08:49:35.232 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-18T08:49:35.232 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-18T08:49:35.232 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-18T08:49:35.232 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-18T08:49:35.232 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-18T08:49:35.233 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-18T08:49:35.234 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-18T08:49:35.234 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-18T08:49:35.234 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-18T08:49:35.234 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-18T08:49:35.234 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-18T08:49:35.235 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-18T08:49:35.236 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-18T08:49:35.236 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-18T08:49:35.236 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-18T08:49:35.236 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-18T08:49:35.236 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-18T08:49:35.237 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-18T08:49:35.238 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-18T08:49:35.238 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-18T08:49:35.238 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-18T08:49:35.239 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-18T08:49:35.240 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-18T08:49:35.240 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-18T08:49:35.240 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-18T08:49:35.240 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-18T08:49:35.241 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-18T08:49:35.242 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-18T08:49:35.242 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-18T08:49:35.242 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-18T08:49:35.242 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-18T08:49:35.243 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-18T08:49:35.243 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-18T08:49:35.243 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-18T08:49:35.243 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-18T08:49:35.243 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-18T08:49:35.244 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-18T08:49:35.245 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-18T08:49:35.246 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-18T08:49:35.246 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-18T08:49:35.246 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-18T08:49:35.246 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-18T08:49:35.246 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-18T08:49:35.247 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-18T08:49:35.248 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-18T08:49:35.248 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-18T08:49:35.248 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-18T08:49:35.248 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-18T08:49:35.248 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-18T08:49:35.249 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-18T08:49:35.249 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-18T08:49:35.249 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-18T08:49:35.249 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-18T08:49:35.249 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-18T08:49:35.250 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-18T08:49:35.250 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-18T08:49:35.250 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-18T08:49:35.250 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-18T08:49:35.250 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-18T08:49:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-18T08:49:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-18T08:49:35.253 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-18T08:49:35.253 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-18T08:49:35.253 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-18T08:49:35.253 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-18T08:49:35.253 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-18T08:49:35.254 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-18T08:49:35.254 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-18T08:49:35.254 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-18T08:49:35.254 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-18T08:49:35.254 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-18T08:49:35.255 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-18T08:49:35.255 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-18T08:49:35.255 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-18T08:49:35.255 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-18T08:49:35.255 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-18T08:49:35.256 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-18T08:49:35.256 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-18T08:49:35.256 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-18T08:49:35.256 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-18T08:49:35.256 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-18T08:49:35.257 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-18T08:49:35.258 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-18T08:49:35.258 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-18T08:49:35.258 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-18T08:49:35.258 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-18T08:49:35.258 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-18T08:49:35.259 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-18T08:49:35.259 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-18T08:49:35.259 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-18T08:49:35.259 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-18T08:49:35.259 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-18T08:49:35.260 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-18T08:49:35.261 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-18T08:49:35.261 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-18T08:49:35.261 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-18T08:49:35.261 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-18T08:49:35.261 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-18T08:49:35.262 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-18T08:49:35.262 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-18T08:49:35.262 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-18T08:49:35.262 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-18T08:49:35.262 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-18T08:49:35.263 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-18T08:49:35.263 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-18T08:49:35.263 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-18T08:49:35.263 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-18T08:49:35.264 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-18T08:49:35.264 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-18T08:49:35.264 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-18T08:49:35.264 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-18T08:49:35.264 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-18T08:49:35.265 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-18T08:49:35.266 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-18T08:49:35.266 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-18T08:49:35.266 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-18T08:49:35.266 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-18T08:49:35.266 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-18T08:49:35.267 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-18T08:49:35.268 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-18T08:49:35.268 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-18T08:49:35.268 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-18T08:49:35.268 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-18T08:49:35.268 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-18T08:49:35.269 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-18T08:49:35.269 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-18T08:49:35.269 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-18T08:49:35.269 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-18T08:49:35.269 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-18T08:49:35.270 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-18T08:49:35.271 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-18T08:49:35.272 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-18T08:49:35.272 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-18T08:49:35.272 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-18T08:49:35.272 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-18T08:49:35.272 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-18T08:49:35.273 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-18T08:49:35.274 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-18T08:49:35.274 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-18T08:49:35.274 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-18T08:49:35.274 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-18T08:49:35.274 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-18T08:49:35.275 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-18T08:49:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-18T08:49:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-18T08:49:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-18T08:49:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-18T08:49:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-18T08:49:35.277 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-18T08:49:35.277 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-18T08:49:35.277 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-18T08:49:35.277 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-18T08:49:35.277 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-18T08:49:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-18T08:49:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-18T08:49:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-18T08:49:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-18T08:49:35.278 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-18T08:49:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-18T08:49:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-18T08:49:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-18T08:49:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-18T08:49:35.279 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-18T08:49:35.280 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-18T08:49:35.281 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-18T08:49:35.281 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-18T08:49:35.281 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-18T08:49:35.281 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-18T08:49:35.281 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-18T08:49:35.282 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-18T08:49:35.282 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-18T08:49:35.282 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-18T08:49:35.282 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-18T08:49:35.282 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-18T08:49:35.283 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-18T08:49:35.283 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-18T08:49:35.283 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-18T08:49:35.283 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-18T08:49:35.283 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-18T08:49:35.284 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-18T08:49:35.285 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-18T08:49:35.285 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-18T08:49:35.285 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-18T08:49:35.285 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-18T08:49:35.285 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-18T08:49:35.286 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-18T08:49:35.286 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-18T08:49:35.286 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-18T08:49:35.286 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-18T08:49:35.286 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-18T08:49:35.287 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-18T08:49:35.288 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-18T08:49:35.288 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-18T08:49:35.288 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-18T08:49:35.288 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-18T08:49:35.288 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-18T08:49:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-18T08:49:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-18T08:49:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-18T08:49:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-18T08:49:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-18T08:49:35.290 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-18T08:49:35.291 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-18T08:49:35.291 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-18T08:49:35.291 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-18T08:49:35.291 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-18T08:49:35.291 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-18T08:49:35.292 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-18T08:49:35.293 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-18T08:49:35.294 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-18T08:49:35.294 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-18T08:49:35.294 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-18T08:49:35.294 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-18T08:49:35.294 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-18T08:49:35.295 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-18T08:49:35.296 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-18T08:49:35.297 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-18T08:49:35.297 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-18T08:49:35.297 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-18T08:49:35.297 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-18T08:49:35.297 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-18T08:49:35.298 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-18T08:49:35.299 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-18T08:49:35.299 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-18T08:49:35.299 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-18T08:49:35.299 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-18T08:49:35.299 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-18T08:49:35.300 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-18T08:49:35.300 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-18T08:49:35.300 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-18T08:49:35.300 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-18T08:49:35.300 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-18T08:49:35.301 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-18T08:49:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-18T08:49:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-18T08:49:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-18T08:49:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-18T08:49:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-18T08:49:35.303 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-18T08:49:35.303 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-18T08:49:35.303 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-18T08:49:35.303 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-18T08:49:35.303 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-18T08:49:35.304 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-18T08:49:35.305 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-18T08:49:35.305 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-18T08:49:35.305 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-18T08:49:35.305 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-18T08:49:35.305 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-18T08:49:35.306 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-18T08:49:35.306 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-18T08:49:35.306 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-18T08:49:35.306 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-18T08:49:35.306 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-18T08:49:35.307 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-18T08:49:35.308 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-18T08:49:35.308 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-18T08:49:35.308 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-18T08:49:35.308 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-18T08:49:35.308 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-18T08:49:35.309 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-18T08:49:35.309 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-18T08:49:35.309 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-18T08:49:35.309 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-18T08:49:35.309 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-18T08:49:35.310 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-18T08:49:35.310 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-18T08:49:35.310 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-18T08:49:35.310 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-18T08:49:35.310 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-18T08:49:35.311 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-18T08:49:35.312 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-18T08:49:35.313 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-18T08:49:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-18T08:49:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-18T08:49:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-18T08:49:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-18T08:49:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-18T08:49:35.315 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-18T08:49:35.315 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-18T08:49:35.315 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-18T08:49:35.315 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-18T08:49:35.315 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-18T08:49:35.316 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-18T08:49:35.317 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-18T08:49:35.318 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-18T08:49:35.318 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-18T08:49:35.318 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-18T08:49:35.318 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-18T08:49:35.318 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-18T08:49:35.319 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-18T08:49:35.319 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-18T08:49:35.319 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-18T08:49:35.319 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-18T08:49:35.319 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-18T08:49:35.320 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-18T08:49:35.321 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-18T08:49:35.322 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-18T08:49:35.322 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-18T08:49:35.322 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-18T08:49:35.322 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-18T08:49:35.322 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-18T08:49:35.323 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-18T08:49:35.323 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-18T08:49:35.323 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-18T08:49:35.323 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-18T08:49:35.323 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-18T08:49:35.324 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-18T08:49:35.325 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-18T08:49:35.326 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-18T08:49:35.326 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-18T08:49:35.326 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-18T08:49:35.326 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-18T08:49:35.327 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-18T08:49:35.327 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-18T08:49:35.327 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-18T08:49:35.327 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-18T08:49:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-18T08:49:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-18T08:49:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-18T08:49:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-18T08:49:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-18T08:49:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-18T08:49:35.330 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-18T08:49:35.330 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-18T08:49:35.330 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-18T08:49:35.330 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-18T08:49:35.330 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-18T08:49:35.331 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-18T08:49:35.331 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-18T08:49:35.331 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-18T08:49:35.331 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-18T08:49:35.331 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-18T08:49:35.332 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-18T08:49:35.332 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-18T08:49:35.332 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-18T08:49:35.332 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-18T08:49:35.332 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-18T08:49:35.333 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-18T08:49:35.334 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-18T08:49:35.334 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-18T08:49:35.334 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-18T08:49:35.334 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-18T08:49:35.334 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-18T08:49:35.335 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-18T08:49:35.336 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-18T08:49:35.336 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-18T08:49:35.336 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-18T08:49:35.336 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-18T08:49:35.336 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-18T08:49:35.337 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-18T08:49:35.337 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-18T08:49:35.337 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-18T08:49:35.337 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-18T08:49:35.337 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-18T08:49:35.338 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-18T08:49:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-18T08:49:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-18T08:49:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-18T08:49:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-18T08:49:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-18T08:49:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-18T08:49:35.341 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-18T08:49:35.341 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-18T08:49:35.341 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-18T08:49:35.341 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-18T08:49:35.341 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-18T08:49:35.342 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-18T08:49:35.343 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-18T08:49:35.343 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-18T08:49:35.343 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-18T08:49:35.343 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-18T08:49:35.343 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-18T08:49:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-18T08:49:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-18T08:49:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-18T08:49:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-18T08:49:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-18T08:49:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-18T08:49:35.346 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-18T08:49:35.346 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-18T08:49:35.346 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-18T08:49:35.346 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-18T08:49:35.346 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-18T08:49:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-18T08:49:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-18T08:49:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-18T08:49:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-18T08:49:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-18T08:49:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-18T08:49:35.349 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-18T08:49:35.350 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-18T08:49:35.351 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-18T08:49:35.351 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-18T08:49:35.351 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-18T08:49:35.351 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-18T08:49:35.351 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-18T08:49:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-18T08:49:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-18T08:49:35.354 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-18T08:49:35.354 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-18T08:49:35.354 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-18T08:49:35.354 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-18T08:49:35.354 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-18T08:49:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-18T08:49:35.356 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-18T08:49:35.356 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-18T08:49:35.356 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-18T08:49:35.356 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-18T08:49:35.356 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-18T08:49:35.357 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-18T08:49:35.358 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-18T08:49:35.358 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-18T08:49:35.358 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-18T08:49:35.358 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-18T08:49:35.358 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-18T08:49:35.359 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-18T08:49:35.359 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-18T08:49:35.359 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-18T08:49:35.359 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-18T08:49:35.359 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-18T08:49:35.360 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-18T08:49:35.361 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-18T08:49:35.362 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-18T08:49:35.362 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-18T08:49:35.362 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-18T08:49:35.362 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-18T08:49:35.362 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-18T08:49:35.363 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-18T08:49:35.364 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-18T08:49:35.365 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-18T08:49:35.366 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-18T08:49:35.366 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-18T08:49:35.366 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-18T08:49:35.366 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-18T08:49:35.366 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-18T08:49:35.367 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-18T08:49:35.368 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-18T08:49:35.368 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-18T08:49:35.368 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-18T08:49:35.368 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-18T08:49:35.368 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-18T08:49:35.369 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-18T08:49:35.369 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-18T08:49:35.369 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-18T08:49:35.369 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-18T08:49:35.369 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-18T08:49:35.370 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-18T08:49:35.370 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-18T08:49:35.370 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-18T08:49:35.370 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-18T08:49:35.370 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-18T08:49:35.371 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-18T08:49:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-18T08:49:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-18T08:49:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-18T08:49:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-18T08:49:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-18T08:49:35.373 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-18T08:49:35.374 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-18T08:49:35.374 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-18T08:49:35.374 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-18T08:49:35.374 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-18T08:49:35.374 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-18T08:49:35.375 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-18T08:49:35.375 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-18T08:49:35.375 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-18T08:49:35.375 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-18T08:49:35.375 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-18T08:49:35.376 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-18T08:49:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-18T08:49:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-18T08:49:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-18T08:49:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-18T08:49:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-18T08:49:35.378 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-18T08:49:35.378 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-18T08:49:35.378 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-18T08:49:35.378 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-18T08:49:35.378 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-18T08:49:35.379 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-18T08:49:35.380 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-18T08:49:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-18T08:49:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-18T08:49:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-18T08:49:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-18T08:49:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-18T08:49:35.382 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-18T08:49:35.383 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-18T08:49:35.383 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-18T08:49:35.383 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-18T08:49:35.383 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-18T08:49:35.383 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-18T08:49:35.384 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-18T08:49:35.385 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-18T08:49:35.386 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-18T08:49:35.386 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-18T08:49:35.386 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-18T08:49:35.386 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-18T08:49:35.386 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-18T08:49:35.387 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-18T08:49:35.388 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-18T08:49:35.388 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-18T08:49:35.388 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-18T08:49:35.388 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-18T08:49:35.388 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-18T08:49:35.389 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-18T08:49:35.389 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-18T08:49:35.389 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-18T08:49:35.389 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-18T08:49:35.389 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-18T08:49:35.390 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-18T08:49:35.390 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-18T08:49:35.390 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-18T08:49:35.390 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-18T08:49:35.390 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-18T08:49:35.391 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-18T08:49:35.391 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-18T08:49:35.391 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-18T08:49:35.391 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-18T08:49:35.391 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-18T08:49:35.392 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-18T08:49:35.392 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-18T08:49:35.392 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-18T08:49:35.392 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-18T08:49:35.392 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-18T08:49:35.393 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-18T08:49:35.394 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-18T08:49:35.394 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-18T08:49:35.394 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-18T08:49:35.394 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-18T08:49:35.394 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-18T08:49:35.395 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-18T08:49:35.395 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-18T08:49:35.395 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-18T08:49:35.395 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-18T08:49:35.395 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-18T08:49:35.396 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-18T08:49:35.397 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-18T08:49:35.397 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-18T08:49:35.397 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-18T08:49:35.397 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-18T08:49:35.397 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-18T08:49:35.398 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-18T08:49:35.398 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-18T08:49:35.398 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-18T08:49:35.398 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-18T08:49:35.398 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-18T08:49:35.399 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-18T08:49:35.399 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-18T08:49:35.399 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-18T08:49:35.399 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-18T08:49:35.399 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-18T08:49:35.400 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-18T08:49:35.401 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-18T08:49:35.402 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-18T08:49:35.402 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-18T08:49:35.402 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-18T08:49:35.402 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-18T08:49:35.402 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-18T08:49:35.403 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-18T08:49:35.403 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-18T08:49:35.403 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-18T08:49:35.403 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-18T08:49:35.403 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-18T08:49:35.404 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-18T08:49:35.404 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-18T08:49:35.404 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-18T08:49:35.404 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-18T08:49:35.404 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-18T08:49:35.405 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-18T08:49:35.405 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-18T08:49:35.405 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-18T08:49:35.405 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-18T08:49:35.405 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-18T08:49:35.406 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-18T08:49:35.406 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-18T08:49:35.406 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-18T08:49:35.406 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-18T08:49:35.406 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-18T08:49:35.407 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-18T08:49:35.408 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-18T08:49:35.408 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-18T08:49:35.408 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-18T08:49:35.408 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-18T08:49:35.408 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-18T08:49:35.409 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-18T08:49:35.410 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-18T08:49:35.410 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-18T08:49:35.410 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-18T08:49:35.410 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-18T08:49:35.410 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-18T08:49:35.411 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-18T08:49:35.411 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-18T08:49:35.411 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-18T08:49:35.411 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-18T08:49:35.411 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-18T08:49:35.412 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-18T08:49:35.413 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-18T08:49:35.413 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-18T08:49:35.413 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-18T08:49:35.413 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-18T08:49:35.413 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-18T08:49:35.414 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-18T08:49:35.414 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-18T08:49:35.414 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-18T08:49:35.414 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-18T08:49:35.414 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-18T08:49:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-18T08:49:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-18T08:49:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-18T08:49:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-18T08:49:35.415 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-18T08:49:35.416 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-18T08:49:35.417 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-18T08:49:35.417 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-18T08:49:35.417 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-18T08:49:35.417 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-18T08:49:35.417 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-18T08:49:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-18T08:49:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-18T08:49:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-18T08:49:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-18T08:49:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-18T08:49:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-18T08:49:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-18T08:49:35.421 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-18T08:49:35.421 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-18T08:49:35.421 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-18T08:49:35.421 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-18T08:49:35.421 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-18T08:49:35.422 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-18T08:49:35.422 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-18T08:49:35.422 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-18T08:49:35.422 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-18T08:49:35.422 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-18T08:49:35.423 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-18T08:49:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-18T08:49:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-18T08:49:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-18T08:49:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-18T08:49:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-18T08:49:35.425 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-18T08:49:35.425 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-18T08:49:35.425 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-18T08:49:35.425 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-18T08:49:35.425 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-18T08:49:35.426 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-18T08:49:35.426 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-18T08:49:35.426 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-18T08:49:35.426 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-18T08:49:35.426 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-18T08:49:35.427 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-18T08:49:35.428 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-18T08:49:35.429 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-18T08:49:35.429 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-18T08:49:35.429 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-18T08:49:35.429 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-18T08:49:35.429 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-18T08:49:35.430 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-18T08:49:35.430 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-18T08:49:35.430 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-18T08:49:35.430 INFO:teuthology.orchestra.run.smithi061.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-18T08:49:35.431 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.431 DEBUG:teuthology.orchestra.run.smithi061:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-18T08:49:35.459 INFO:teuthology.orchestra.run.smithi061.stdout:postinstall scriptlet (using /bin/sh): 2023-10-18T08:49:35.459 INFO:teuthology.orchestra.run.smithi061.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-18T08:49:35.459 INFO:teuthology.orchestra.run.smithi061.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:fi 2023-10-18T08:49:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-18T08:49:35.461 INFO:teuthology.orchestra.run.smithi061.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-18T08:49:35.461 INFO:teuthology.orchestra.run.smithi061.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-18T08:49:35.461 INFO:teuthology.orchestra.run.smithi061.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-18T08:49:35.461 INFO:teuthology.orchestra.run.smithi061.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.461 INFO:teuthology.orchestra.run.smithi061.stdout:fi 2023-10-18T08:49:35.462 INFO:teuthology.orchestra.run.smithi061.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-18T08:49:35.462 INFO:teuthology.orchestra.run.smithi061.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-18T08:49:35.462 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.462 INFO:teuthology.orchestra.run.smithi061.stdout:fi 2023-10-18T08:49:35.463 DEBUG:teuthology.orchestra.run.smithi061:> sudo rpm -qi grub2-tools 2023-10-18T08:49:35.543 INFO:teuthology.orchestra.run.smithi061.stdout:Name : grub2-tools 2023-10-18T08:49:35.543 INFO:teuthology.orchestra.run.smithi061.stdout:Epoch : 1 2023-10-18T08:49:35.543 INFO:teuthology.orchestra.run.smithi061.stdout:Version : 2.02 2023-10-18T08:49:35.543 INFO:teuthology.orchestra.run.smithi061.stdout:Release : 123.el8 2023-10-18T08:49:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:Architecture: x86_64 2023-10-18T08:49:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-18T08:49:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:Group : System Environment/Base 2023-10-18T08:49:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:Size : 9265450 2023-10-18T08:49:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:License : GPLv3+ 2023-10-18T08:49:35.545 INFO:teuthology.orchestra.run.smithi061.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-18T08:49:35.545 INFO:teuthology.orchestra.run.smithi061.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-18T08:49:35.545 INFO:teuthology.orchestra.run.smithi061.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-18T08:49:35.545 INFO:teuthology.orchestra.run.smithi061.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-18T08:49:35.545 INFO:teuthology.orchestra.run.smithi061.stdout:Relocations : (not relocatable) 2023-10-18T08:49:35.546 INFO:teuthology.orchestra.run.smithi061.stdout:Packager : Red Hat, Inc. 2023-10-18T08:49:35.546 INFO:teuthology.orchestra.run.smithi061.stdout:Vendor : Red Hat, Inc. 2023-10-18T08:49:35.546 INFO:teuthology.orchestra.run.smithi061.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-18T08:49:35.546 INFO:teuthology.orchestra.run.smithi061.stdout:Summary : Support tools for GRUB. 2023-10-18T08:49:35.546 INFO:teuthology.orchestra.run.smithi061.stdout:Description : 2023-10-18T08:49:35.547 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T08:49:35.547 INFO:teuthology.orchestra.run.smithi061.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-18T08:49:35.547 INFO:teuthology.orchestra.run.smithi061.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-18T08:49:35.547 INFO:teuthology.orchestra.run.smithi061.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-18T08:49:35.547 INFO:teuthology.orchestra.run.smithi061.stdout:hardware devices. 2023-10-18T08:49:35.548 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T08:49:35.548 INFO:teuthology.orchestra.run.smithi061.stdout:This subpackage provides tools for support of all platforms. 2023-10-18T08:49:35.549 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-18T08:49:35.549 INFO:teuthology.task.kernel:Updating grub on smithi061 to boot 6.6.0-rc5-g47672119149b 2023-10-18T08:49:35.549 DEBUG:teuthology.orchestra.run.smithi061:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-18T08:49:40.133 INFO:teuthology.orchestra.run.smithi061.stderr:Generating grub configuration file ... 2023-10-18T08:49:42.329 INFO:teuthology.orchestra.run.smithi061.stderr:done 2023-10-18T08:49:42.331 DEBUG:teuthology.orchestra.run.smithi061:> mktemp 2023-10-18T08:49:42.348 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.YM0ve4keN3 2023-10-18T08:49:42.349 DEBUG:teuthology.orchestra.run.smithi061:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YM0ve4keN3 2023-10-18T08:49:42.420 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.YM0ve4keN3 2023-10-18T08:49:42.564 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.YM0ve4keN3 is 6KB 2023-10-18T08:49:42.612 DEBUG:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.YM0ve4keN3 2023-10-18T08:49:42.629 DEBUG:teuthology.orchestra.run.smithi061:> sudo /bin/ls /boot/loader/entries || true 2023-10-18T08:49:42.712 INFO:teuthology.orchestra.run.smithi061.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-18T08:49:42.713 INFO:teuthology.orchestra.run.smithi061.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-18T08:49:42.713 INFO:teuthology.orchestra.run.smithi061.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b.conf 2023-10-18T08:49:42.714 DEBUG:teuthology.orchestra.run.smithi061:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b 2023-10-18T08:49:42.831 DEBUG:teuthology.orchestra.run.smithi061:> sudo shutdown -r now 2023-10-18T08:50:12.834 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-18T08:50:12.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T08:50:12.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:50:31.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.61 2023-10-18T08:50:40.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T08:50:40.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:50:43.226 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.61 2023-10-18T08:50:55.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T08:50:55.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:51:13.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.61 2023-10-18T08:51:28.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T08:51:28.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T08:51:29.030 DEBUG:teuthology.orchestra.run.smithi061:> true 2023-10-18T08:51:29.433 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T08:51:29.433 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-18T08:51:29.433 DEBUG:teuthology.orchestra.run.smithi061:> uname -r 2023-10-18T08:51:29.478 INFO:teuthology.orchestra.run.smithi061.stdout:6.6.0-rc5-g47672119149b 2023-10-18T08:51:29.478 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-18T08:51:29.479 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-18T08:51:29.479 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-18T08:51:30.480 DEBUG:teuthology.parallel:result is None 2023-10-18T08:51:30.481 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-18T08:51:30.489 INFO:teuthology.task.internal:Creating test directory... 2023-10-18T08:51:30.489 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T08:51:30.494 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T08:51:30.497 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-18T08:51:30.535 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-18T08:51:30.542 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-18T08:51:30.549 INFO:teuthology.task.internal:Creating archive directory... 2023-10-18T08:51:30.549 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T08:51:30.580 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T08:51:30.583 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-18T08:51:30.638 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-18T08:51:30.645 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-18T08:51:30.645 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T08:51:30.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:51:30.665 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-18T08:51:30.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:51:30.684 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T08:51:30.689 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T08:51:30.711 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-18T08:51:30.768 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.782 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.787 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.802 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.854 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.887 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-18T08:51:30.891 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-18T08:51:30.899 INFO:teuthology.task.internal:Configuring sudo... 2023-10-18T08:51:30.899 DEBUG:teuthology.orchestra.run.smithi061:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T08:51:30.903 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T08:51:30.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-18T08:51:30.972 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-18T08:51:30.981 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-18T08:51:30.982 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T08:51:30.987 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T08:51:30.989 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-18T08:51:31.026 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.145 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.224 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.294 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.364 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T08:51:31.364 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T08:51:31.443 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.490 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.548 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.678 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T08:51:31.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T08:51:31.747 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-18T08:51:31.846 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-18T08:51:31.975 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T08:51:31.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-18T08:51:32.043 DEBUG:teuthology.orchestra.run.smithi061:> sudo service rsyslog restart 2023-10-18T08:51:32.048 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2023-10-18T08:51:32.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-10-18T08:51:32.123 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T08:51:32.138 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T08:51:32.181 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T08:51:32.476 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-18T08:51:32.484 INFO:teuthology.task.internal:Starting timer... 2023-10-18T08:51:32.484 INFO:teuthology.run_tasks:Running task pcp... 2023-10-18T08:51:32.505 INFO:teuthology.run_tasks:Running task selinux... 2023-10-18T08:51:32.514 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-18T08:51:32.514 DEBUG:teuthology.orchestra.run.smithi061:> sudo service auditd rotate 2023-10-18T08:51:32.633 INFO:teuthology.orchestra.run.smithi061.stdout:Rotating logs: 2023-10-18T08:51:32.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2023-10-18T08:51:32.692 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2023-10-18T08:51:32.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2023-10-18T08:51:32.773 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2023-10-18T08:51:32.775 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-18T08:51:32.775 DEBUG:teuthology.orchestra.run.smithi061:> /usr/sbin/getenforce 2023-10-18T08:51:32.821 INFO:teuthology.orchestra.run.smithi061.stdout:Permissive 2023-10-18T08:51:32.822 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2023-10-18T08:51:32.850 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2023-10-18T08:51:32.851 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2023-10-18T08:51:32.879 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2023-10-18T08:51:32.879 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi061.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2023-10-18T08:51:32.879 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T08:51:32.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:51:32.951 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T08:51:32.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:51:32.979 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T08:51:33.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T08:51:33.007 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-18T08:51:33.007 DEBUG:teuthology.orchestra.run.smithi061:> sudo /usr/sbin/setenforce permissive 2023-10-18T08:51:33.076 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2023-10-18T08:51:33.103 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2023-10-18T08:51:33.131 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-18T08:51:33.141 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-18T08:51:33.147 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-18T08:51:33.303 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-18T08:51:33.320 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-18T08:51:33.321 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi061.front.sepia.ceph.com,smithi110.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-18T09:04:19.995 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi061.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2023-10-18T09:04:19.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T09:04:19.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T09:04:20.089 DEBUG:teuthology.orchestra.run.smithi061:> true 2023-10-18T09:04:20.202 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-10-18T09:04:20.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-10-18T09:04:20.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T09:04:20.276 DEBUG:teuthology.orchestra.run.smithi110:> true 2023-10-18T09:04:20.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2023-10-18T09:04:20.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-18T09:04:20.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-18T09:04:20.421 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-10-18T09:04:20.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-18T09:04:20.489 INFO:teuthology.run_tasks:Running task clock... 2023-10-18T09:04:20.506 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-18T09:04:20.506 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T09:04:20.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:04:20.512 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T09:04:20.513 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:04:20.515 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-18T09:04:20.515 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:04:20.550 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T09:04:20.563 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T09:04:20.566 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T09:04:20.579 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T09:04:20.585 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-18T09:04:20.621 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-18T09:04:20.639 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2023-10-18T09:04:20.651 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2023-10-18T09:04:20.655 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2023-10-18T09:04:20.666 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T09:04:20.667 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2023-10-18T09:04:20.681 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T09:04:20.682 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T09:04:20.696 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T09:04:20.721 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: ntpd: command not found 2023-10-18T09:04:20.752 INFO:teuthology.orchestra.run.smithi061.stdout:506 Cannot talk to daemon 2023-10-18T09:04:20.755 INFO:teuthology.orchestra.run.smithi110.stderr:bash: ntpq: command not found 2023-10-18T09:04:20.759 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:04:20.759 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2023-10-18T09:04:20.760 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.760 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.760 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.760 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-10-18T09:04:20.779 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:04:20.779 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-18T09:04:20.780 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.780 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.780 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.780 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.787 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-18T09:04:20.822 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-18T09:04:20.932 INFO:teuthology.orchestra.run.smithi061.stderr:bash: ntpq: command not found 2023-10-18T09:04:20.940 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:04:20.941 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-10-18T09:04:20.941 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.941 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.941 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.941 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:04:20.942 INFO:teuthology.run_tasks:Running task pexec... 2023-10-18T09:04:20.953 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-18T09:04:20.954 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:04:20.954 DEBUG:teuthology.orchestra.run.smithi061:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T09:04:20.955 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi110.front.sepia.ceph.com 2023-10-18T09:04:20.955 DEBUG:teuthology.orchestra.run.smithi110:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T09:04:20.956 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2023-10-18T09:04:20.956 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-18T09:04:21.326 INFO:teuthology.orchestra.run.smithi110.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:21.331 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:21.470 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:22.294 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:31 ago on Wed 18 Oct 2023 08:59:50 AM UTC. 2023-10-18T09:04:22.340 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:04:33 ago on Wed 18 Oct 2023 08:59:48 AM UTC. 2023-10-18T09:04:22.743 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:49 ago on Wed 18 Oct 2023 09:02:33 AM UTC. 2023-10-18T09:04:24.086 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-10-18T09:04:24.087 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-10-18T09:04:24.087 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2023-10-18T09:04:24.087 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-10-18T09:04:24.087 INFO:teuthology.orchestra.run.smithi110.stdout:Resetting modules: 2023-10-18T09:04:24.088 INFO:teuthology.orchestra.run.smithi110.stdout: container-tools 2023-10-18T09:04:24.088 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:24.088 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-10-18T09:04:24.088 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2023-10-18T09:04:24.088 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:24.180 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-18T09:04:24.181 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-18T09:04:24.181 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2023-10-18T09:04:24.182 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-18T09:04:24.182 INFO:teuthology.orchestra.run.smithi170.stdout:Resetting modules: 2023-10-18T09:04:24.182 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools 2023-10-18T09:04:24.182 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:24.182 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-18T09:04:24.183 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2023-10-18T09:04:24.183 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:24.747 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-10-18T09:04:24.748 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-10-18T09:04:24.748 INFO:teuthology.orchestra.run.smithi061.stdout: Package Architecture Version Repository Size 2023-10-18T09:04:24.749 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-10-18T09:04:24.749 INFO:teuthology.orchestra.run.smithi061.stdout:Resetting modules: 2023-10-18T09:04:24.749 INFO:teuthology.orchestra.run.smithi061.stdout: container-tools 2023-10-18T09:04:24.749 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:24.749 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-10-18T09:04:24.750 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-10-18T09:04:24.750 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:25.041 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:04:36 ago on Wed 18 Oct 2023 08:59:48 AM UTC. 2023-10-18T09:04:25.135 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:34 ago on Wed 18 Oct 2023 08:59:50 AM UTC. 2023-10-18T09:04:26.043 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:52 ago on Wed 18 Oct 2023 09:02:33 AM UTC. 2023-10-18T09:04:28.059 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-18T09:04:28.201 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-10-18T09:04:28.533 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:28.662 INFO:teuthology.orchestra.run.smithi110.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:29.054 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-10-18T09:04:29.419 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:38 ago on Wed 18 Oct 2023 08:59:50 AM UTC. 2023-10-18T09:04:29.578 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:04:41 ago on Wed 18 Oct 2023 08:59:48 AM UTC. 2023-10-18T09:04:29.686 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:31.132 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-18T09:04:31.227 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-10-18T09:04:31.227 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-10-18T09:04:31.228 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-10-18T09:04:31.228 INFO:teuthology.orchestra.run.smithi170.stdout:Installing group/module packages: 2023-10-18T09:04:31.228 INFO:teuthology.orchestra.run.smithi170.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T09:04:31.228 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T09:04:31.229 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-10-18T09:04:31.230 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T09:04:31.230 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T09:04:31.230 INFO:teuthology.orchestra.run.smithi170.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T09:04:31.230 INFO:teuthology.orchestra.run.smithi170.stdout:Installing module profiles: 2023-10-18T09:04:31.230 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools/common 2023-10-18T09:04:31.231 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:31.231 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-10-18T09:04:31.231 INFO:teuthology.orchestra.run.smithi170.stdout: container-tools rhel8 2023-10-18T09:04:31.231 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:31.231 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-18T09:04:31.232 INFO:teuthology.orchestra.run.smithi170.stdout:============================================================================================================ 2023-10-18T09:04:31.232 INFO:teuthology.orchestra.run.smithi170.stdout:Install 10 Packages 2023-10-18T09:04:31.232 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:31.233 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 18 M 2023-10-18T09:04:31.233 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 65 M 2023-10-18T09:04:31.233 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-10-18T09:04:31.370 INFO:teuthology.orchestra.run.smithi170.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 668 kB/s | 94 kB 00:00 2023-10-18T09:04:31.395 INFO:teuthology.orchestra.run.smithi170.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 246 kB/s | 41 kB 00:00 2023-10-18T09:04:31.421 INFO:teuthology.orchestra.run.smithi170.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 130 kB/s | 25 kB 00:00 2023-10-18T09:04:31.436 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-10-18T09:04:31.501 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:57 ago on Wed 18 Oct 2023 09:02:33 AM UTC. 2023-10-18T09:04:31.530 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================================================ 2023-10-18T09:04:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-10-18T09:04:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================================================ 2023-10-18T09:04:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:Installing group/module packages: 2023-10-18T09:04:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T09:04:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T09:04:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T09:04:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T09:04:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T09:04:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T09:04:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T09:04:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-10-18T09:04:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T09:04:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T09:04:31.533 INFO:teuthology.orchestra.run.smithi110.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T09:04:31.533 INFO:teuthology.orchestra.run.smithi110.stdout:Installing module profiles: 2023-10-18T09:04:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: container-tools/common 2023-10-18T09:04:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:31.534 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling module streams: 2023-10-18T09:04:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: container-tools rhel8 2023-10-18T09:04:31.534 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-10-18T09:04:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:============================================================================================================ 2023-10-18T09:04:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:Install 10 Packages 2023-10-18T09:04:31.535 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 18 M 2023-10-18T09:04:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 65 M 2023-10-18T09:04:31.536 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-10-18T09:04:31.621 INFO:teuthology.orchestra.run.smithi170.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 197 kB/s | 49 kB 00:00 2023-10-18T09:04:31.704 INFO:teuthology.orchestra.run.smithi110.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 145 kB/s | 25 kB 00:00 2023-10-18T09:04:31.746 INFO:teuthology.orchestra.run.smithi170.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 1.5 MB/s | 524 kB 00:00 2023-10-18T09:04:31.754 INFO:teuthology.orchestra.run.smithi110.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 184 kB/s | 41 kB 00:00 2023-10-18T09:04:31.772 INFO:teuthology.orchestra.run.smithi170.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 995 kB/s | 149 kB 00:00 2023-10-18T09:04:31.813 INFO:teuthology.orchestra.run.smithi110.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 335 kB/s | 94 kB 00:00 2023-10-18T09:04:31.838 INFO:teuthology.orchestra.run.smithi110.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 369 kB/s | 49 kB 00:00 2023-10-18T09:04:31.847 INFO:teuthology.orchestra.run.smithi170.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 2.1 MB/s | 213 kB 00:00 2023-10-18T09:04:31.897 INFO:teuthology.orchestra.run.smithi110.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.6 MB/s | 524 kB 00:00 2023-10-18T09:04:31.964 INFO:teuthology.orchestra.run.smithi170.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 15 MB/s | 8.0 MB 00:00 2023-10-18T09:04:31.997 INFO:teuthology.orchestra.run.smithi110.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 941 kB/s | 149 kB 00:00 2023-10-18T09:04:32.039 INFO:teuthology.orchestra.run.smithi170.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-10-18T09:04:32.047 INFO:teuthology.orchestra.run.smithi110.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.4 MB/s | 213 kB 00:00 2023-10-18T09:04:32.139 INFO:teuthology.orchestra.run.smithi110.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-10-18T09:04:32.163 INFO:teuthology.orchestra.run.smithi170.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 18 MB/s | 7.0 MB 00:00 2023-10-18T09:04:32.163 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:04:32.164 INFO:teuthology.orchestra.run.smithi170.stdout:Total 20 MB/s | 18 MB 00:00 2023-10-18T09:04:32.206 INFO:teuthology.orchestra.run.smithi110.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 14 MB/s | 2.2 MB 00:00 2023-10-18T09:04:32.315 INFO:teuthology.orchestra.run.smithi110.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 22 MB/s | 7.0 MB 00:00 2023-10-18T09:04:32.315 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:04:32.316 INFO:teuthology.orchestra.run.smithi110.stdout:Total 23 MB/s | 18 MB 00:00 2023-10-18T09:04:32.362 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-10-18T09:04:32.381 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-10-18T09:04:32.381 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-10-18T09:04:32.514 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-10-18T09:04:32.533 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-10-18T09:04:32.533 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-10-18T09:04:32.563 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-10-18T09:04:32.567 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-10-18T09:04:32.717 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-10-18T09:04:32.720 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-10-18T09:04:33.146 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-10-18T09:04:33.249 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-10-18T09:04:33.354 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T09:04:33.429 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T09:04:33.512 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-10-18T09:04:33.523 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:33.597 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:33.616 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================================ 2023-10-18T09:04:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-10-18T09:04:33.616 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================================ 2023-10-18T09:04:33.616 INFO:teuthology.orchestra.run.smithi061.stdout:Installing group/module packages: 2023-10-18T09:04:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-18T09:04:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-18T09:04:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-18T09:04:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-18T09:04:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-18T09:04:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-18T09:04:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-18T09:04:33.618 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-10-18T09:04:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-18T09:04:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout:Installing module profiles: 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: container-tools/common 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling module streams: 2023-10-18T09:04:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: container-tools rhel8 2023-10-18T09:04:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:33.620 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-10-18T09:04:33.620 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================================ 2023-10-18T09:04:33.620 INFO:teuthology.orchestra.run.smithi061.stdout:Install 10 Packages 2023-10-18T09:04:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:33.621 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 18 M 2023-10-18T09:04:33.621 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 65 M 2023-10-18T09:04:33.622 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-10-18T09:04:33.709 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T09:04:33.763 INFO:teuthology.orchestra.run.smithi061.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 173 kB/s | 25 kB 00:00 2023-10-18T09:04:33.783 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T09:04:33.788 INFO:teuthology.orchestra.run.smithi061.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 241 kB/s | 41 kB 00:00 2023-10-18T09:04:33.830 INFO:teuthology.orchestra.run.smithi061.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 445 kB/s | 94 kB 00:00 2023-10-18T09:04:33.906 INFO:teuthology.orchestra.run.smithi061.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 4.4 MB/s | 524 kB 00:00 2023-10-18T09:04:33.908 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T09:04:33.931 INFO:teuthology.orchestra.run.smithi061.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 295 kB/s | 49 kB 00:00 2023-10-18T09:04:34.009 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T09:04:34.023 INFO:teuthology.orchestra.run.smithi061.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.3 MB/s | 149 kB 00:00 2023-10-18T09:04:34.056 INFO:teuthology.orchestra.run.smithi061.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.7 MB/s | 213 kB 00:00 2023-10-18T09:04:34.199 INFO:teuthology.orchestra.run.smithi061.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 22 MB/s | 8.0 MB 00:00 2023-10-18T09:04:34.278 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T09:04:34.315 INFO:teuthology.orchestra.run.smithi061.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 8.5 MB/s | 2.2 MB 00:00 2023-10-18T09:04:34.418 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T09:04:34.441 INFO:teuthology.orchestra.run.smithi061.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-10-18T09:04:34.441 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:04:34.442 INFO:teuthology.orchestra.run.smithi061.stdout:Total 22 MB/s | 18 MB 00:00 2023-10-18T09:04:34.789 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-10-18T09:04:34.813 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-10-18T09:04:34.814 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-10-18T09:04:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T09:04:35.192 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T09:04:35.348 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-10-18T09:04:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-10-18T09:04:35.935 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T09:04:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-10-18T09:04:36.080 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T09:04:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-18T09:04:36.196 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T09:04:36.313 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T09:04:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:36.439 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T09:04:36.521 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T09:04:36.591 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-18T09:04:36.633 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:36.723 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-18T09:04:36.903 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T09:04:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T09:04:36.904 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T09:04:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T09:04:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T09:04:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T09:04:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T09:04:36.905 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T09:04:36.967 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:36.967 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T09:04:36.967 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:36.967 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T09:04:36.968 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T09:04:36.968 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T09:04:36.968 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T09:04:36.968 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T09:04:36.968 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T09:04:36.969 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T09:04:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-18T09:04:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-18T09:04:38.069 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T09:04:38.069 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:04:47 ago on Wed 18 Oct 2023 08:59:50 AM UTC. 2023-10-18T09:04:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T09:04:38.147 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:04:49 ago on Wed 18 Oct 2023 08:59:48 AM UTC. 2023-10-18T09:04:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-18T09:04:39.258 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-18T09:04:39.435 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-18T09:04:39.694 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:40.088 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-18T09:04:40.088 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-18T09:04:40.089 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-18T09:04:40.089 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-18T09:04:40.089 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-18T09:04:40.089 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-18T09:04:40.089 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-18T09:04:40.090 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-18T09:04:40.090 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-18T09:04:40.090 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-18T09:04:41.572 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-18T09:04:41.573 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:02:07 ago on Wed 18 Oct 2023 09:02:33 AM UTC. 2023-10-18T09:04:43.378 INFO:teuthology.orchestra.run.smithi110.stdout:Installed products updated. 2023-10-18T09:04:43.589 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:43.590 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-10-18T09:04:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:43.590 INFO:teuthology.orchestra.run.smithi110.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T09:04:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T09:04:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T09:04:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:43.591 INFO:teuthology.orchestra.run.smithi110.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T09:04:43.592 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:04:43.592 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-10-18T09:04:43.773 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:43.842 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-10-18T09:04:44.071 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:44.072 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-10-18T09:04:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T09:04:44.074 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:04:44.074 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-18T09:04:44.246 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:49.077 INFO:teuthology.orchestra.run.smithi061.stdout:Installed products updated. 2023-10-18T09:04:49.296 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:49.296 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-10-18T09:04:49.297 INFO:teuthology.orchestra.run.smithi061.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:49.297 INFO:teuthology.orchestra.run.smithi061.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:49.297 INFO:teuthology.orchestra.run.smithi061.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:49.297 INFO:teuthology.orchestra.run.smithi061.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-18T09:04:49.298 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-18T09:04:49.298 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-18T09:04:49.298 INFO:teuthology.orchestra.run.smithi061.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:49.298 INFO:teuthology.orchestra.run.smithi061.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-18T09:04:49.299 INFO:teuthology.orchestra.run.smithi061.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-18T09:04:49.299 INFO:teuthology.orchestra.run.smithi061.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-18T09:04:49.299 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:04:49.299 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-10-18T09:04:49.508 DEBUG:teuthology.parallel:result is None 2023-10-18T09:04:49.508 INFO:teuthology.run_tasks:Running task install... 2023-10-18T09:04:49.517 DEBUG:teuthology.task.install:project ceph 2023-10-18T09:04:49.518 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'}} 2023-10-18T09:04:49.518 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb'} 2023-10-18T09:04:49.518 INFO:teuthology.task.install:Using flavor: default 2023-10-18T09:04:49.524 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-18T09:04:49.525 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-18T09:04:49.525 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T09:04:49.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:04:49.531 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T09:04:49.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:04:49.532 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'tag': None, 'wait_for_package': False} 2023-10-18T09:04:49.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:04:49.674 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T09:04:49.675 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T09:04:49.773 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T09:04:49.773 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T09:04:49.777 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/ 2023-10-18T09:04:49.778 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6786.gf7420d7f 2023-10-18T09:04:49.881 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T09:04:49.882 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:04:49.882 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T09:04:49.885 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T09:04:49.886 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:04:49.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T09:04:49.909 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T09:04:49.909 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T09:04:49.958 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-18T09:04:49.958 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:04:49.959 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-18T09:04:49.960 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T09:04:49.960 DEBUG:teuthology.orchestra.run.smithi061:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T09:04:49.976 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T09:04:49.986 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-18T09:04:49.987 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f7420d7f8e55ceea786dcbf536c338ce920c21eb/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-18T09:04:50.054 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T09:04:50.055 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T09:04:50.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-18T09:04:50.122 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2023-10-18T09:04:50.124 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2023-10-18T09:04:50.131 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T09:04:50.171 DEBUG:teuthology.orchestra.run.smithi061:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-18T09:04:50.196 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2023-10-18T09:04:50.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2023-10-18T09:04:50.240 INFO:teuthology.orchestra.run.smithi061.stdout:check_obsoletes = 1 2023-10-18T09:04:50.243 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum clean all 2023-10-18T09:04:50.508 INFO:teuthology.orchestra.run.smithi110.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:50.593 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:50.725 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:50.791 INFO:teuthology.orchestra.run.smithi110.stdout:44 files removed 2023-10-18T09:04:50.839 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T09:04:50.874 INFO:teuthology.orchestra.run.smithi170.stdout:44 files removed 2023-10-18T09:04:50.924 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T09:04:51.052 INFO:teuthology.orchestra.run.smithi061.stdout:44 files removed 2023-10-18T09:04:51.108 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-18T09:04:51.186 INFO:teuthology.orchestra.run.smithi110.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:51.272 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:51.590 INFO:teuthology.orchestra.run.smithi061.stdout:Updating Subscription Management repositories. 2023-10-18T09:04:51.979 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 227 kB/s | 81 kB 00:00 2023-10-18T09:04:52.001 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 260 kB/s | 81 kB 00:00 2023-10-18T09:04:52.304 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2023-10-18T09:04:52.352 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-10-18T09:04:52.460 INFO:teuthology.orchestra.run.smithi061.stdout:ceph packages for x86_64 217 kB/s | 81 kB 00:00 2023-10-18T09:04:52.644 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-10-18T09:04:52.663 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-10-18T09:04:52.778 INFO:teuthology.orchestra.run.smithi061.stdout:ceph noarch packages 34 kB/s | 9.4 kB 00:00 2023-10-18T09:04:52.806 INFO:teuthology.orchestra.run.smithi110.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-18T09:04:52.828 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-10-18T09:04:53.103 INFO:teuthology.orchestra.run.smithi061.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-10-18T09:04:53.271 INFO:teuthology.orchestra.run.smithi061.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-18T09:04:54.006 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2023-10-18T09:04:54.673 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2023-10-18T09:04:57.540 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-10-18T09:04:58.179 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 393 kB/s | 24 kB 00:00 2023-10-18T09:04:58.426 INFO:teuthology.orchestra.run.smithi110.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-10-18T09:04:59.027 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 16 MB 00:05 2023-10-18T09:04:59.123 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-10-18T09:05:03.405 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 278 kB/s | 24 kB 00:00 2023-10-18T09:05:04.523 INFO:teuthology.orchestra.run.smithi061.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 47 MB 00:01 2023-10-18T09:05:06.828 INFO:teuthology.orchestra.run.smithi110.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 39 MB/s | 53 MB 00:01 2023-10-18T09:05:07.315 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-10-18T09:05:13.658 INFO:teuthology.orchestra.run.smithi061.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-10-18T09:05:17.851 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:17.852 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:17.867 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:17.868 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:17.870 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T09:05:17.871 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T09:05:17.873 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T09:05:17.876 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T09:05:17.878 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T09:05:18.009 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2023-10-18T09:05:18.021 INFO:teuthology.orchestra.run.smithi110.stdout:=========================================================================================================================== 2023-10-18T09:05:18.021 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2023-10-18T09:05:18.022 INFO:teuthology.orchestra.run.smithi110.stdout:=========================================================================================================================== 2023-10-18T09:05:18.022 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2023-10-18T09:05:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T09:05:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T09:05:18.022 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T09:05:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T09:05:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T09:05:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T09:05:18.023 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T09:05:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T09:05:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T09:05:18.025 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T09:05:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T09:05:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T09:05:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T09:05:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T09:05:18.026 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T09:05:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T09:05:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T09:05:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T09:05:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T09:05:18.027 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T09:05:18.028 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T09:05:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T09:05:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T09:05:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T09:05:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T09:05:18.029 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T09:05:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T09:05:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T09:05:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T09:05:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T09:05:18.030 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T09:05:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T09:05:18.031 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2023-10-18T09:05:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T09:05:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:18.031 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:18.032 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2023-10-18T09:05:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T09:05:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T09:05:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T09:05:18.032 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T09:05:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T09:05:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T09:05:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T09:05:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T09:05:18.033 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T09:05:18.034 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T09:05:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T09:05:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T09:05:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T09:05:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T09:05:18.035 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T09:05:18.036 INFO:teuthology.orchestra.run.smithi110.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T09:05:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T09:05:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T09:05:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T09:05:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T09:05:18.037 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T09:05:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T09:05:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T09:05:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T09:05:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T09:05:18.038 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T09:05:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T09:05:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T09:05:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T09:05:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T09:05:18.039 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T09:05:18.040 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T09:05:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T09:05:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T09:05:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T09:05:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T09:05:18.041 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T09:05:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T09:05:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T09:05:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T09:05:18.042 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T09:05:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T09:05:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T09:05:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T09:05:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T09:05:18.043 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T09:05:18.044 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T09:05:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T09:05:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T09:05:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T09:05:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T09:05:18.045 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T09:05:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T09:05:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T09:05:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T09:05:18.046 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T09:05:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T09:05:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T09:05:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T09:05:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T09:05:18.047 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T09:05:18.048 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T09:05:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T09:05:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T09:05:18.049 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling module streams: 2023-10-18T09:05:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql 10 2023-10-18T09:05:18.049 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:05:18.050 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2023-10-18T09:05:18.050 INFO:teuthology.orchestra.run.smithi110.stdout:=========================================================================================================================== 2023-10-18T09:05:18.050 INFO:teuthology.orchestra.run.smithi110.stdout:Install 128 Packages 2023-10-18T09:05:18.050 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 3 Packages 2023-10-18T09:05:18.050 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:05:18.051 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 209 M 2023-10-18T09:05:18.051 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2023-10-18T09:05:18.229 INFO:teuthology.orchestra.run.smithi110.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-10-18T09:05:18.390 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:18.391 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:18.405 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:18.406 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:18.408 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T09:05:18.409 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T09:05:18.411 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T09:05:18.414 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T09:05:18.415 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T09:05:18.550 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-18T09:05:18.562 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-10-18T09:05:18.562 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-10-18T09:05:18.562 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-10-18T09:05:18.563 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2023-10-18T09:05:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T09:05:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T09:05:18.563 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T09:05:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T09:05:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T09:05:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T09:05:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T09:05:18.564 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T09:05:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T09:05:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T09:05:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T09:05:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T09:05:18.565 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T09:05:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T09:05:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T09:05:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T09:05:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T09:05:18.566 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T09:05:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T09:05:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T09:05:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T09:05:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T09:05:18.567 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T09:05:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T09:05:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T09:05:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T09:05:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T09:05:18.568 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T09:05:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T09:05:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T09:05:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T09:05:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T09:05:18.569 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T09:05:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T09:05:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T09:05:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T09:05:18.570 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T09:05:18.570 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-10-18T09:05:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T09:05:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:18.571 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-10-18T09:05:18.571 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T09:05:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T09:05:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T09:05:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T09:05:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T09:05:18.572 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T09:05:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T09:05:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T09:05:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T09:05:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T09:05:18.573 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T09:05:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T09:05:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T09:05:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T09:05:18.574 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T09:05:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T09:05:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T09:05:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T09:05:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T09:05:18.575 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T09:05:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T09:05:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T09:05:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T09:05:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T09:05:18.576 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T09:05:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T09:05:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T09:05:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T09:05:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T09:05:18.577 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T09:05:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T09:05:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T09:05:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T09:05:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T09:05:18.578 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T09:05:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T09:05:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T09:05:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T09:05:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T09:05:18.579 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T09:05:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T09:05:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T09:05:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T09:05:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T09:05:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T09:05:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T09:05:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T09:05:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T09:05:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T09:05:18.581 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T09:05:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T09:05:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T09:05:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T09:05:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:18.582 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T09:05:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T09:05:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T09:05:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T09:05:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T09:05:18.583 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T09:05:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T09:05:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T09:05:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T09:05:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T09:05:18.584 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T09:05:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T09:05:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T09:05:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T09:05:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T09:05:18.585 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T09:05:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T09:05:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T09:05:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T09:05:18.586 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T09:05:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T09:05:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T09:05:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T09:05:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T09:05:18.587 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T09:05:18.588 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T09:05:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T09:05:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T09:05:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T09:05:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T09:05:18.589 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T09:05:18.590 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2023-10-18T09:05:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql 10 2023-10-18T09:05:18.590 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:05:18.590 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-18T09:05:18.590 INFO:teuthology.orchestra.run.smithi170.stdout:=========================================================================================================================== 2023-10-18T09:05:18.591 INFO:teuthology.orchestra.run.smithi170.stdout:Install 128 Packages 2023-10-18T09:05:18.591 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 3 Packages 2023-10-18T09:05:18.591 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:05:18.592 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 209 M 2023-10-18T09:05:18.592 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-10-18T09:05:18.645 INFO:teuthology.orchestra.run.smithi110.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 2.2 MB/s | 919 kB 00:00 2023-10-18T09:05:18.720 INFO:teuthology.orchestra.run.smithi110.stdout:(3/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x8 8.8 MB/s | 5.4 MB 00:00 2023-10-18T09:05:18.746 INFO:teuthology.orchestra.run.smithi110.stdout:(4/131): ceph-immutable-object-cache-18.0.0-678 1.6 MB/s | 167 kB 00:00 2023-10-18T09:05:18.962 INFO:teuthology.orchestra.run.smithi110.stdout:(5/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 9.6 MB/s | 2.3 MB 00:00 2023-10-18T09:05:19.108 INFO:teuthology.orchestra.run.smithi170.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-10-18T09:05:19.388 INFO:teuthology.orchestra.run.smithi110.stdout:(6/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 2.5 MB/s | 1.6 MB 00:00 2023-10-18T09:05:19.463 INFO:teuthology.orchestra.run.smithi110.stdout:(7/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 9.2 MB/s | 4.6 MB 00:00 2023-10-18T09:05:19.951 INFO:teuthology.orchestra.run.smithi170.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.1 MB/s | 919 kB 00:00 2023-10-18T09:05:19.963 INFO:teuthology.orchestra.run.smithi110.stdout:(8/131): ceph-common-18.0.0-6786.gf7420d7f.el8. 12 MB/s | 23 MB 00:01 2023-10-18T09:05:20.005 INFO:teuthology.orchestra.run.smithi110.stdout:(9/131): ceph-selinux-18.0.0-6786.gf7420d7f.el8 589 kB/s | 24 kB 00:00 2023-10-18T09:05:20.069 INFO:teuthology.orchestra.run.smithi170.stdout:(3/131): ceph-immutable-object-cache-18.0.0-678 1.4 MB/s | 167 kB 00:00 2023-10-18T09:05:20.664 INFO:teuthology.orchestra.run.smithi110.stdout:(10/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el 12 MB/s | 14 MB 00:01 2023-10-18T09:05:20.748 INFO:teuthology.orchestra.run.smithi110.stdout:(11/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 3.1 MB/s | 261 kB 00:00 2023-10-18T09:05:20.831 INFO:teuthology.orchestra.run.smithi110.stdout:(12/131): libcephfs-devel-18.0.0-6786.gf7420d7f 373 kB/s | 31 kB 00:00 2023-10-18T09:05:21.218 INFO:teuthology.orchestra.run.smithi170.stdout:(4/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-10-18T09:05:21.709 INFO:teuthology.orchestra.run.smithi170.stdout:(5/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 3.3 MB/s | 1.6 MB 00:00 2023-10-18T09:05:21.923 INFO:teuthology.orchestra.run.smithi110.stdout:(13/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 770 kB/s | 840 kB 00:01 2023-10-18T09:05:22.290 INFO:teuthology.orchestra.run.smithi110.stdout:(14/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 504 kB/s | 185 kB 00:00 2023-10-18T09:05:22.474 INFO:teuthology.orchestra.run.smithi110.stdout:(15/131): librados-devel-18.0.0-6786.gf7420d7f. 759 kB/s | 139 kB 00:00 2023-10-18T09:05:22.743 INFO:teuthology.orchestra.run.smithi170.stdout:(6/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 4.5 MB/s | 4.6 MB 00:01 2023-10-18T09:05:22.816 INFO:teuthology.orchestra.run.smithi110.stdout:(16/131): libradosstriper1-18.0.0-6786.gf7420d7 1.5 MB/s | 518 kB 00:00 2023-10-18T09:05:25.110 INFO:teuthology.orchestra.run.smithi170.stdout:(7/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x86 7.6 MB/s | 18 MB 00:02 2023-10-18T09:05:25.383 INFO:teuthology.orchestra.run.smithi110.stdout:(17/131): ceph-test-18.0.0-6786.gf7420d7f.el8.x 9.1 MB/s | 49 MB 00:05 2023-10-18T09:05:25.466 INFO:teuthology.orchestra.run.smithi110.stdout:(18/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 2.6 MB/s | 6.9 MB 00:02 2023-10-18T09:05:25.492 INFO:teuthology.orchestra.run.smithi110.stdout:(19/131): python3-ceph-argparse-18.0.0-6786.gf7 413 kB/s | 45 kB 00:00 2023-10-18T09:05:25.592 INFO:teuthology.orchestra.run.smithi110.stdout:(20/131): python3-ceph-common-18.0.0-6786.gf742 1.0 MB/s | 125 kB 00:00 2023-10-18T09:05:25.617 INFO:teuthology.orchestra.run.smithi110.stdout:(21/131): python3-cephfs-18.0.0-6786.gf7420d7f. 1.5 MB/s | 196 kB 00:00 2023-10-18T09:05:25.742 INFO:teuthology.orchestra.run.smithi110.stdout:(22/131): python3-rados-18.0.0-6786.gf7420d7f.e 2.4 MB/s | 370 kB 00:00 2023-10-18T09:05:25.768 INFO:teuthology.orchestra.run.smithi110.stdout:(23/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 2.4 MB/s | 364 kB 00:00 2023-10-18T09:05:25.826 INFO:teuthology.orchestra.run.smithi110.stdout:(24/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.2 MB/s | 107 kB 00:00 2023-10-18T09:05:25.852 INFO:teuthology.orchestra.run.smithi110.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.1 MB/s | 94 kB 00:00 2023-10-18T09:05:25.943 INFO:teuthology.orchestra.run.smithi110.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 1.9 MB/s | 178 kB 00:00 2023-10-18T09:05:26.036 INFO:teuthology.orchestra.run.smithi110.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 251 kB/s | 23 kB 00:00 2023-10-18T09:05:26.119 INFO:teuthology.orchestra.run.smithi110.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.6 MB/s | 132 kB 00:00 2023-10-18T09:05:26.259 INFO:teuthology.orchestra.run.smithi061.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:26.260 INFO:teuthology.orchestra.run.smithi061.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-18T09:05:26.275 INFO:teuthology.orchestra.run.smithi061.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:26.277 INFO:teuthology.orchestra.run.smithi061.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-18T09:05:26.278 INFO:teuthology.orchestra.run.smithi061.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-18T09:05:26.279 INFO:teuthology.orchestra.run.smithi061.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-18T09:05:26.282 INFO:teuthology.orchestra.run.smithi061.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-18T09:05:26.285 INFO:teuthology.orchestra.run.smithi061.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-18T09:05:26.287 INFO:teuthology.orchestra.run.smithi061.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-18T09:05:26.436 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-10-18T09:05:26.450 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-10-18T09:05:26.450 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-10-18T09:05:26.450 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-10-18T09:05:26.450 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-10-18T09:05:26.451 INFO:teuthology.orchestra.run.smithi061.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-18T09:05:26.451 INFO:teuthology.orchestra.run.smithi061.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-18T09:05:26.451 INFO:teuthology.orchestra.run.smithi061.stdout: ceph x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.4 k 2023-10-18T09:05:26.451 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 5.4 M 2023-10-18T09:05:26.451 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 919 k 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 167 k 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 1.6 M 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 132 k 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 1.8 M 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 7.4 M 2023-10-18T09:05:26.452 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 47 k 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 14 M 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 49 M 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 261 k 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 192 k 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 261 k 2023-10-18T09:05:26.453 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-top noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 18 k 2023-10-18T09:05:26.454 INFO:teuthology.orchestra.run.smithi061.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-18T09:05:26.454 INFO:teuthology.orchestra.run.smithi061.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-18T09:05:26.454 INFO:teuthology.orchestra.run.smithi061.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-18T09:05:26.454 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 31 k 2023-10-18T09:05:26.454 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 840 k 2023-10-18T09:05:26.455 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 139 k 2023-10-18T09:05:26.455 INFO:teuthology.orchestra.run.smithi061.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-18T09:05:26.455 INFO:teuthology.orchestra.run.smithi061.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-18T09:05:26.455 INFO:teuthology.orchestra.run.smithi061.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-18T09:05:26.455 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 196 k 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 370 k 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 364 k 2023-10-18T09:05:26.456 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 107 k 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 94 k 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.6 M 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 178 k 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrading: 2023-10-18T09:05:26.457 INFO:teuthology.orchestra.run.smithi061.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-18T09:05:26.458 INFO:teuthology.orchestra.run.smithi061.stdout: librados2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:26.458 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 3.7 M 2023-10-18T09:05:26.458 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-10-18T09:05:26.458 INFO:teuthology.orchestra.run.smithi061.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-18T09:05:26.458 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 23 M 2023-10-18T09:05:26.459 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 23 k 2023-10-18T09:05:26.459 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 2.3 M 2023-10-18T09:05:26.459 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 242 k 2023-10-18T09:05:26.459 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 4.6 M 2023-10-18T09:05:26.459 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 18 M 2023-10-18T09:05:26.460 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6786.gf7420d7f.el8 ceph-noarch 15 k 2023-10-18T09:05:26.460 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 24 k 2023-10-18T09:05:26.460 INFO:teuthology.orchestra.run.smithi061.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-18T09:05:26.460 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-18T09:05:26.461 INFO:teuthology.orchestra.run.smithi061.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-18T09:05:26.461 INFO:teuthology.orchestra.run.smithi061.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-18T09:05:26.461 INFO:teuthology.orchestra.run.smithi061.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-18T09:05:26.461 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 185 k 2023-10-18T09:05:26.461 INFO:teuthology.orchestra.run.smithi061.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-18T09:05:26.462 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-18T09:05:26.462 INFO:teuthology.orchestra.run.smithi061.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-18T09:05:26.462 INFO:teuthology.orchestra.run.smithi061.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-18T09:05:26.462 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-18T09:05:26.462 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-18T09:05:26.463 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 518 k 2023-10-18T09:05:26.463 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-18T09:05:26.463 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2 x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 6.9 M 2023-10-18T09:05:26.463 INFO:teuthology.orchestra.run.smithi061.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-18T09:05:26.463 INFO:teuthology.orchestra.run.smithi061.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-18T09:05:26.464 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-18T09:05:26.464 INFO:teuthology.orchestra.run.smithi061.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-18T09:05:26.464 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-18T09:05:26.464 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-18T09:05:26.464 INFO:teuthology.orchestra.run.smithi061.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-18T09:05:26.465 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-18T09:05:26.465 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-18T09:05:26.465 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-18T09:05:26.465 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-18T09:05:26.465 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 45 k 2023-10-18T09:05:26.466 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common x86_64 2:18.0.0-6786.gf7420d7f.el8 ceph 125 k 2023-10-18T09:05:26.467 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-18T09:05:26.467 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-18T09:05:26.467 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-18T09:05:26.467 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-18T09:05:26.467 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-18T09:05:26.468 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-18T09:05:26.469 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-18T09:05:26.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-18T09:05:26.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-18T09:05:26.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-18T09:05:26.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-18T09:05:26.470 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-18T09:05:26.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-18T09:05:26.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-18T09:05:26.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-18T09:05:26.471 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-18T09:05:26.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-18T09:05:26.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-18T09:05:26.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-18T09:05:26.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-18T09:05:26.472 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-18T09:05:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-18T09:05:26.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-18T09:05:26.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-18T09:05:26.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-18T09:05:26.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-18T09:05:26.474 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-18T09:05:26.475 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-18T09:05:26.476 INFO:teuthology.orchestra.run.smithi061.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-18T09:05:26.476 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling module streams: 2023-10-18T09:05:26.476 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql 10 2023-10-18T09:05:26.476 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:05:26.476 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-10-18T09:05:26.477 INFO:teuthology.orchestra.run.smithi061.stdout:=========================================================================================================================== 2023-10-18T09:05:26.477 INFO:teuthology.orchestra.run.smithi061.stdout:Install 128 Packages 2023-10-18T09:05:26.477 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrade 3 Packages 2023-10-18T09:05:26.477 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:05:26.478 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 209 M 2023-10-18T09:05:26.478 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-10-18T09:05:26.478 INFO:teuthology.orchestra.run.smithi110.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 4.9 MB/s | 1.8 MB 00:00 2023-10-18T09:05:26.535 INFO:teuthology.orchestra.run.smithi170.stdout:(8/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el8 9.8 MB/s | 14 MB 00:01 2023-10-18T09:05:26.610 INFO:teuthology.orchestra.run.smithi170.stdout:(9/131): ceph-selinux-18.0.0-6786.gf7420d7f.el8 326 kB/s | 24 kB 00:00 2023-10-18T09:05:26.651 INFO:teuthology.orchestra.run.smithi061.stdout:(1/131): ceph-18.0.0-6786.gf7420d7f.el8.x86_64. 44 kB/s | 6.4 kB 00:00 2023-10-18T09:05:26.753 INFO:teuthology.orchestra.run.smithi110.stdout:(30/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x8 2.4 MB/s | 18 MB 00:07 2023-10-18T09:05:26.928 INFO:teuthology.orchestra.run.smithi110.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 1.4 MB/s | 242 kB 00:00 2023-10-18T09:05:27.012 INFO:teuthology.orchestra.run.smithi110.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 570 kB/s | 47 kB 00:00 2023-10-18T09:05:27.070 INFO:teuthology.orchestra.run.smithi110.stdout:(33/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 2.9 MB/s | 3.6 MB 00:01 2023-10-18T09:05:27.087 INFO:teuthology.orchestra.run.smithi110.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6786.gf 199 kB/s | 15 kB 00:00 2023-10-18T09:05:27.321 INFO:teuthology.orchestra.run.smithi110.stdout:(35/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 1.0 MB/s | 261 kB 00:00 2023-10-18T09:05:27.346 INFO:teuthology.orchestra.run.smithi110.stdout:(36/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 744 kB/s | 192 kB 00:00 2023-10-18T09:05:27.389 INFO:teuthology.orchestra.run.smithi110.stdout:(37/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 269 kB/s | 18 kB 00:00 2023-10-18T09:05:27.476 INFO:teuthology.orchestra.run.smithi061.stdout:(2/131): ceph-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.1 MB/s | 919 kB 00:00 2023-10-18T09:05:27.626 INFO:teuthology.orchestra.run.smithi061.stdout:(3/131): ceph-immutable-object-cache-18.0.0-678 1.1 MB/s | 167 kB 00:00 2023-10-18T09:05:27.631 INFO:teuthology.orchestra.run.smithi110.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 248 kB/s | 59 kB 00:00 2023-10-18T09:05:27.665 INFO:teuthology.orchestra.run.smithi110.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 964 kB/s | 306 kB 00:00 2023-10-18T09:05:27.698 INFO:teuthology.orchestra.run.smithi110.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-18T09:05:27.765 INFO:teuthology.orchestra.run.smithi110.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2023-10-18T09:05:27.790 INFO:teuthology.orchestra.run.smithi110.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-10-18T09:05:27.857 INFO:teuthology.orchestra.run.smithi110.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 496 kB/s | 33 kB 00:00 2023-10-18T09:05:27.882 INFO:teuthology.orchestra.run.smithi110.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.5 MB/s | 185 kB 00:00 2023-10-18T09:05:27.960 INFO:teuthology.orchestra.run.smithi170.stdout:(10/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x 602 kB/s | 5.4 MB 00:09 2023-10-18T09:05:27.975 INFO:teuthology.orchestra.run.smithi110.stdout:(45/131): ceph-mgr-diskprediction-local-18.0.0- 4.9 MB/s | 7.4 MB 00:01 2023-10-18T09:05:28.000 INFO:teuthology.orchestra.run.smithi110.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 115 kB/s | 16 kB 00:00 2023-10-18T09:05:28.025 INFO:teuthology.orchestra.run.smithi110.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-10-18T09:05:28.067 INFO:teuthology.orchestra.run.smithi110.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2023-10-18T09:05:28.092 INFO:teuthology.orchestra.run.smithi110.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 178 kB/s | 11 kB 00:00 2023-10-18T09:05:28.135 INFO:teuthology.orchestra.run.smithi110.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-10-18T09:05:28.201 INFO:teuthology.orchestra.run.smithi110.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 734 kB/s | 49 kB 00:00 2023-10-18T09:05:28.276 INFO:teuthology.orchestra.run.smithi110.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 6.4 MB/s | 1.2 MB 00:00 2023-10-18T09:05:28.277 INFO:teuthology.orchestra.run.smithi170.stdout:(11/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 825 kB/s | 261 kB 00:00 2023-10-18T09:05:28.322 INFO:teuthology.orchestra.run.smithi110.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 486 kB/s | 59 kB 00:00 2023-10-18T09:05:28.352 INFO:teuthology.orchestra.run.smithi110.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-10-18T09:05:28.369 INFO:teuthology.orchestra.run.smithi170.stdout:(12/131): libcephfs-devel-18.0.0-6786.gf7420d7f 339 kB/s | 31 kB 00:00 2023-10-18T09:05:28.377 INFO:teuthology.orchestra.run.smithi110.stdout:(55/131): python3-cherrypy-18.4.0-1.el8.noarch. 956 kB/s | 384 kB 00:00 2023-10-18T09:05:28.403 INFO:teuthology.orchestra.run.smithi110.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.4 MB/s | 283 kB 00:00 2023-10-18T09:05:28.428 INFO:teuthology.orchestra.run.smithi110.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-10-18T09:05:28.453 INFO:teuthology.orchestra.run.smithi110.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-10-18T09:05:28.478 INFO:teuthology.orchestra.run.smithi110.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 2.6 MB/s | 196 kB 00:00 2023-10-18T09:05:28.504 INFO:teuthology.orchestra.run.smithi110.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 850 kB/s | 64 kB 00:00 2023-10-18T09:05:28.529 INFO:teuthology.orchestra.run.smithi110.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-10-18T09:05:28.554 INFO:teuthology.orchestra.run.smithi110.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2023-10-18T09:05:28.580 INFO:teuthology.orchestra.run.smithi110.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-10-18T09:05:28.593 INFO:teuthology.orchestra.run.smithi061.stdout:(4/131): ceph-mds-18.0.0-6786.gf7420d7f.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-10-18T09:05:28.605 INFO:teuthology.orchestra.run.smithi110.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-10-18T09:05:28.630 INFO:teuthology.orchestra.run.smithi110.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-10-18T09:05:28.656 INFO:teuthology.orchestra.run.smithi110.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2023-10-18T09:05:28.681 INFO:teuthology.orchestra.run.smithi110.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 807 kB/s | 61 kB 00:00 2023-10-18T09:05:28.706 INFO:teuthology.orchestra.run.smithi110.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-18T09:05:28.723 INFO:teuthology.orchestra.run.smithi110.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 342 kB/s | 23 kB 00:00 2023-10-18T09:05:28.740 INFO:teuthology.orchestra.run.smithi110.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-10-18T09:05:28.791 INFO:teuthology.orchestra.run.smithi110.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.4 MB/s | 110 kB 00:00 2023-10-18T09:05:28.824 INFO:teuthology.orchestra.run.smithi110.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-18T09:05:28.874 INFO:teuthology.orchestra.run.smithi110.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-10-18T09:05:28.900 INFO:teuthology.orchestra.run.smithi110.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-10-18T09:05:28.934 INFO:teuthology.orchestra.run.smithi110.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.4 MB/s | 39 kB 00:00 2023-10-18T09:05:28.967 INFO:teuthology.orchestra.run.smithi110.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-10-18T09:05:29.009 INFO:teuthology.orchestra.run.smithi110.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 8.9 MB/s | 688 kB 00:00 2023-10-18T09:05:29.043 INFO:teuthology.orchestra.run.smithi110.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-10-18T09:05:29.060 INFO:teuthology.orchestra.run.smithi110.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.1 MB/s | 157 kB 00:00 2023-10-18T09:05:29.093 INFO:teuthology.orchestra.run.smithi110.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-10-18T09:05:29.118 INFO:teuthology.orchestra.run.smithi061.stdout:(5/131): ceph-base-18.0.0-6786.gf7420d7f.el8.x8 2.1 MB/s | 5.4 MB 00:02 2023-10-18T09:05:29.127 INFO:teuthology.orchestra.run.smithi110.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-18T09:05:29.160 INFO:teuthology.orchestra.run.smithi110.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 794 kB/s | 27 kB 00:00 2023-10-18T09:05:29.178 INFO:teuthology.orchestra.run.smithi170.stdout:(13/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 1.0 MB/s | 840 kB 00:00 2023-10-18T09:05:29.202 INFO:teuthology.orchestra.run.smithi110.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-18T09:05:29.218 INFO:teuthology.orchestra.run.smithi061.stdout:(6/131): ceph-mgr-18.0.0-6786.gf7420d7f.el8.x86 2.6 MB/s | 1.6 MB 00:00 2023-10-18T09:05:29.236 INFO:teuthology.orchestra.run.smithi110.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-18T09:05:29.320 INFO:teuthology.orchestra.run.smithi110.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-10-18T09:05:29.328 INFO:teuthology.orchestra.run.smithi170.stdout:(14/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 1.2 MB/s | 185 kB 00:00 2023-10-18T09:05:29.362 INFO:teuthology.orchestra.run.smithi110.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-10-18T09:05:29.520 INFO:teuthology.orchestra.run.smithi110.stdout:(87/131): python3-scipy-1.0.0-21.module+el8.5.0 17 MB/s | 14 MB 00:00 2023-10-18T09:05:29.546 INFO:teuthology.orchestra.run.smithi110.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.2 MB/s | 225 kB 00:00 2023-10-18T09:05:29.571 INFO:teuthology.orchestra.run.smithi110.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-18T09:05:29.596 INFO:teuthology.orchestra.run.smithi110.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-18T09:05:29.622 INFO:teuthology.orchestra.run.smithi110.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-10-18T09:05:29.638 INFO:teuthology.orchestra.run.smithi110.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-10-18T09:05:29.672 INFO:teuthology.orchestra.run.smithi110.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-10-18T09:05:29.729 INFO:teuthology.orchestra.run.smithi170.stdout:(15/131): librados-devel-18.0.0-6786.gf7420d7f. 348 kB/s | 139 kB 00:00 2023-10-18T09:05:29.731 INFO:teuthology.orchestra.run.smithi110.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 7.5 MB/s | 3.7 MB 00:00 2023-10-18T09:05:29.748 INFO:teuthology.orchestra.run.smithi110.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-10-18T09:05:29.790 INFO:teuthology.orchestra.run.smithi110.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-18T09:05:29.832 INFO:teuthology.orchestra.run.smithi110.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-18T09:05:29.924 INFO:teuthology.orchestra.run.smithi110.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-10-18T09:05:29.991 INFO:teuthology.orchestra.run.smithi110.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-10-18T09:05:30.024 INFO:teuthology.orchestra.run.smithi110.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.9 MB/s | 808 kB 00:00 2023-10-18T09:05:30.050 INFO:teuthology.orchestra.run.smithi110.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-10-18T09:05:30.085 INFO:teuthology.orchestra.run.smithi061.stdout:(7/131): ceph-mon-18.0.0-6786.gf7420d7f.el8.x86 4.8 MB/s | 4.6 MB 00:00 2023-10-18T09:05:30.092 INFO:teuthology.orchestra.run.smithi110.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-10-18T09:05:30.117 INFO:teuthology.orchestra.run.smithi110.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-10-18T09:05:30.134 INFO:teuthology.orchestra.run.smithi110.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-18T09:05:30.159 INFO:teuthology.orchestra.run.smithi110.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-10-18T09:05:30.176 INFO:teuthology.orchestra.run.smithi110.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-10-18T09:05:30.201 INFO:teuthology.orchestra.run.smithi110.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-10-18T09:05:30.227 INFO:teuthology.orchestra.run.smithi110.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 844 kB/s | 43 kB 00:00 2023-10-18T09:05:30.252 INFO:teuthology.orchestra.run.smithi110.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-18T09:05:30.262 INFO:teuthology.orchestra.run.smithi170.stdout:(16/131): libradosstriper1-18.0.0-6786.gf7420d7 971 kB/s | 518 kB 00:00 2023-10-18T09:05:30.277 INFO:teuthology.orchestra.run.smithi110.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-18T09:05:30.353 INFO:teuthology.orchestra.run.smithi110.stdout:(111/131): postgresql-server-10.21-2.module+el8 9.7 MB/s | 5.1 MB 00:00 2023-10-18T09:05:30.378 INFO:teuthology.orchestra.run.smithi110.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 186 kB/s | 19 kB 00:00 2023-10-18T09:05:30.403 INFO:teuthology.orchestra.run.smithi110.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.6 MB/s | 560 kB 00:00 2023-10-18T09:05:30.429 INFO:teuthology.orchestra.run.smithi110.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-10-18T09:05:30.454 INFO:teuthology.orchestra.run.smithi110.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 356 kB/s | 18 kB 00:00 2023-10-18T09:05:30.487 INFO:teuthology.orchestra.run.smithi110.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-10-18T09:05:30.521 INFO:teuthology.orchestra.run.smithi110.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-10-18T09:05:30.547 INFO:teuthology.orchestra.run.smithi110.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-10-18T09:05:30.572 INFO:teuthology.orchestra.run.smithi110.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-10-18T09:05:30.598 INFO:teuthology.orchestra.run.smithi110.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-18T09:05:30.639 INFO:teuthology.orchestra.run.smithi110.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 888 kB/s | 60 kB 00:00 2023-10-18T09:05:30.665 INFO:teuthology.orchestra.run.smithi110.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.4 MB/s | 98 kB 00:00 2023-10-18T09:05:30.682 INFO:teuthology.orchestra.run.smithi110.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-10-18T09:05:30.699 INFO:teuthology.orchestra.run.smithi110.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-10-18T09:05:30.732 INFO:teuthology.orchestra.run.smithi110.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 13 MB/s | 644 kB 00:00 2023-10-18T09:05:30.758 INFO:teuthology.orchestra.run.smithi110.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-10-18T09:05:30.783 INFO:teuthology.orchestra.run.smithi110.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-18T09:05:30.833 INFO:teuthology.orchestra.run.smithi110.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.7 MB/s | 2.3 MB 00:00 2023-10-18T09:05:30.884 INFO:teuthology.orchestra.run.smithi110.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-18T09:05:32.975 INFO:teuthology.orchestra.run.smithi110.stdout:(130/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 1.7 MB/s | 3.7 MB 00:02 2023-10-18T09:05:33.059 INFO:teuthology.orchestra.run.smithi110.stdout:(131/131): librados2-18.0.0-6786.gf7420d7f.el8. 1.6 MB/s | 3.7 MB 00:02 2023-10-18T09:05:33.063 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:05:33.064 INFO:teuthology.orchestra.run.smithi110.stdout:Total 14 MB/s | 209 MB 00:15 2023-10-18T09:05:33.343 INFO:teuthology.orchestra.run.smithi061.stdout:(8/131): ceph-radosgw-18.0.0-6786.gf7420d7f.el8 4.3 MB/s | 14 MB 00:03 2023-10-18T09:05:33.585 INFO:teuthology.orchestra.run.smithi061.stdout:(9/131): ceph-common-18.0.0-6786.gf7420d7f.el8. 3.2 MB/s | 23 MB 00:07 2023-10-18T09:05:33.587 INFO:teuthology.orchestra.run.smithi170.stdout:(17/131): ceph-test-18.0.0-6786.gf7420d7f.el8.x 7.0 MB/s | 49 MB 00:06 2023-10-18T09:05:33.611 INFO:teuthology.orchestra.run.smithi061.stdout:(10/131): ceph-selinux-18.0.0-6786.gf7420d7f.el 91 kB/s | 24 kB 00:00 2023-10-18T09:05:33.671 INFO:teuthology.orchestra.run.smithi170.stdout:(18/131): python3-ceph-argparse-18.0.0-6786.gf7 539 kB/s | 45 kB 00:00 2023-10-18T09:05:33.711 INFO:teuthology.orchestra.run.smithi061.stdout:(11/131): cephfs-mirror-18.0.0-6786.gf7420d7f.e 2.6 MB/s | 261 kB 00:00 2023-10-18T09:05:33.761 INFO:teuthology.orchestra.run.smithi061.stdout:(12/131): libcephfs-devel-18.0.0-6786.gf7420d7f 622 kB/s | 31 kB 00:00 2023-10-18T09:05:33.788 INFO:teuthology.orchestra.run.smithi170.stdout:(19/131): python3-ceph-common-18.0.0-6786.gf742 1.1 MB/s | 125 kB 00:00 2023-10-18T09:05:33.867 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2023-10-18T09:05:33.897 INFO:teuthology.orchestra.run.smithi170.stdout:(20/131): python3-cephfs-18.0.0-6786.gf7420d7f. 1.8 MB/s | 196 kB 00:00 2023-10-18T09:05:33.903 INFO:teuthology.orchestra.run.smithi061.stdout:(13/131): libcephfs2-18.0.0-6786.gf7420d7f.el8. 5.8 MB/s | 840 kB 00:00 2023-10-18T09:05:33.961 INFO:teuthology.orchestra.run.smithi061.stdout:(14/131): libcephsqlite-18.0.0-6786.gf7420d7f.e 3.1 MB/s | 185 kB 00:00 2023-10-18T09:05:33.987 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2023-10-18T09:05:33.987 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2023-10-18T09:05:34.020 INFO:teuthology.orchestra.run.smithi061.stdout:(15/131): librados-devel-18.0.0-6786.gf7420d7f. 2.3 MB/s | 139 kB 00:00 2023-10-18T09:05:34.064 INFO:teuthology.orchestra.run.smithi170.stdout:(21/131): python3-rados-18.0.0-6786.gf7420d7f.e 2.2 MB/s | 370 kB 00:00 2023-10-18T09:05:34.112 INFO:teuthology.orchestra.run.smithi061.stdout:(16/131): libradosstriper1-18.0.0-6786.gf7420d7 5.5 MB/s | 518 kB 00:00 2023-10-18T09:05:34.272 INFO:teuthology.orchestra.run.smithi170.stdout:(22/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 1.7 MB/s | 6.9 MB 00:04 2023-10-18T09:05:34.306 INFO:teuthology.orchestra.run.smithi170.stdout:(23/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 1.5 MB/s | 364 kB 00:00 2023-10-18T09:05:34.348 INFO:teuthology.orchestra.run.smithi170.stdout:(24/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.4 MB/s | 107 kB 00:00 2023-10-18T09:05:34.398 INFO:teuthology.orchestra.run.smithi170.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.0 MB/s | 94 kB 00:00 2023-10-18T09:05:34.507 INFO:teuthology.orchestra.run.smithi170.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 1.6 MB/s | 178 kB 00:00 2023-10-18T09:05:34.682 INFO:teuthology.orchestra.run.smithi170.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 132 kB/s | 23 kB 00:00 2023-10-18T09:05:34.799 INFO:teuthology.orchestra.run.smithi170.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.1 MB/s | 132 kB 00:00 2023-10-18T09:05:34.946 INFO:teuthology.orchestra.run.smithi061.stdout:(17/131): ceph-osd-18.0.0-6786.gf7420d7f.el8.x8 3.1 MB/s | 18 MB 00:05 2023-10-18T09:05:35.021 INFO:teuthology.orchestra.run.smithi061.stdout:(18/131): python3-ceph-argparse-18.0.0-6786.gf7 599 kB/s | 45 kB 00:00 2023-10-18T09:05:35.105 INFO:teuthology.orchestra.run.smithi061.stdout:(19/131): python3-ceph-common-18.0.0-6786.gf742 1.5 MB/s | 125 kB 00:00 2023-10-18T09:05:35.131 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2023-10-18T09:05:35.135 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2023-10-18T09:05:35.197 INFO:teuthology.orchestra.run.smithi061.stdout:(20/131): python3-cephfs-18.0.0-6786.gf7420d7f. 2.1 MB/s | 196 kB 00:00 2023-10-18T09:05:35.266 INFO:teuthology.orchestra.run.smithi170.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 3.8 MB/s | 1.8 MB 00:00 2023-10-18T09:05:35.314 INFO:teuthology.orchestra.run.smithi061.stdout:(21/131): librgw2-18.0.0-6786.gf7420d7f.el8.x86 5.8 MB/s | 6.9 MB 00:01 2023-10-18T09:05:35.347 INFO:teuthology.orchestra.run.smithi061.stdout:(22/131): python3-rados-18.0.0-6786.gf7420d7f.e 2.4 MB/s | 370 kB 00:00 2023-10-18T09:05:35.431 INFO:teuthology.orchestra.run.smithi061.stdout:(23/131): python3-rgw-18.0.0-6786.gf7420d7f.el8 1.3 MB/s | 107 kB 00:00 2023-10-18T09:05:35.489 INFO:teuthology.orchestra.run.smithi061.stdout:(24/131): python3-rbd-18.0.0-6786.gf7420d7f.el8 2.0 MB/s | 364 kB 00:00 2023-10-18T09:05:35.515 INFO:teuthology.orchestra.run.smithi061.stdout:(25/131): rbd-fuse-18.0.0-6786.gf7420d7f.el8.x8 1.1 MB/s | 94 kB 00:00 2023-10-18T09:05:35.598 INFO:teuthology.orchestra.run.smithi061.stdout:(26/131): rbd-nbd-18.0.0-6786.gf7420d7f.el8.x86 2.1 MB/s | 178 kB 00:00 2023-10-18T09:05:35.674 INFO:teuthology.orchestra.run.smithi061.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6786.g 308 kB/s | 23 kB 00:00 2023-10-18T09:05:35.766 INFO:teuthology.orchestra.run.smithi061.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6786.gf7420d7 1.4 MB/s | 132 kB 00:00 2023-10-18T09:05:35.949 INFO:teuthology.orchestra.run.smithi170.stdout:(30/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 2.2 MB/s | 3.6 MB 00:01 2023-10-18T09:05:36.108 INFO:teuthology.orchestra.run.smithi170.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 1.5 MB/s | 242 kB 00:00 2023-10-18T09:05:36.224 INFO:teuthology.orchestra.run.smithi170.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 409 kB/s | 47 kB 00:00 2023-10-18T09:05:36.300 INFO:teuthology.orchestra.run.smithi170.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6786.gf 198 kB/s | 15 kB 00:00 2023-10-18T09:05:36.314 INFO:teuthology.orchestra.run.smithi061.stdout:(29/131): rbd-mirror-18.0.0-6786.gf7420d7f.el8. 4.3 MB/s | 3.6 MB 00:00 2023-10-18T09:05:36.358 INFO:teuthology.orchestra.run.smithi061.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6786.gf7420 3.0 MB/s | 1.8 MB 00:00 2023-10-18T09:05:36.426 INFO:teuthology.orchestra.run.smithi170.stdout:(34/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 2.0 MB/s | 261 kB 00:00 2023-10-18T09:05:36.492 INFO:teuthology.orchestra.run.smithi061.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6786.gf7 1.8 MB/s | 242 kB 00:00 2023-10-18T09:05:36.551 INFO:teuthology.orchestra.run.smithi170.stdout:(35/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 1.5 MB/s | 192 kB 00:00 2023-10-18T09:05:36.559 INFO:teuthology.orchestra.run.smithi061.stdout:(32/131): ceph-mgr-rook-18.0.0-6786.gf7420d7f.e 713 kB/s | 47 kB 00:00 2023-10-18T09:05:36.618 INFO:teuthology.orchestra.run.smithi170.stdout:(36/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 272 kB/s | 18 kB 00:00 2023-10-18T09:05:36.625 INFO:teuthology.orchestra.run.smithi061.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6786.gf 225 kB/s | 15 kB 00:00 2023-10-18T09:05:36.751 INFO:teuthology.orchestra.run.smithi061.stdout:(34/131): ceph-volume-18.0.0-6786.gf7420d7f.el8 2.0 MB/s | 261 kB 00:00 2023-10-18T09:05:36.760 INFO:teuthology.orchestra.run.smithi170.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-10-18T09:05:36.818 INFO:teuthology.orchestra.run.smithi061.stdout:(35/131): cephadm-18.0.0-6786.gf7420d7f.el8.noa 2.8 MB/s | 192 kB 00:00 2023-10-18T09:05:36.877 INFO:teuthology.orchestra.run.smithi061.stdout:(36/131): cephfs-top-18.0.0-6786.gf7420d7f.el8. 312 kB/s | 18 kB 00:00 2023-10-18T09:05:37.177 INFO:teuthology.orchestra.run.smithi170.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 143 kB/s | 59 kB 00:00 2023-10-18T09:05:37.277 INFO:teuthology.orchestra.run.smithi061.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.7 MB/s | 7.4 MB 00:00 2023-10-18T09:05:37.285 INFO:teuthology.orchestra.run.smithi170.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 459 kB/s | 306 kB 00:00 2023-10-18T09:05:37.328 INFO:teuthology.orchestra.run.smithi170.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 499 kB/s | 75 kB 00:00 2023-10-18T09:05:37.461 INFO:teuthology.orchestra.run.smithi170.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2023-10-18T09:05:37.495 INFO:teuthology.orchestra.run.smithi170.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-10-18T09:05:37.604 INFO:teuthology.orchestra.run.smithi170.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-10-18T09:05:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 886 kB/s | 185 kB 00:00 2023-10-18T09:05:37.696 INFO:teuthology.orchestra.run.smithi170.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2023-10-18T09:05:37.804 INFO:teuthology.orchestra.run.smithi170.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-10-18T09:05:37.830 INFO:teuthology.orchestra.run.smithi170.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-10-18T09:05:37.922 INFO:teuthology.orchestra.run.smithi170.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 695 kB/s | 81 kB 00:00 2023-10-18T09:05:37.955 INFO:teuthology.orchestra.run.smithi170.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-10-18T09:05:38.022 INFO:teuthology.orchestra.run.smithi170.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-10-18T09:05:38.131 INFO:teuthology.orchestra.run.smithi170.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 450 kB/s | 49 kB 00:00 2023-10-18T09:05:38.231 INFO:teuthology.orchestra.run.smithi170.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 588 kB/s | 59 kB 00:00 2023-10-18T09:05:38.265 INFO:teuthology.orchestra.run.smithi170.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.8 MB/s | 1.2 MB 00:00 2023-10-18T09:05:38.332 INFO:teuthology.orchestra.run.smithi170.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2023-10-18T09:05:38.377 INFO:teuthology.orchestra.run.smithi061.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 53 kB/s | 59 kB 00:01 2023-10-18T09:05:38.382 INFO:teuthology.orchestra.run.smithi170.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2023-10-18T09:05:38.432 INFO:teuthology.orchestra.run.smithi170.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-10-18T09:05:38.661 INFO:teuthology.orchestra.run.smithi061.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 172 kB/s | 306 kB 00:01 2023-10-18T09:05:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:(57/131): ceph-common-18.0.0-6786.gf7420d7f.el8 1.1 MB/s | 23 MB 00:19 2023-10-18T09:05:38.733 INFO:teuthology.orchestra.run.smithi170.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 96 kB/s | 34 kB 00:00 2023-10-18T09:05:38.766 INFO:teuthology.orchestra.run.smithi170.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 588 kB/s | 196 kB 00:00 2023-10-18T09:05:38.786 INFO:teuthology.orchestra.run.smithi061.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 184 kB/s | 75 kB 00:00 2023-10-18T09:05:38.833 INFO:teuthology.orchestra.run.smithi170.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 187 kB/s | 19 kB 00:00 2023-10-18T09:05:38.875 INFO:teuthology.orchestra.run.smithi170.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 229 kB/s | 24 kB 00:00 2023-10-18T09:05:38.934 INFO:teuthology.orchestra.run.smithi170.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-10-18T09:05:38.976 INFO:teuthology.orchestra.run.smithi170.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 278 kB/s | 27 kB 00:00 2023-10-18T09:05:39.043 INFO:teuthology.orchestra.run.smithi170.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-10-18T09:05:39.086 INFO:teuthology.orchestra.run.smithi061.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 145 kB/s | 44 kB 00:00 2023-10-18T09:05:39.102 INFO:teuthology.orchestra.run.smithi170.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2023-10-18T09:05:39.144 INFO:teuthology.orchestra.run.smithi170.stdout:(66/131): python3-rsa-4.9-2.el8.noarch.rpm 137 kB/s | 64 kB 00:00 2023-10-18T09:05:39.169 INFO:teuthology.orchestra.run.smithi170.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 487 kB/s | 61 kB 00:00 2023-10-18T09:05:39.195 INFO:teuthology.orchestra.run.smithi061.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 891 kB/s | 476 kB 00:00 2023-10-18T09:05:39.211 INFO:teuthology.orchestra.run.smithi170.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 784 kB/s | 85 kB 00:00 2023-10-18T09:05:39.236 INFO:teuthology.orchestra.run.smithi170.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-18T09:05:39.261 INFO:teuthology.orchestra.run.smithi170.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 195 kB/s | 23 kB 00:00 2023-10-18T09:05:39.312 INFO:teuthology.orchestra.run.smithi170.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 620 kB/s | 47 kB 00:00 2023-10-18T09:05:39.337 INFO:teuthology.orchestra.run.smithi170.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-10-18T09:05:39.363 INFO:teuthology.orchestra.run.smithi170.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 850 kB/s | 43 kB 00:00 2023-10-18T09:05:39.405 INFO:teuthology.orchestra.run.smithi170.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-10-18T09:05:39.430 INFO:teuthology.orchestra.run.smithi170.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 717 kB/s | 48 kB 00:00 2023-10-18T09:05:39.437 INFO:teuthology.orchestra.run.smithi061.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 137 kB/s | 33 kB 00:00 2023-10-18T09:05:39.455 INFO:teuthology.orchestra.run.smithi170.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-10-18T09:05:39.480 INFO:teuthology.orchestra.run.smithi170.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 954 kB/s | 43 kB 00:00 2023-10-18T09:05:39.498 INFO:teuthology.orchestra.run.smithi170.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 940 kB/s | 87 kB 00:00 2023-10-18T09:05:39.523 INFO:teuthology.orchestra.run.smithi170.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-10-18T09:05:39.573 INFO:teuthology.orchestra.run.smithi170.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-10-18T09:05:39.598 INFO:teuthology.orchestra.run.smithi170.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 249 kB/s | 19 kB 00:00 2023-10-18T09:05:39.621 INFO:teuthology.orchestra.run.smithi061.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 347 kB/s | 185 kB 00:00 2023-10-18T09:05:39.624 INFO:teuthology.orchestra.run.smithi170.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-10-18T09:05:39.658 INFO:teuthology.orchestra.run.smithi170.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-10-18T09:05:39.671 INFO:teuthology.orchestra.run.smithi061.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 70 kB/s | 16 kB 00:00 2023-10-18T09:05:39.683 INFO:teuthology.orchestra.run.smithi170.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-10-18T09:05:39.716 INFO:teuthology.orchestra.run.smithi170.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 4.6 MB/s | 559 kB 00:00 2023-10-18T09:05:39.955 INFO:teuthology.orchestra.run.smithi061.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 517 kB/s | 173 kB 00:00 2023-10-18T09:05:40.029 INFO:teuthology.orchestra.run.smithi170.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 687 kB/s | 238 kB 00:00 2023-10-18T09:05:40.067 INFO:teuthology.orchestra.run.smithi170.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 1.4 MB/s | 605 kB 00:00 2023-10-18T09:05:40.092 INFO:teuthology.orchestra.run.smithi170.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.1 MB/s | 72 kB 00:00 2023-10-18T09:05:40.109 INFO:teuthology.orchestra.run.smithi170.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-10-18T09:05:40.135 INFO:teuthology.orchestra.run.smithi170.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-10-18T09:05:40.160 INFO:teuthology.orchestra.run.smithi170.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-18T09:05:40.185 INFO:teuthology.orchestra.run.smithi170.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-18T09:05:40.187 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2023-10-18T09:05:40.188 INFO:teuthology.orchestra.run.smithi061.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 744 kB/s | 384 kB 00:00 2023-10-18T09:05:40.210 INFO:teuthology.orchestra.run.smithi170.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-10-18T09:05:40.222 INFO:teuthology.orchestra.run.smithi061.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 305 kB/s | 81 kB 00:00 2023-10-18T09:05:40.253 INFO:teuthology.orchestra.run.smithi170.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-10-18T09:05:40.286 INFO:teuthology.orchestra.run.smithi170.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.2 MB/s | 171 kB 00:00 2023-10-18T09:05:40.312 INFO:teuthology.orchestra.run.smithi170.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-10-18T09:05:40.357 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T09:05:40.390 INFO:teuthology.orchestra.run.smithi170.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 3.5 MB/s | 2.3 MB 00:00 2023-10-18T09:05:40.413 INFO:teuthology.orchestra.run.smithi170.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-10-18T09:05:40.422 INFO:teuthology.orchestra.run.smithi061.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 48 kB/s | 11 kB 00:00 2023-10-18T09:05:40.438 INFO:teuthology.orchestra.run.smithi170.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-18T09:05:40.456 INFO:teuthology.orchestra.run.smithi061.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 75 kB/s | 18 kB 00:00 2023-10-18T09:05:40.463 INFO:teuthology.orchestra.run.smithi170.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-10-18T09:05:40.497 INFO:teuthology.orchestra.run.smithi170.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-10-18T09:05:40.522 INFO:teuthology.orchestra.run.smithi170.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 663 kB/s | 39 kB 00:00 2023-10-18T09:05:40.539 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T09:05:40.556 INFO:teuthology.orchestra.run.smithi170.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-10-18T09:05:40.589 INFO:teuthology.orchestra.run.smithi170.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-10-18T09:05:40.614 INFO:teuthology.orchestra.run.smithi170.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-18T09:05:40.631 INFO:teuthology.orchestra.run.smithi170.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-10-18T09:05:40.673 INFO:teuthology.orchestra.run.smithi170.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-10-18T09:05:40.715 INFO:teuthology.orchestra.run.smithi170.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 640 kB/s | 27 kB 00:00 2023-10-18T09:05:40.757 INFO:teuthology.orchestra.run.smithi170.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-18T09:05:40.782 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-18T09:05:40.791 INFO:teuthology.orchestra.run.smithi170.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-10-18T09:05:40.875 INFO:teuthology.orchestra.run.smithi170.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-10-18T09:05:40.917 INFO:teuthology.orchestra.run.smithi170.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-10-18T09:05:40.940 INFO:teuthology.orchestra.run.smithi061.stdout:(51/131): ceph-test-18.0.0-6786.gf7420d7f.el8.x 6.7 MB/s | 49 MB 00:07 2023-10-18T09:05:40.958 INFO:teuthology.orchestra.run.smithi170.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-10-18T09:05:40.959 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-18T09:05:40.965 INFO:teuthology.orchestra.run.smithi061.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 96 kB/s | 49 kB 00:00 2023-10-18T09:05:41.007 INFO:teuthology.orchestra.run.smithi061.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 2.0 MB/s | 1.2 MB 00:00 2023-10-18T09:05:41.025 INFO:teuthology.orchestra.run.smithi170.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2023-10-18T09:05:41.050 INFO:teuthology.orchestra.run.smithi170.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 697 kB/s | 64 kB 00:00 2023-10-18T09:05:41.076 INFO:teuthology.orchestra.run.smithi170.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-18T09:05:41.076 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-18T09:05:41.101 INFO:teuthology.orchestra.run.smithi170.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-10-18T09:05:41.126 INFO:teuthology.orchestra.run.smithi170.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-18T09:05:41.168 INFO:teuthology.orchestra.run.smithi170.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-18T09:05:41.319 INFO:teuthology.orchestra.run.smithi170.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-10-18T09:05:41.334 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-18T09:05:41.344 INFO:teuthology.orchestra.run.smithi170.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.1 MB/s | 198 kB 00:00 2023-10-18T09:05:41.377 INFO:teuthology.orchestra.run.smithi170.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 5.9 MB/s | 202 kB 00:00 2023-10-18T09:05:41.418 INFO:teuthology.orchestra.run.smithi061.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 133 kB/s | 60 kB 00:00 2023-10-18T09:05:41.419 INFO:teuthology.orchestra.run.smithi170.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-18T09:05:41.458 INFO:teuthology.orchestra.run.smithi061.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 628 kB/s | 283 kB 00:00 2023-10-18T09:05:41.495 INFO:teuthology.orchestra.run.smithi170.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-10-18T09:05:41.570 INFO:teuthology.orchestra.run.smithi170.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-10-18T09:05:41.604 INFO:teuthology.orchestra.run.smithi170.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-10-18T09:05:41.654 INFO:teuthology.orchestra.run.smithi170.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-10-18T09:05:41.658 INFO:teuthology.orchestra.run.smithi061.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 65 kB/s | 16 kB 00:00 2023-10-18T09:05:41.691 INFO:teuthology.orchestra.run.smithi061.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 144 kB/s | 34 kB 00:00 2023-10-18T09:05:41.729 INFO:teuthology.orchestra.run.smithi170.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-10-18T09:05:41.746 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-18T09:05:41.788 INFO:teuthology.orchestra.run.smithi170.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-10-18T09:05:41.933 INFO:teuthology.orchestra.run.smithi061.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 266 kB/s | 64 kB 00:00 2023-10-18T09:05:41.959 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:41.992 INFO:teuthology.orchestra.run.smithi061.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 589 kB/s | 196 kB 00:00 2023-10-18T09:05:42.101 INFO:teuthology.orchestra.run.smithi061.stdout:(60/131): python3-more-itertools-7.2.0-3.el8.no 51 kB/s | 59 kB 00:01 2023-10-18T09:05:42.159 INFO:teuthology.orchestra.run.smithi061.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 83 kB/s | 19 kB 00:00 2023-10-18T09:05:42.247 INFO:teuthology.orchestra.run.smithi061.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 94 kB/s | 24 kB 00:00 2023-10-18T09:05:42.255 INFO:teuthology.orchestra.run.smithi170.stdout:(130/131): librados2-18.0.0-6786.gf7420d7f.el8. 5.8 MB/s | 3.7 MB 00:00 2023-10-18T09:05:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:42.376 INFO:teuthology.orchestra.run.smithi061.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 106 kB/s | 29 kB 00:00 2023-10-18T09:05:42.393 INFO:teuthology.orchestra.run.smithi061.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 118 kB/s | 27 kB 00:00 2023-10-18T09:05:42.542 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:42.627 INFO:teuthology.orchestra.run.smithi061.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 262 kB/s | 61 kB 00:00 2023-10-18T09:05:42.673 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:42.747 INFO:teuthology.orchestra.run.smithi170.stdout:(131/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 3.4 MB/s | 3.7 MB 00:01 2023-10-18T09:05:42.751 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:05:42.751 INFO:teuthology.orchestra.run.smithi170.stdout:Total 8.6 MB/s | 209 MB 00:24 2023-10-18T09:05:42.769 INFO:teuthology.orchestra.run.smithi061.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 489 kB/s | 255 kB 00:00 2023-10-18T09:05:42.869 INFO:teuthology.orchestra.run.smithi061.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 353 kB/s | 85 kB 00:00 2023-10-18T09:05:42.875 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:05:42.976 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:05:42.994 INFO:teuthology.orchestra.run.smithi061.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 102 kB/s | 23 kB 00:00 2023-10-18T09:05:43.020 INFO:teuthology.orchestra.run.smithi061.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-18T09:05:43.159 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T09:05:43.204 INFO:teuthology.orchestra.run.smithi061.stdout:(70/131): python3-webob-1.8.5-1.el8.1.noarch.rp 304 kB/s | 251 kB 00:00 2023-10-18T09:05:43.254 INFO:teuthology.orchestra.run.smithi061.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 4.5 MB/s | 1.7 MB 00:00 2023-10-18T09:05:43.304 INFO:teuthology.orchestra.run.smithi061.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-10-18T09:05:43.329 INFO:teuthology.orchestra.run.smithi061.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 812 kB/s | 54 kB 00:00 2023-10-18T09:05:43.363 INFO:teuthology.orchestra.run.smithi061.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-18T09:05:43.396 INFO:teuthology.orchestra.run.smithi061.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-18T09:05:43.420 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T09:05:43.421 INFO:teuthology.orchestra.run.smithi061.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 663 kB/s | 39 kB 00:00 2023-10-18T09:05:43.463 INFO:teuthology.orchestra.run.smithi061.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-18T09:05:43.531 INFO:teuthology.orchestra.run.smithi061.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 4.8 MB/s | 538 kB 00:00 2023-10-18T09:05:43.556 INFO:teuthology.orchestra.run.smithi061.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-10-18T09:05:43.573 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-10-18T09:05:43.581 INFO:teuthology.orchestra.run.smithi061.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-18T09:05:43.623 INFO:teuthology.orchestra.run.smithi061.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-10-18T09:05:43.657 INFO:teuthology.orchestra.run.smithi061.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 796 kB/s | 27 kB 00:00 2023-10-18T09:05:43.694 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-10-18T09:05:43.694 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-10-18T09:05:43.707 INFO:teuthology.orchestra.run.smithi061.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-18T09:05:43.741 INFO:teuthology.orchestra.run.smithi061.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-18T09:05:43.872 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T09:05:43.914 INFO:teuthology.orchestra.run.smithi061.stdout:(85/131): python3-scipy-1.0.0-21.module+el8.5.0 15 MB/s | 14 MB 00:00 2023-10-18T09:05:43.975 INFO:teuthology.orchestra.run.smithi061.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.4 MB/s | 457 kB 00:00 2023-10-18T09:05:44.016 INFO:teuthology.orchestra.run.smithi061.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-10-18T09:05:44.109 INFO:teuthology.orchestra.run.smithi061.stdout:(88/131): python3-babel-2.5.1-7.el8.noarch.rpm 9.1 MB/s | 4.8 MB 00:00 2023-10-18T09:05:44.133 INFO:teuthology.orchestra.run.smithi061.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 548 kB/s | 64 kB 00:00 2023-10-18T09:05:44.159 INFO:teuthology.orchestra.run.smithi061.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-10-18T09:05:44.184 INFO:teuthology.orchestra.run.smithi061.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-10-18T09:05:44.209 INFO:teuthology.orchestra.run.smithi061.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-10-18T09:05:44.253 INFO:teuthology.orchestra.run.smithi061.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-10-18T09:05:44.255 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T09:05:44.352 INFO:teuthology.orchestra.run.smithi061.stdout:(94/131): python3-numpy-1.14.3-10.el8.x86_64.rp 6.1 MB/s | 3.7 MB 00:00 2023-10-18T09:05:44.385 INFO:teuthology.orchestra.run.smithi061.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-10-18T09:05:44.427 INFO:teuthology.orchestra.run.smithi061.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-18T09:05:44.478 INFO:teuthology.orchestra.run.smithi061.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-10-18T09:05:44.524 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T09:05:44.603 INFO:teuthology.orchestra.run.smithi061.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 12 MB/s | 4.9 MB 00:00 2023-10-18T09:05:44.667 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:44.687 INFO:teuthology.orchestra.run.smithi061.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-10-18T09:05:44.758 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:44.770 INFO:teuthology.orchestra.run.smithi061.stdout:(100/131): postgresql-contrib-10.21-2.module+el 4.7 MB/s | 808 kB 00:00 2023-10-18T09:05:44.813 INFO:teuthology.orchestra.run.smithi061.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-10-18T09:05:44.832 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-10-18T09:05:44.836 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-10-18T09:05:44.871 INFO:teuthology.orchestra.run.smithi061.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 8.2 MB/s | 1.5 MB 00:00 2023-10-18T09:05:44.897 INFO:teuthology.orchestra.run.smithi061.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.3 MB/s | 200 kB 00:00 2023-10-18T09:05:44.917 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:05:44.921 INFO:teuthology.orchestra.run.smithi061.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-10-18T09:05:44.955 INFO:teuthology.orchestra.run.smithi061.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-10-18T09:05:44.980 INFO:teuthology.orchestra.run.smithi061.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 822 kB/s | 48 kB 00:00 2023-10-18T09:05:45.002 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:05:45.006 INFO:teuthology.orchestra.run.smithi061.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-18T09:05:45.031 INFO:teuthology.orchestra.run.smithi061.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-10-18T09:05:45.056 INFO:teuthology.orchestra.run.smithi061.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-18T09:05:45.123 INFO:teuthology.orchestra.run.smithi061.stdout:(110/131): postgresql-server-10.21-2.module+el8 7.8 MB/s | 5.1 MB 00:00 2023-10-18T09:05:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-18T09:05:45.182 INFO:teuthology.orchestra.run.smithi061.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-10-18T09:05:45.207 INFO:teuthology.orchestra.run.smithi061.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 224 kB/s | 19 kB 00:00 2023-10-18T09:05:45.240 INFO:teuthology.orchestra.run.smithi061.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-10-18T09:05:45.274 INFO:teuthology.orchestra.run.smithi061.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-10-18T09:05:45.287 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-18T09:05:45.300 INFO:teuthology.orchestra.run.smithi061.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-10-18T09:05:45.325 INFO:teuthology.orchestra.run.smithi061.stdout:(116/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.1 MB/s | 155 kB 00:00 2023-10-18T09:05:45.358 INFO:teuthology.orchestra.run.smithi061.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-18T09:05:45.392 INFO:teuthology.orchestra.run.smithi061.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.0 MB/s | 605 kB 00:00 2023-10-18T09:05:45.417 INFO:teuthology.orchestra.run.smithi061.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.2 MB/s | 72 kB 00:00 2023-10-18T09:05:45.451 INFO:teuthology.orchestra.run.smithi061.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-10-18T09:05:45.476 INFO:teuthology.orchestra.run.smithi061.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.0 MB/s | 60 kB 00:00 2023-10-18T09:05:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-18T09:05:45.501 INFO:teuthology.orchestra.run.smithi061.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-10-18T09:05:45.527 INFO:teuthology.orchestra.run.smithi061.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-10-18T09:05:45.574 INFO:teuthology.orchestra.run.smithi061.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 533 kB/s | 39 kB 00:00 2023-10-18T09:05:45.619 INFO:teuthology.orchestra.run.smithi061.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.9 MB/s | 644 kB 00:00 2023-10-18T09:05:45.644 INFO:teuthology.orchestra.run.smithi061.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.4 MB/s | 171 kB 00:00 2023-10-18T09:05:45.669 INFO:teuthology.orchestra.run.smithi061.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-10-18T09:05:45.728 INFO:teuthology.orchestra.run.smithi061.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.8 MB/s | 2.3 MB 00:00 2023-10-18T09:05:45.779 INFO:teuthology.orchestra.run.smithi061.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-18T09:05:46.070 INFO:teuthology.orchestra.run.smithi061.stdout:(130/131): librbd1-18.0.0-6786.gf7420d7f.el8.x8 9.3 MB/s | 3.7 MB 00:00 2023-10-18T09:05:46.092 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-18T09:05:46.213 INFO:teuthology.orchestra.run.smithi061.stdout:(131/131): librados2-18.0.0-6786.gf7420d7f.el8. 6.6 MB/s | 3.7 MB 00:00 2023-10-18T09:05:46.217 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:05:46.217 INFO:teuthology.orchestra.run.smithi061.stdout:Total 11 MB/s | 209 MB 00:19 2023-10-18T09:05:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-18T09:05:46.868 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-18T09:05:47.067 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-18T09:05:47.756 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-18T09:05:47.941 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-10-18T09:05:48.048 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-18T09:05:48.087 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-10-18T09:05:48.087 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-10-18T09:05:48.269 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-18T09:05:48.487 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-18T09:05:48.689 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-18T09:05:48.906 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-18T09:05:49.177 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T09:05:49.373 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T09:05:49.502 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T09:05:49.722 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-10-18T09:05:49.727 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-10-18T09:05:50.072 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-10-18T09:05:50.238 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T09:05:50.422 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T09:05:50.447 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T09:05:50.632 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T09:05:50.728 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-18T09:05:50.732 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-18T09:05:50.965 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T09:05:51.001 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-18T09:05:51.242 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-18T09:05:51.403 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-18T09:05:51.499 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-18T09:05:51.640 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T09:05:52.054 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 5/134 2023-10-18T09:05:52.232 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:52.361 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:52.557 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:52.660 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:52.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:05:53.008 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-18T09:05:53.219 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T09:05:53.249 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-18T09:05:53.434 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T09:05:53.584 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-18T09:05:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T09:05:53.843 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T09:05:54.035 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T09:05:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T09:05:54.156 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:54.246 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T09:05:54.263 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:54.415 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:05:54.454 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T09:05:54.551 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:05:54.616 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-18T09:05:54.752 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-18T09:05:54.785 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-18T09:05:54.937 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-18T09:05:55.018 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-18T09:05:55.023 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-10-18T09:05:55.095 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-18T09:05:55.178 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-18T09:05:55.241 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-18T09:05:55.247 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T09:05:55.397 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-18T09:05:55.407 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-18T09:05:55.420 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-18T09:05:55.548 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T09:05:55.622 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-18T09:05:55.662 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-18T09:05:55.825 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-18T09:05:56.005 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-18T09:05:56.010 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-18T09:05:56.160 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-18T09:05:56.217 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-18T09:05:56.319 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-18T09:05:56.339 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T09:05:56.509 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-18T09:05:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-18T09:05:56.728 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-18T09:05:56.944 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-18T09:05:56.950 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:56.970 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-18T09:05:57.147 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T09:05:57.331 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-18T09:05:57.449 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:05:57.556 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-18T09:05:57.650 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:57.741 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-18T09:05:57.813 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:05:57.840 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-18T09:05:57.969 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-18T09:05:58.017 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:05:58.121 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-18T09:05:58.138 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:05:58.152 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-18T09:05:58.311 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-18T09:05:58.347 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T09:05:58.360 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-18T09:05:58.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T09:05:58.628 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-18T09:05:58.695 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-18T09:05:58.770 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T09:05:58.881 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-18T09:05:58.908 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T09:05:58.944 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-18T09:05:59.126 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T09:05:59.186 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-18T09:05:59.298 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-18T09:05:59.314 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T09:05:59.422 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-18T09:05:59.525 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-18T09:05:59.630 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-18T09:05:59.708 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:59.774 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-18T09:05:59.830 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 14/134 2023-10-18T09:05:59.973 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-18T09:05:59.982 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:06:00.044 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-18T09:06:00.117 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 15/134 2023-10-18T09:06:00.283 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-18T09:06:00.306 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-18T09:06:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-18T09:06:00.510 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-18T09:06:00.641 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-18T09:06:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-18T09:06:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-18T09:06:00.986 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-18T09:06:01.088 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-18T09:06:01.266 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-18T09:06:01.367 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-18T09:06:01.441 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T09:06:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-18T09:06:01.655 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T09:06:02.025 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T09:06:02.129 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-18T09:06:02.264 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-18T09:06:02.390 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T09:06:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-18T09:06:02.470 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-18T09:06:02.545 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T09:06:02.702 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T09:06:02.740 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-18T09:06:02.889 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T09:06:02.975 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-18T09:06:03.142 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T09:06:03.191 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T09:06:03.332 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T09:06:03.342 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-18T09:06:03.373 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T09:06:03.518 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T09:06:03.527 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T09:06:03.599 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-18T09:06:03.670 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T09:06:03.714 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T09:06:03.870 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-18T09:06:03.877 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-18T09:06:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T09:06:04.023 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-18T09:06:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-18T09:06:04.159 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T09:06:04.213 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T09:06:04.269 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-18T09:06:04.326 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-18T09:06:04.370 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T09:06:04.489 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-18T09:06:04.571 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-18T09:06:04.699 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T09:06:04.708 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-18T09:06:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-18T09:06:04.854 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T09:06:04.931 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-18T09:06:05.048 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 26/134 2023-10-18T09:06:05.063 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T09:06:05.094 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-18T09:06:05.188 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 27/134 2023-10-18T09:06:05.242 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-18T09:06:05.267 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T09:06:05.438 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-18T09:06:05.510 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T09:06:05.607 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-18T09:06:05.782 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-18T09:06:05.933 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-18T09:06:06.137 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-18T09:06:06.259 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 28/134 2023-10-18T09:06:06.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-18T09:06:06.604 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-18T09:06:06.612 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T09:06:06.730 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-18T09:06:06.900 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T09:06:06.926 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-18T09:06:07.064 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-18T09:06:07.180 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T09:06:07.236 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-18T09:06:07.291 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-18T09:06:07.513 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T09:06:07.517 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-18T09:06:07.573 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-18T09:06:07.717 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-18T09:06:07.881 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T09:06:08.063 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-18T09:06:08.089 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T09:06:08.253 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-18T09:06:08.430 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-18T09:06:08.641 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-18T09:06:08.841 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-18T09:06:08.912 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T09:06:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-18T09:06:09.089 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:09.172 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:09.336 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-18T09:06:09.381 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T09:06:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-18T09:06:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-18T09:06:09.858 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-18T09:06:10.080 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-18T09:06:10.091 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-18T09:06:10.236 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T09:06:10.305 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-18T09:06:10.432 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T09:06:10.531 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-18T09:06:10.706 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T09:06:10.723 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 36/134 2023-10-18T09:06:10.927 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T09:06:10.954 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 37/134 2023-10-18T09:06:11.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T09:06:11.117 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-18T09:06:11.239 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T09:06:11.326 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-18T09:06:11.420 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T09:06:11.518 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-18T09:06:11.624 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T09:06:11.731 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-18T09:06:11.789 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T09:06:11.940 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T09:06:12.110 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T09:06:12.308 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-18T09:06:12.352 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T09:06:12.540 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-18T09:06:12.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T09:06:12.759 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T09:06:12.986 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-18T09:06:13.042 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T09:06:13.160 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:13.281 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-18T09:06:13.294 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T09:06:13.422 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:13.486 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T09:06:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-18T09:06:13.721 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T09:06:13.730 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-18T09:06:14.001 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:14.010 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-18T09:06:14.276 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-18T09:06:14.380 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:14.563 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:14.575 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-18T09:06:14.790 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-18T09:06:14.841 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T09:06:15.009 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-18T09:06:15.106 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T09:06:15.223 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-18T09:06:15.377 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T09:06:15.412 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-18T09:06:15.653 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T09:06:15.776 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-18T09:06:15.908 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T09:06:16.091 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-18T09:06:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T09:06:16.350 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-18T09:06:16.581 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-18T09:06:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-18T09:06:16.921 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T09:06:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-18T09:06:17.119 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:17.199 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:17.256 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-18T09:06:17.346 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-18T09:06:17.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T09:06:17.598 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-18T09:06:17.805 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-18T09:06:17.984 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-18T09:06:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-18T09:06:18.301 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-18T09:06:18.493 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-18T09:06:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-18T09:06:18.892 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-18T09:06:19.184 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-18T09:06:19.441 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-18T09:06:19.604 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-18T09:06:19.797 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-18T09:06:20.001 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-18T09:06:20.218 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-18T09:06:20.408 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-18T09:06:20.618 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-18T09:06:20.764 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-18T09:06:21.037 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-18T09:06:21.131 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-18T09:06:21.361 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:21.471 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-18T09:06:21.763 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-18T09:06:21.946 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:21.982 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-18T09:06:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-18T09:06:22.443 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-18T09:06:22.580 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:22.655 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:22.793 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:23.829 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-18T09:06:24.102 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-18T09:06:24.404 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-18T09:06:24.770 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-18T09:06:25.082 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-18T09:06:25.280 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-18T09:06:26.212 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-18T09:06:26.438 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:26.534 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 92/134 2023-10-18T09:06:26.788 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 93/134 2023-10-18T09:06:31.973 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:32.204 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:33.156 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 94/134 2023-10-18T09:06:33.401 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:33.476 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:33.477 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2023-10-18T09:06:33.477 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2023-10-18T09:06:33.477 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /dev 2023-10-18T09:06:33.477 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /run 2023-10-18T09:06:33.478 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2023-10-18T09:06:33.478 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2023-10-18T09:06:33.478 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2023-10-18T09:06:33.478 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2023-10-18T09:06:33.479 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /dev 2023-10-18T09:06:33.479 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:06:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 95/134 2023-10-18T09:06:33.712 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:33.836 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:35.500 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:35.682 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:35.902 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:36.094 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:36.561 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:36.719 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:36.736 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:38.456 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:06:38.659 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:06:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T09:06:39.010 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T09:06:39.218 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:06:39.440 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:06:39.747 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:06:40.707 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:06:40.901 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:06:40.992 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:06:41.372 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T09:06:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:06:41.643 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:06:41.756 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:41.757 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2023-10-18T09:06:41.757 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2023-10-18T09:06:41.757 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-10-18T09:06:41.757 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2023-10-18T09:06:41.757 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2023-10-18T09:06:41.758 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2023-10-18T09:06:41.758 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2023-10-18T09:06:41.758 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2023-10-18T09:06:41.758 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-10-18T09:06:41.758 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:06:41.801 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:06:41.812 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:06:42.091 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:43.144 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T09:06:43.361 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:06:43.394 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:06:43.560 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:06:43.601 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:06:43.686 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:43.759 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:06:43.852 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:44.117 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:06:44.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:44.301 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:06:44.730 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:44.873 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:44.890 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:46.638 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:06:46.845 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:06:46.971 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T09:06:47.181 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T09:06:47.364 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:06:47.585 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:06:47.884 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:06:48.843 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:06:48.926 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:06:49.039 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:06:49.129 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:06:49.241 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T09:06:49.400 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-18T09:06:49.459 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T09:06:49.589 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-18T09:06:49.654 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:06:49.689 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:06:49.823 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:06:49.834 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:06:50.442 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-18T09:06:50.770 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-18T09:06:50.986 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 119/134 2023-10-18T09:06:51.185 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T09:06:51.204 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-18T09:06:51.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:06:51.371 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T09:06:51.397 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:06:51.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:06:51.584 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T09:06:51.589 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:06:51.764 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:06:52.123 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:06:52.281 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:06:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:06:54.058 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:06:54.075 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:06:54.258 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T09:06:54.541 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T09:06:54.818 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 96/134 2023-10-18T09:06:54.818 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /sys 2023-10-18T09:06:54.818 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /proc 2023-10-18T09:06:54.818 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /dev 2023-10-18T09:06:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /run 2023-10-18T09:06:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /mnt 2023-10-18T09:06:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /var/tmp 2023-10-18T09:06:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /home 2023-10-18T09:06:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /tmp 2023-10-18T09:06:54.820 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /dev 2023-10-18T09:06:54.820 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:06:54.839 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T09:06:55.185 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:55.193 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T09:06:55.349 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:06:55.358 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:06:55.549 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-18T09:06:55.758 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-18T09:06:55.841 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-18T09:06:56.083 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-18T09:06:56.095 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-18T09:06:56.096 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:06:56.302 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:06:56.303 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:06:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:06:56.782 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:06:56.915 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:06:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 97/134 2023-10-18T09:06:57.190 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:57.301 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:06:57.302 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T09:06:57.302 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T09:06:57.303 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T09:06:57.303 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T09:06:57.303 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T09:06:57.303 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:06:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:06:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:06:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T09:06:57.304 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T09:06:57.305 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T09:06:57.306 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T09:06:57.306 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T09:06:57.306 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T09:06:57.306 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T09:06:57.306 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T09:06:57.307 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T09:06:57.307 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T09:06:57.307 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T09:06:57.307 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T09:06:57.307 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T09:06:57.308 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T09:06:57.308 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T09:06:57.308 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T09:06:57.308 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T09:06:57.322 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T09:06:57.322 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T09:06:57.323 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T09:06:57.323 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T09:06:57.323 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T09:06:57.324 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T09:06:57.324 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T09:06:57.324 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T09:06:57.325 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T09:06:57.325 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T09:06:57.325 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T09:06:57.326 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T09:06:57.326 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T09:06:57.326 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T09:06:57.327 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T09:06:57.327 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T09:06:57.327 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T09:06:57.328 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T09:06:57.328 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T09:06:57.328 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T09:06:57.329 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T09:06:57.329 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T09:06:57.329 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T09:06:57.330 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T09:06:57.330 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T09:06:57.331 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T09:06:57.331 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T09:06:57.331 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T09:06:57.331 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T09:06:57.332 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T09:06:57.332 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T09:06:57.332 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T09:06:57.333 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T09:06:57.333 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T09:06:57.333 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T09:06:57.334 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T09:06:57.334 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T09:06:57.334 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T09:06:57.335 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T09:06:57.335 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T09:06:57.335 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T09:06:57.336 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-18T09:06:57.336 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-18T09:06:57.336 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-18T09:06:57.337 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-18T09:06:57.337 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-18T09:06:57.338 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-18T09:06:57.338 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-18T09:06:57.338 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-18T09:06:57.338 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-18T09:06:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-18T09:06:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-18T09:06:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-18T09:06:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-18T09:06:57.339 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-18T09:06:57.340 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-18T09:06:57.340 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-18T09:06:57.340 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-18T09:06:57.340 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-18T09:06:57.340 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-18T09:06:57.341 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-18T09:06:57.341 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-18T09:06:57.341 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-18T09:06:57.341 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-18T09:06:57.341 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-18T09:06:57.342 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-18T09:06:57.342 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-18T09:06:57.342 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-18T09:06:57.342 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-18T09:06:57.343 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-18T09:06:57.343 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-18T09:06:57.343 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-18T09:06:57.343 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-18T09:06:57.344 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-18T09:06:57.344 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-18T09:06:57.344 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-18T09:06:57.344 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-18T09:06:57.344 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-18T09:06:57.345 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-18T09:06:57.345 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-18T09:06:57.345 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-18T09:06:57.345 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-18T09:06:57.345 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-18T09:06:57.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-18T09:06:57.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-18T09:06:57.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-18T09:06:57.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-18T09:06:57.350 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-18T09:06:57.350 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-18T09:06:57.351 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-18T09:06:57.351 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-18T09:06:57.351 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T09:06:57.351 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-18T09:06:57.351 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T09:06:57.352 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T09:06:57.353 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T09:06:57.353 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:06:57.353 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T09:06:57.474 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 98/134 2023-10-18T09:06:57.642 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:57.778 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T09:06:58.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-18T09:06:58.198 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 99/134 2023-10-18T09:06:58.249 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 117/134 2023-10-18T09:06:58.393 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:58.436 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 100/134 2023-10-18T09:06:58.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-18T09:06:58.721 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T09:06:58.910 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-18T09:06:59.183 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-18T09:06:59.342 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T09:06:59.497 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T09:06:59.788 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:06:59.788 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:53 ago on Wed 18 Oct 2023 09:05:06 AM UTC. 2023-10-18T09:07:01.380 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:07:01.576 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 101/134 2023-10-18T09:07:01.682 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:01.709 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 102/134 2023-10-18T09:07:01.929 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:01.941 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 103/134 2023-10-18T09:07:01.946 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:02.071 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T09:07:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:07:02.259 INFO:teuthology.orchestra.run.smithi110.stdout:Installed products updated. 2023-10-18T09:07:02.370 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T09:07:02.409 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 104/134 2023-10-18T09:07:02.474 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:07:02.475 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2023-10-18T09:07:02.475 INFO:teuthology.orchestra.run.smithi110.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T09:07:02.475 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.475 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.477 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2023-10-18T09:07:02.477 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T09:07:02.477 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T09:07:02.478 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T09:07:02.478 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.478 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.478 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.478 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.479 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.479 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.479 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.479 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.479 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.480 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.480 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.480 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.480 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.480 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.481 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.481 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.481 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.481 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.481 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.482 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.482 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.482 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.482 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:02.482 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: jq-1.6-3.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T09:07:02.483 INFO:teuthology.orchestra.run.smithi110.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T09:07:02.484 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T09:07:02.484 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.484 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.484 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.484 INFO:teuthology.orchestra.run.smithi110.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T09:07:02.485 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:02.485 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T09:07:02.485 INFO:teuthology.orchestra.run.smithi110.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T09:07:02.485 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:02.485 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T09:07:02.486 INFO:teuthology.orchestra.run.smithi110.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T09:07:02.487 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T09:07:02.487 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T09:07:02.487 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T09:07:02.487 INFO:teuthology.orchestra.run.smithi110.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T09:07:02.487 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T09:07:02.488 INFO:teuthology.orchestra.run.smithi110.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T09:07:02.488 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T09:07:02.488 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T09:07:02.488 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T09:07:02.488 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T09:07:02.489 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T09:07:02.489 INFO:teuthology.orchestra.run.smithi110.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T09:07:02.489 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:02.489 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:02.489 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:02.490 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T09:07:02.490 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T09:07:02.490 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T09:07:02.490 INFO:teuthology.orchestra.run.smithi110.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T09:07:02.491 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T09:07:02.492 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T09:07:02.492 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T09:07:02.492 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T09:07:02.492 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T09:07:02.492 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T09:07:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T09:07:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T09:07:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T09:07:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T09:07:02.493 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T09:07:02.494 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T09:07:02.494 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T09:07:02.494 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T09:07:02.494 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T09:07:02.494 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T09:07:02.495 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T09:07:02.495 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:02.495 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:02.495 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T09:07:02.495 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T09:07:02.496 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T09:07:02.497 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T09:07:02.497 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.497 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.497 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T09:07:02.497 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T09:07:02.498 INFO:teuthology.orchestra.run.smithi110.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T09:07:02.499 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T09:07:02.499 INFO:teuthology.orchestra.run.smithi110.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T09:07:02.499 INFO:teuthology.orchestra.run.smithi110.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T09:07:02.499 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T09:07:02.499 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T09:07:02.500 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T09:07:02.500 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T09:07:02.500 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T09:07:02.500 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.500 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.501 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:02.501 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T09:07:02.501 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T09:07:02.501 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T09:07:02.501 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T09:07:02.502 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T09:07:02.502 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T09:07:02.502 INFO:teuthology.orchestra.run.smithi110.stdout: 2023-10-18T09:07:02.503 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2023-10-18T09:07:02.597 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:07:02.654 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:02.689 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T09:07:03.002 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T09:07:03.187 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T09:07:03.290 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-18T09:07:03.453 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T09:07:03.466 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-18T09:07:03.612 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T09:07:03.619 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-18T09:07:03.730 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-18T09:07:03.731 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:03.755 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 105/134 2023-10-18T09:07:03.923 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:03.923 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:07:03.975 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:07:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 106/134 2023-10-18T09:07:04.183 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:07:04.369 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:04.470 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 107/134 2023-10-18T09:07:04.687 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:07:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 108/134 2023-10-18T09:07:04.859 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:04.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T09:07:04.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T09:07:04.859 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T09:07:04.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T09:07:04.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T09:07:04.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:07:04.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:07:04.860 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T09:07:04.861 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T09:07:04.862 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T09:07:04.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T09:07:04.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T09:07:04.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T09:07:04.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T09:07:04.863 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T09:07:04.864 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T09:07:04.865 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T09:07:04.866 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T09:07:04.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T09:07:04.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T09:07:04.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T09:07:04.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T09:07:04.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T09:07:04.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T09:07:04.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T09:07:04.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T09:07:04.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T09:07:04.868 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T09:07:04.869 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T09:07:04.870 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T09:07:04.870 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T09:07:04.870 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T09:07:04.870 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T09:07:04.870 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T09:07:04.871 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-18T09:07:04.872 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-18T09:07:04.873 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-18T09:07:04.873 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-18T09:07:04.873 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-18T09:07:04.873 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-18T09:07:04.873 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-18T09:07:04.874 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-18T09:07:04.874 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-18T09:07:04.874 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-18T09:07:04.874 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-18T09:07:04.874 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-18T09:07:04.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-18T09:07:04.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-18T09:07:04.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-18T09:07:04.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-18T09:07:04.875 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-18T09:07:04.876 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-18T09:07:04.877 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-18T09:07:04.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-18T09:07:04.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-18T09:07:04.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-18T09:07:04.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-18T09:07:04.878 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-18T09:07:04.879 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-18T09:07:04.879 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-18T09:07:04.879 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-18T09:07:04.879 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-18T09:07:04.879 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-18T09:07:04.880 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-18T09:07:04.880 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-18T09:07:04.880 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-18T09:07:04.880 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-18T09:07:04.880 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-18T09:07:04.881 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-18T09:07:04.882 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-18T09:07:04.882 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-18T09:07:04.882 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-18T09:07:04.882 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-18T09:07:04.882 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T09:07:04.883 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T09:07:04.883 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T09:07:04.883 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:04.883 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T09:07:04.958 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:07:04.988 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 109/134 2023-10-18T09:07:06.520 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 110/134 2023-10-18T09:07:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:07:06.775 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 111/134 2023-10-18T09:07:06.966 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:07:07.050 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6786.gf7420 112/134 2023-10-18T09:07:07.242 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:07:07.306 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:07.307 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:02:01 ago on Wed 18 Oct 2023 09:05:06 AM UTC. 2023-10-18T09:07:07.686 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 113/134 2023-10-18T09:07:07.833 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:07:09.905 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-10-18T09:07:10.118 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:07:10.119 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-10-18T09:07:10.119 INFO:teuthology.orchestra.run.smithi170.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T09:07:10.119 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T09:07:10.120 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.121 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.121 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.121 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.121 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.121 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.122 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.122 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.122 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.122 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.123 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.123 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.123 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.123 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.123 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.124 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T09:07:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T09:07:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T09:07:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T09:07:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-3.el8.x86_64 2023-10-18T09:07:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T09:07:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T09:07:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:10.127 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T09:07:10.128 INFO:teuthology.orchestra.run.smithi170.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T09:07:10.128 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:10.128 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T09:07:10.128 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.129 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.129 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T09:07:10.129 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.129 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T09:07:10.129 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T09:07:10.130 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T09:07:10.130 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T09:07:10.130 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T09:07:10.130 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T09:07:10.130 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T09:07:10.131 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T09:07:10.132 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.133 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.134 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T09:07:10.134 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T09:07:10.134 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T09:07:10.134 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T09:07:10.134 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T09:07:10.135 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T09:07:10.135 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T09:07:10.135 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T09:07:10.135 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T09:07:10.135 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T09:07:10.136 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T09:07:10.137 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T09:07:10.138 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T09:07:10.138 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T09:07:10.138 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T09:07:10.138 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T09:07:10.138 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T09:07:10.139 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T09:07:10.140 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.140 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T09:07:10.140 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T09:07:10.140 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T09:07:10.140 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T09:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T09:07:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T09:07:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T09:07:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T09:07:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.142 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T09:07:10.143 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T09:07:10.144 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T09:07:10.144 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:07:10.144 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-18T09:07:10.316 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:13.242 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 114/134 2023-10-18T09:07:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 115/134 2023-10-18T09:07:13.715 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-18T09:07:13.996 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-18T09:07:15.345 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-18T09:07:15.698 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-18T09:07:15.905 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 119/134 2023-10-18T09:07:16.104 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-18T09:07:16.321 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 121/134 2023-10-18T09:07:16.466 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T09:07:20.098 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:20.423 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:20.462 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-18T09:07:20.641 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 124/134 2023-10-18T09:07:20.866 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 125/134 2023-10-18T09:07:21.196 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 126/134 2023-10-18T09:07:21.525 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-18T09:07:21.747 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:07:21.771 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:07:21.998 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-18T09:07:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-18T09:07:22.315 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-18T09:07:22.556 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-18T09:07:22.586 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-18T09:07:22.586 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:22.779 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:22.779 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:07:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-18T09:07:23.256 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:23.537 INFO:teuthology.orchestra.run.smithi061.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-18T09:07:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:07:24.191 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 1/134 2023-10-18T09:07:24.191 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2/134 2023-10-18T09:07:24.191 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 3/134 2023-10-18T09:07:24.191 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 4/134 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6786.gf7420 5/134 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 6/134 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 7/134 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 8/134 2023-10-18T09:07:24.192 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 9/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 10/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 11/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 12/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 13/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_ 14/134 2023-10-18T09:07:24.193 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 15/134 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 16/134 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_6 17/134 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86 18/134 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 19/134 2023-10-18T09:07:24.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el 20/134 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8. 21/134 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_6 22/134 2023-10-18T09:07:24.195 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 23/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 24/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 25/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 26/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 27/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 28/134 2023-10-18T09:07:24.196 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f. 29/134 2023-10-18T09:07:24.197 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noa 30/134 2023-10-18T09:07:24.205 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.n 31/134 2023-10-18T09:07:24.206 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6786.gf74 32/134 2023-10-18T09:07:24.206 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el 33/134 2023-10-18T09:07:24.206 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 34/134 2023-10-18T09:07:24.206 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.e 35/134 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 36/134 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 37/134 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 38/134 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-18T09:07:24.207 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-18T09:07:24.208 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-18T09:07:24.209 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-18T09:07:24.210 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-18T09:07:24.210 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-18T09:07:24.210 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-18T09:07:24.210 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-18T09:07:24.210 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-18T09:07:24.211 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-18T09:07:24.212 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-18T09:07:24.213 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-18T09:07:24.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-18T09:07:24.215 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-18T09:07:24.216 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-18T09:07:24.216 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-18T09:07:24.216 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-18T09:07:24.216 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-18T09:07:24.216 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-18T09:07:24.217 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-18T09:07:24.218 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-18T09:07:24.219 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-18T09:07:24.220 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-18T09:07:24.221 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-18T09:07:24.222 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-18T09:07:24.223 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-18T09:07:24.224 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 129/134 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 131/134 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-18T09:07:24.225 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-18T09:07:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-18T09:07:26.942 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:02:13 ago on Wed 18 Oct 2023 09:05:13 AM UTC. 2023-10-18T09:07:29.613 INFO:teuthology.orchestra.run.smithi061.stdout:Installed products updated. 2023-10-18T09:07:29.855 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:07:29.855 INFO:teuthology.orchestra.run.smithi061.stdout:Upgraded: 2023-10-18T09:07:29.856 INFO:teuthology.orchestra.run.smithi061.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-18T09:07:29.856 INFO:teuthology.orchestra.run.smithi061.stdout: librados2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.856 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.856 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-10-18T09:07:29.856 INFO:teuthology.orchestra.run.smithi061.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-18T09:07:29.857 INFO:teuthology.orchestra.run.smithi061.stdout: automake-1.16.1-7.el8.noarch 2023-10-18T09:07:29.857 INFO:teuthology.orchestra.run.smithi061.stdout: bison-3.0.4-10.el8.x86_64 2023-10-18T09:07:29.857 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.857 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.857 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.858 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.858 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.858 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.858 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.858 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.859 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.859 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.859 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.859 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.860 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.860 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.860 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.860 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.860 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.861 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.861 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.861 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.861 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.862 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.862 INFO:teuthology.orchestra.run.smithi061.stdout: cephfs-top-2:18.0.0-6786.gf7420d7f.el8.noarch 2023-10-18T09:07:29.862 INFO:teuthology.orchestra.run.smithi061.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-18T09:07:29.862 INFO:teuthology.orchestra.run.smithi061.stdout: flex-2.6.1-9.el8.x86_64 2023-10-18T09:07:29.862 INFO:teuthology.orchestra.run.smithi061.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-18T09:07:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-18T09:07:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: jq-1.6-3.el8.x86_64 2023-10-18T09:07:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-18T09:07:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-18T09:07:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-18T09:07:29.864 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.864 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.864 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.864 INFO:teuthology.orchestra.run.smithi061.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-18T09:07:29.864 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: libpq-13.5-1.el8.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.865 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.866 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-18T09:07:29.866 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.866 INFO:teuthology.orchestra.run.smithi061.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-18T09:07:29.866 INFO:teuthology.orchestra.run.smithi061.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-18T09:07:29.866 INFO:teuthology.orchestra.run.smithi061.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-18T09:07:29.867 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-18T09:07:29.867 INFO:teuthology.orchestra.run.smithi061.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-18T09:07:29.867 INFO:teuthology.orchestra.run.smithi061.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-18T09:07:29.867 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-18T09:07:29.867 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-18T09:07:29.868 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-18T09:07:29.868 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-18T09:07:29.868 INFO:teuthology.orchestra.run.smithi061.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-18T09:07:29.868 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-18T09:07:29.868 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-18T09:07:29.869 INFO:teuthology.orchestra.run.smithi061.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-18T09:07:29.869 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:29.869 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:29.869 INFO:teuthology.orchestra.run.smithi061.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-18T09:07:29.869 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-18T09:07:29.870 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-18T09:07:29.870 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-18T09:07:29.870 INFO:teuthology.orchestra.run.smithi061.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-18T09:07:29.870 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-18T09:07:29.870 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-18T09:07:29.871 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-18T09:07:29.872 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-18T09:07:29.873 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-18T09:07:29.873 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-18T09:07:29.873 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-18T09:07:29.873 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-18T09:07:29.874 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-18T09:07:29.874 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-18T09:07:29.874 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-18T09:07:29.874 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-18T09:07:29.874 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:29.875 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-18T09:07:29.875 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-18T09:07:29.875 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-18T09:07:29.875 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend-2.6-1.el8.noarch 2023-10-18T09:07:29.876 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-18T09:07:29.876 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-18T09:07:29.876 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-18T09:07:29.876 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-18T09:07:29.876 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-18T09:07:29.877 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.878 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-18T09:07:29.878 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-18T09:07:29.878 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-18T09:07:29.878 INFO:teuthology.orchestra.run.smithi061.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-18T09:07:29.878 INFO:teuthology.orchestra.run.smithi061.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-18T09:07:29.879 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-18T09:07:29.879 INFO:teuthology.orchestra.run.smithi061.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-18T09:07:29.879 INFO:teuthology.orchestra.run.smithi061.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-18T09:07:29.879 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-18T09:07:29.879 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-18T09:07:29.880 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-18T09:07:29.880 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-18T09:07:29.880 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-18T09:07:29.880 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.880 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd-2:18.0.0-6786.gf7420d7f.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-18T09:07:29.881 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-18T09:07:29.882 INFO:teuthology.orchestra.run.smithi061.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-18T09:07:29.882 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:07:29.883 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-10-18T09:07:30.165 DEBUG:teuthology.parallel:result is None 2023-10-18T09:07:30.166 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:30.396 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T09:07:30.454 INFO:teuthology.orchestra.run.smithi061.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:30.454 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:30.455 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T09:07:30.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:30.791 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T09:07:30.819 INFO:teuthology.orchestra.run.smithi110.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:30.820 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:30.820 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T09:07:30.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:31.086 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-18T09:07:31.115 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:31.116 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6786.gf7420d7f.el8 2023-10-18T09:07:31.116 INFO:teuthology.task.install:The correct ceph version 18.0.0-6786.gf7420d7f is installed. 2023-10-18T09:07:31.116 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-18T09:07:31.117 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:07:31.117 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T09:07:31.213 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:07:31.213 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T09:07:31.261 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:07:31.261 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-18T09:07:31.309 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-18T09:07:31.309 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:07:31.310 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T09:07:31.383 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T09:07:31.495 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:07:31.496 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T09:07:31.528 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T09:07:31.598 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:07:31.598 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-10-18T09:07:31.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-18T09:07:31.701 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-18T09:07:31.701 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:07:31.702 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T09:07:31.775 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T09:07:31.885 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:07:31.886 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T09:07:31.918 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T09:07:31.988 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:07:31.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-18T09:07:32.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-18T09:07:32.089 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-18T09:07:32.090 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:07:32.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T09:07:32.160 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T09:07:32.269 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:07:32.270 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T09:07:32.302 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T09:07:32.372 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:07:32.372 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-10-18T09:07:32.404 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-18T09:07:32.474 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-18T09:07:32.563 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-18T09:07:32.564 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:32.565 INFO:tasks.cephadm:Cluster fsid is c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:07:32.565 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-18T09:07:32.566 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.61', 'mon.b': '172.21.15.110', 'mon.c': '172.21.15.170'} 2023-10-18T09:07:32.566 INFO:tasks.cephadm:First mon is mon.a on smithi061 2023-10-18T09:07:32.566 INFO:tasks.cephadm:First mgr is x 2023-10-18T09:07:32.567 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-18T09:07:32.567 DEBUG:teuthology.orchestra.run.smithi061:> sudo hostname $(hostname -s) 2023-10-18T09:07:32.639 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2023-10-18T09:07:32.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2023-10-18T09:07:32.704 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-18T09:07:32.704 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:32.926 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73406/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6786-gf7420d7f', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6786.gf7420d7f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-16 23:26:05.686958', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20231016.203825', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-10-18T09:07:33.142 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-batrick-testing-20231016.203825, sha1 f7420d7f8e55ceea786dcbf536c338ce920c21eb from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=f7420d7f8e55ceea786dcbf536c338ce920c21eb 2023-10-18T09:07:33.142 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm 2023-10-18T09:07:33.143 DEBUG:teuthology.orchestra.run.smithi061:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:33.960 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 09:07 /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:33.961 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:34.691 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 09:07 /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:34.692 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20231016.203825/f7420d7f8e55ceea786dcbf536c338ce920c21eb/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:35.649 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-rw-r--. 1 ubuntu ubuntu 793356 Oct 18 09:07 /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:35.650 DEBUG:teuthology.orchestra.run.smithi061:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:35.702 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:35.723 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-18T09:07:35.759 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb on all hosts... 2023-10-18T09:07:35.760 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T09:07:35.764 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T09:07:35.767 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb pull 2023-10-18T09:07:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T09:07:37.529 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T09:07:40.105 INFO:teuthology.orchestra.run.smithi061.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T09:08:02.330 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2023-10-18T09:08:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T09:08:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T09:08:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2023-10-18T09:08:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T09:08:02.331 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2023-10-18T09:08:02.332 INFO:teuthology.orchestra.run.smithi110.stdout:} 2023-10-18T09:08:06.828 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-18T09:08:06.829 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T09:08:06.829 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T09:08:06.829 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2023-10-18T09:08:06.829 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T09:08:06.830 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-18T09:08:06.830 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-18T09:08:08.688 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:08:08.688 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T09:08:08.688 INFO:teuthology.orchestra.run.smithi061.stdout: "image_id": "34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120", 2023-10-18T09:08:08.689 INFO:teuthology.orchestra.run.smithi061.stdout: "repo_digests": [ 2023-10-18T09:08:08.689 INFO:teuthology.orchestra.run.smithi061.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b60cde6693b3225f0a93b57dd34fe281c6ae755adca32bcc00f7ca131ec348" 2023-10-18T09:08:08.689 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:08:08.689 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:08:08.714 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph 2023-10-18T09:08:08.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2023-10-18T09:08:08.823 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2023-10-18T09:08:08.860 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 777 /etc/ceph 2023-10-18T09:08:08.929 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2023-10-18T09:08:08.960 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2023-10-18T09:08:08.993 INFO:tasks.cephadm:Writing seed config... 2023-10-18T09:08:08.994 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-18T09:08:08.994 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-10-18T09:08:08.995 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-18T09:08:08.995 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-18T09:08:08.995 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-18T09:08:08.995 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-18T09:08:08.996 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-18T09:08:08.996 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-18T09:08:08.996 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-18T09:08:08.996 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-18T09:08:08.997 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-18T09:08:08.997 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-18T09:08:08.997 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-18T09:08:08.997 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-18T09:08:08.997 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-18T09:08:08.998 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-18T09:08:08.998 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-18T09:08:08.998 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-18T09:08:08.999 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-18T09:08:08.999 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-18T09:08:08.999 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-18T09:08:08.999 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-18T09:08:09.000 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:08:09.000 DEBUG:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-18T09:08:09.044 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c578e0ac-6d95-11ee-8db6-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2023-10-18T09:08:09.045 DEBUG:teuthology.orchestra.run.smithi061:mon.a> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service 2023-10-18T09:08:09.089 DEBUG:teuthology.orchestra.run.smithi061:mgr.x> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x.service 2023-10-18T09:08:09.134 INFO:tasks.cephadm:Bootstrapping... 2023-10-18T09:08:09.134 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb -v bootstrap --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.61 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:08:09.157 INFO:journalctl@ceph.mon.a.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:08:09.202 INFO:journalctl@ceph.mgr.x.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:08:09.372 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-10-18T09:08:09.372 INFO:teuthology.orchestra.run.smithi061.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', '-v', 'bootstrap', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.61', '--skip-admin-label'] 2023-10-18T09:08:09.550 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stdout 4.1.1 2023-10-18T09:08:09.550 INFO:teuthology.orchestra.run.smithi061.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-18T09:08:09.551 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying podman|docker is present... 2023-10-18T09:08:09.723 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stdout 4.1.1 2023-10-18T09:08:09.724 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying lvm2 is present... 2023-10-18T09:08:09.724 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying time synchronization is in place... 2023-10-18T09:08:09.738 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-18T09:08:09.739 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-18T09:08:09.752 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-18T09:08:09.752 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:08:09.765 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout enabled 2023-10-18T09:08:09.779 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout active 2023-10-18T09:08:09.779 INFO:teuthology.orchestra.run.smithi061.stdout:Unit chronyd.service is enabled and running 2023-10-18T09:08:09.779 INFO:teuthology.orchestra.run.smithi061.stdout:Repeating the final host check... 2023-10-18T09:08:09.953 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stdout 4.1.1 2023-10-18T09:08:09.953 INFO:teuthology.orchestra.run.smithi061.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-18T09:08:09.953 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl is present 2023-10-18T09:08:09.953 INFO:teuthology.orchestra.run.smithi061.stdout:lvcreate is present 2023-10-18T09:08:09.966 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-18T09:08:09.967 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-18T09:08:09.979 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-18T09:08:09.980 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:08:09.993 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout enabled 2023-10-18T09:08:10.006 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout active 2023-10-18T09:08:10.007 INFO:teuthology.orchestra.run.smithi061.stdout:Unit chronyd.service is enabled and running 2023-10-18T09:08:10.007 INFO:teuthology.orchestra.run.smithi061.stdout:Host looks OK 2023-10-18T09:08:10.007 INFO:teuthology.orchestra.run.smithi061.stdout:Cluster fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:10.008 INFO:teuthology.orchestra.run.smithi061.stdout:Acquiring lock 140439812105272 on /run/cephadm/c578e0ac-6d95-11ee-8db6-212e2dc638e7.lock 2023-10-18T09:08:10.008 INFO:teuthology.orchestra.run.smithi061.stdout:Lock 140439812105272 acquired on /run/cephadm/c578e0ac-6d95-11ee-8db6-212e2dc638e7.lock 2023-10-18T09:08:10.008 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying IP 172.21.15.61 port 3300 ... 2023-10-18T09:08:10.009 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying IP 172.21.15.61 port 6789 ... 2023-10-18T09:08:10.009 INFO:teuthology.orchestra.run.smithi061.stdout:Base mon IP(s) is [172.21.15.61:3300, 172.21.15.61:6789], mon addrv is [v2:172.21.15.61:3300,v1:172.21.15.61:6789] 2023-10-18T09:08:10.016 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-18T09:08:10.017 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.61 metric 100 2023-10-18T09:08:10.023 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-18T09:08:10.024 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-10-18T09:08:10.024 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-10-18T09:08:10.030 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-18T09:08:10.030 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-18T09:08:10.030 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-18T09:08:10.030 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-18T09:08:10.031 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout inet6 fe80::e081:b724:74b9:7104/64 scope link noprefixroute 2023-10-18T09:08:10.031 INFO:teuthology.orchestra.run.smithi061.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-18T09:08:10.032 INFO:teuthology.orchestra.run.smithi061.stdout:Mon IP `172.21.15.61` is in CIDR network `172.21.0.0/20` 2023-10-18T09:08:10.032 INFO:teuthology.orchestra.run.smithi061.stdout:Mon IP `172.21.15.61` is in CIDR network `172.21.0.0/20` 2023-10-18T09:08:10.032 INFO:teuthology.orchestra.run.smithi061.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-18T09:08:10.032 INFO:teuthology.orchestra.run.smithi061.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-18T09:08:10.033 INFO:teuthology.orchestra.run.smithi061.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T09:08:10.512 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stdout 34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120 2023-10-18T09:08:10.512 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb... 2023-10-18T09:08:10.512 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Getting image source signatures 2023-10-18T09:08:10.513 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Copying blob sha256:bde102e0dc1fccb96f1164f2fe1ba9d3f5d7242e22d2b672b1db3aedff4d056f 2023-10-18T09:08:10.513 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Copying blob sha256:8ed15414cb45b8b895a21f106743ceee45272cee0d9e7e5b1f49bd1e877e4a11 2023-10-18T09:08:10.513 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Copying config sha256:34f3475ed6c6b39cd122896e26d95104b6c809dcc4158afee48f92e1da145120 2023-10-18T09:08:10.513 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-18T09:08:10.514 INFO:teuthology.orchestra.run.smithi061.stdout:/bin/podman: stderr Storing signatures 2023-10-18T09:08:11.489 INFO:teuthology.orchestra.run.smithi061.stdout:ceph: stdout ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev) 2023-10-18T09:08:11.490 INFO:teuthology.orchestra.run.smithi061.stdout:Ceph version: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev) 2023-10-18T09:08:11.490 INFO:teuthology.orchestra.run.smithi061.stdout:Extracting ceph user uid/gid from container image... 2023-10-18T09:08:12.375 INFO:teuthology.orchestra.run.smithi061.stdout:stat: stdout 167 167 2023-10-18T09:08:12.376 INFO:teuthology.orchestra.run.smithi061.stdout:Creating initial keys... 2023-10-18T09:08:13.356 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph-authtool: stdout AQB8oC9lnxcwLxAAomN+LSIERrlz+wImxDHP+w== 2023-10-18T09:08:14.232 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph-authtool: stdout AQB9oC9lCfUuLxAAWB3Vb1dwCd0nFeSeDNl0Rg== 2023-10-18T09:08:15.109 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph-authtool: stdout AQB+oC9l8Ow8JhAAITt+5Hv0xEOVhjzdKm2EOg== 2023-10-18T09:08:15.110 INFO:teuthology.orchestra.run.smithi061.stdout:Creating initial monmap... 2023-10-18T09:08:15.991 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-18T09:08:15.991 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-18T09:08:15.991 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:15.991 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout:monmaptool for a [v2:172.21.15.61:3300,v1:172.21.15.61:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout:setting min_mon_release = pacific 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: set fsid to c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:08:15.992 INFO:teuthology.orchestra.run.smithi061.stdout:Creating mon... 2023-10-18T09:08:18.174 INFO:teuthology.orchestra.run.smithi061.stdout:create mon.a on 2023-10-18T09:08:18.429 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-18T09:08:18.650 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-18T09:08:18.899 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target. 2023-10-18T09:08:18.899 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target. 2023-10-18T09:08:19.285 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a 2023-10-18T09:08:19.286 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service: Unit ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service not loaded. 2023-10-18T09:08:19.505 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target.wants/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@.service. 2023-10-18T09:08:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:19 smithi061 systemd[1]: Starting Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:08:20.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:19 smithi061 podman[98022]: 2023-10-18T09:08:20.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:20 smithi061 bash[98022]: bf7219339987e19352d7933523a423e78944c7a971ce5115fc962861ccfa8ea7 2023-10-18T09:08:20.320 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T09:08:20.320 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout disabled 2023-10-18T09:08:20.336 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T09:08:20.336 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:08:20.336 INFO:teuthology.orchestra.run.smithi061.stdout:firewalld.service is not enabled 2023-10-18T09:08:20.337 INFO:teuthology.orchestra.run.smithi061.stdout:Not possible to enable service . firewalld.service is not available 2023-10-18T09:08:20.337 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mon to start... 2023-10-18T09:08:20.337 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mon... 2023-10-18T09:08:20.649 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:20 smithi061 ceph-mon[98062]: mkfs c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:20.649 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:20 smithi061 ceph-mon[98062]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T09:08:20.650 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:20 smithi061 systemd[1]: Started Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:08:21.420 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout cluster: 2023-10-18T09:08:21.421 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout id: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:21.422 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-18T09:08:21.422 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:21.422 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout services: 2023-10-18T09:08:21.422 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.747373s) 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout data: 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-18T09:08:21.423 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-18T09:08:21.424 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-18T09:08:21.424 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout pgs: 2023-10-18T09:08:21.424 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:21.425 INFO:teuthology.orchestra.run.smithi061.stdout:mon is available 2023-10-18T09:08:21.425 INFO:teuthology.orchestra.run.smithi061.stdout:Assimilating anything we can from ceph.conf... 2023-10-18T09:08:21.434 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T09:08:21.434 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: monmap e1: 1 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:08:21.435 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: fsmap 2023-10-18T09:08:21.435 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: osdmap e1: 0 total, 0 up, 0 in 2023-10-18T09:08:21.435 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: mgrmap e1: no daemons active 2023-10-18T09:08:21.435 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:21 smithi061 ceph-mon[98062]: from='client.? 172.21.15.61:0/1467290728' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-18T09:08:22.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:22 smithi061 ceph-mon[98062]: from='client.? 172.21.15.61:0/1691442039' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-18T09:08:22.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:22 smithi061 ceph-mon[98062]: from='client.? 172.21.15.61:0/1691442039' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-18T09:08:22.505 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:22.505 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [global] 2023-10-18T09:08:22.506 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout fsid = c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:22.506 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.61:3300,v1:172.21.15.61:6789] 2023-10-18T09:08:22.506 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-18T09:08:22.506 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-18T09:08:22.506 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-18T09:08:22.507 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-18T09:08:22.507 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:22.507 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-18T09:08:22.507 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-18T09:08:22.507 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:22.508 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [osd] 2023-10-18T09:08:22.508 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-18T09:08:22.508 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-18T09:08:22.508 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-18T09:08:22.508 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-18T09:08:22.509 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-18T09:08:22.509 INFO:teuthology.orchestra.run.smithi061.stdout:Generating new minimal ceph.conf... 2023-10-18T09:08:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:23 smithi061 ceph-mon[98062]: from='client.? 172.21.15.61:0/3187804909' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:08:24.086 INFO:teuthology.orchestra.run.smithi061.stdout:Restarting the monitor... 2023-10-18T09:08:24.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:24 smithi061 systemd[1]: Stopping Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:08:25.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a[98038]: 2023-10-18T09:08:24.556+0000 7f537ace7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:08:25.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a[98038]: 2023-10-18T09:08:24.556+0000 7f537ace7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-18T09:08:25.382 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:25 smithi061 bash[98401]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a 2023-10-18T09:08:25.636 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:25 smithi061 systemd[1]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service: Succeeded. 2023-10-18T09:08:25.636 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:25 smithi061 systemd[1]: Stopped Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:08:25.636 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:25 smithi061 systemd[1]: Starting Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:08:26.356 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 podman[98533]: 2023-10-18T09:08:26.356 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 bash[98533]: 0a47830f90836173fbd42fcdba0dd974de6f9d1ae2e5ff06286b2da0b18d08f3 2023-10-18T09:08:26.357 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T09:08:26.399 INFO:teuthology.orchestra.run.smithi061.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-18T09:08:26.698 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 7 2023-10-18T09:08:26.698 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: pidfile_write: ignore empty --pid-file 2023-10-18T09:08:26.698 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: load: jerasure load: lrc 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: RocksDB version: 7.9.2 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Git sha 0 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: DB SUMMARY 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: DB Session ID: 7OZM614IZ00Y4QRXWBY3 2023-10-18T09:08:26.699 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: CURRENT file: CURRENT 2023-10-18T09:08:26.700 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T09:08:26.700 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-18T09:08:26.700 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-18T09:08:26.700 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88782 ; 2023-10-18T09:08:26.700 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.error_if_exists: 0 2023-10-18T09:08:26.701 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.create_if_missing: 0 2023-10-18T09:08:26.701 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.paranoid_checks: 1 2023-10-18T09:08:26.701 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T09:08:26.701 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T09:08:26.701 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T09:08:26.702 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.env: 0x55de09c67da0 2023-10-18T09:08:26.702 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T09:08:26.702 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.info_log: 0x55de0c1396a0 2023-10-18T09:08:26.702 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T09:08:26.702 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.statistics: (nil) 2023-10-18T09:08:26.703 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.use_fsync: 0 2023-10-18T09:08:26.703 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_log_file_size: 0 2023-10-18T09:08:26.703 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T09:08:26.703 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T09:08:26.704 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T09:08:26.704 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T09:08:26.704 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_fallocate: 1 2023-10-18T09:08:26.704 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T09:08:26.705 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T09:08:26.705 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.use_direct_reads: 0 2023-10-18T09:08:26.705 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T09:08:26.705 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T09:08:26.706 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.db_log_dir: 2023-10-18T09:08:26.706 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.wal_dir: 2023-10-18T09:08:26.706 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T09:08:26.706 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T09:08:26.706 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T09:08:26.707 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T09:08:26.707 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T09:08:26.707 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T09:08:26.707 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T09:08:26.707 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T09:08:26.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.write_buffer_manager: 0x55de0c0f1360 2023-10-18T09:08:26.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T09:08:26.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T09:08:26.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T09:08:26.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T09:08:26.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T09:08:26.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T09:08:26.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T09:08:26.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T09:08:26.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.unordered_write: 0 2023-10-18T09:08:26.710 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T09:08:26.710 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T09:08:26.710 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T09:08:26.710 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T09:08:26.711 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.row_cache: None 2023-10-18T09:08:26.711 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.wal_filter: None 2023-10-18T09:08:26.711 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T09:08:26.711 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T09:08:26.711 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.two_write_queues: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.wal_compression: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.atomic_flush: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T09:08:26.712 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T09:08:26.713 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.log_readahead_size: 0 2023-10-18T09:08:26.713 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T09:08:26.713 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T09:08:26.714 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T09:08:26.714 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T09:08:26.714 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T09:08:26.715 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T09:08:26.715 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T09:08:26.716 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_background_jobs: 2 2023-10-18T09:08:26.716 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_background_compactions: -1 2023-10-18T09:08:26.716 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_subcompactions: 1 2023-10-18T09:08:26.717 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T09:08:26.717 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T09:08:26.717 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T09:08:26.718 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T09:08:26.718 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T09:08:26.719 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T09:08:26.719 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T09:08:26.719 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T09:08:26.720 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_open_files: -1 2023-10-18T09:08:26.720 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T09:08:26.720 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T09:08:26.721 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T09:08:26.721 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T09:08:26.722 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_background_flushes: -1 2023-10-18T09:08:26.722 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Compression algorithms supported: 2023-10-18T09:08:26.722 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T09:08:26.722 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kZSTD supported: 0 2023-10-18T09:08:26.723 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kXpressCompression supported: 0 2023-10-18T09:08:26.723 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T09:08:26.723 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kZlibCompression supported: 1 2023-10-18T09:08:26.723 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kSnappyCompression supported: 1 2023-10-18T09:08:26.723 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kLZ4Compression supported: 1 2023-10-18T09:08:26.724 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: kBZip2Compression supported: 0 2023-10-18T09:08:26.724 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T09:08:26.724 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T09:08:26.724 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-18T09:08:26.724 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T09:08:26.725 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T09:08:26.725 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.merge_operator: 2023-10-18T09:08:26.725 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_filter: None 2023-10-18T09:08:26.725 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T09:08:26.725 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T09:08:26.726 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T09:08:26.726 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T09:08:26.726 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55de0c1397a0) 2023-10-18T09:08:26.726 INFO:journalctl@ceph.mon.a.smithi061.stdout: cache_index_and_filter_blocks: 1 2023-10-18T09:08:26.726 INFO:journalctl@ceph.mon.a.smithi061.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: pin_top_level_index_and_filter: 1 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: index_type: 0 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: data_block_index_type: 0 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: index_shortening: 1 2023-10-18T09:08:26.727 INFO:journalctl@ceph.mon.a.smithi061.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T09:08:26.728 INFO:journalctl@ceph.mon.a.smithi061.stdout: checksum: 4 2023-10-18T09:08:26.728 INFO:journalctl@ceph.mon.a.smithi061.stdout: no_block_cache: 0 2023-10-18T09:08:26.728 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_cache: 0x55de0c18f1f0 2023-10-18T09:08:26.728 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_cache_name: BinnedLRUCache 2023-10-18T09:08:26.728 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_cache_options: 2023-10-18T09:08:26.729 INFO:journalctl@ceph.mon.a.smithi061.stdout: capacity : 536870912 2023-10-18T09:08:26.729 INFO:journalctl@ceph.mon.a.smithi061.stdout: num_shard_bits : 4 2023-10-18T09:08:26.729 INFO:journalctl@ceph.mon.a.smithi061.stdout: strict_capacity_limit : 0 2023-10-18T09:08:26.729 INFO:journalctl@ceph.mon.a.smithi061.stdout: high_pri_pool_ratio: 0.000 2023-10-18T09:08:26.729 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_cache_compressed: (nil) 2023-10-18T09:08:26.730 INFO:journalctl@ceph.mon.a.smithi061.stdout: persistent_cache: (nil) 2023-10-18T09:08:26.730 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_size: 4096 2023-10-18T09:08:26.730 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_size_deviation: 10 2023-10-18T09:08:26.730 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_restart_interval: 16 2023-10-18T09:08:26.730 INFO:journalctl@ceph.mon.a.smithi061.stdout: index_block_restart_interval: 1 2023-10-18T09:08:26.731 INFO:journalctl@ceph.mon.a.smithi061.stdout: metadata_block_size: 4096 2023-10-18T09:08:26.731 INFO:journalctl@ceph.mon.a.smithi061.stdout: partition_filters: 0 2023-10-18T09:08:26.731 INFO:journalctl@ceph.mon.a.smithi061.stdout: use_delta_encoding: 1 2023-10-18T09:08:26.731 INFO:journalctl@ceph.mon.a.smithi061.stdout: filter_policy: bloomfilter 2023-10-18T09:08:26.731 INFO:journalctl@ceph.mon.a.smithi061.stdout: whole_key_filtering: 1 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: verify_compression: 0 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: read_amp_bytes_per_bit: 0 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: format_version: 5 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: enable_index_compression: 1 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: block_align: 0 2023-10-18T09:08:26.732 INFO:journalctl@ceph.mon.a.smithi061.stdout: max_auto_readahead_size: 262144 2023-10-18T09:08:26.733 INFO:journalctl@ceph.mon.a.smithi061.stdout: prepopulate_block_cache: 0 2023-10-18T09:08:26.733 INFO:journalctl@ceph.mon.a.smithi061.stdout: initial_auto_readahead_size: 8192 2023-10-18T09:08:26.733 INFO:journalctl@ceph.mon.a.smithi061.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T09:08:26.733 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T09:08:26.733 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T09:08:26.734 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression: NoCompression 2023-10-18T09:08:26.734 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T09:08:26.734 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T09:08:26.734 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T09:08:26.734 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.num_levels: 7 2023-10-18T09:08:26.735 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T09:08:26.735 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T09:08:26.735 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T09:08:26.735 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T09:08:26.735 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T09:08:26.736 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T09:08:26.736 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T09:08:26.736 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:08:26.736 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T09:08:26.737 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T09:08:26.737 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:08:26.738 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:08:26.738 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T09:08:26.738 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T09:08:26.738 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T09:08:26.739 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T09:08:26.739 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:08:26.739 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:08:26.739 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T09:08:26.739 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T09:08:26.740 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:08:26.740 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T09:08:26.740 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T09:08:26.740 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T09:08:26.740 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T09:08:26.741 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T09:08:26.741 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T09:08:26.741 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T09:08:26.741 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T09:08:26.741 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T09:08:26.742 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T09:08:26.742 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T09:08:26.742 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T09:08:26.742 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T09:08:26.743 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T09:08:26.743 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T09:08:26.743 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T09:08:26.743 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T09:08:26.744 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T09:08:26.744 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T09:08:26.744 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T09:08:26.744 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T09:08:26.744 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T09:08:26.745 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T09:08:26.746 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T09:08:26.746 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T09:08:26.746 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T09:08:26.746 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T09:08:26.746 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.table_properties_collectors: 2023-10-18T09:08:26.747 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.inplace_update_support: 0 2023-10-18T09:08:26.747 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T09:08:26.747 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T09:08:26.747 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T09:08:26.747 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T09:08:26.748 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.bloom_locality: 0 2023-10-18T09:08:26.748 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.max_successive_merges: 0 2023-10-18T09:08:26.748 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T09:08:26.748 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T09:08:26.748 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T09:08:26.749 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T09:08:26.749 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.ttl: 2592000 2023-10-18T09:08:26.749 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T09:08:26.749 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T09:08:26.749 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T09:08:26.750 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enable_blob_files: false 2023-10-18T09:08:26.750 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.min_blob_size: 0 2023-10-18T09:08:26.750 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T09:08:26.751 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T09:08:26.751 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T09:08:26.751 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T09:08:26.752 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T09:08:26.752 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T09:08:26.752 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T09:08:26.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T09:08:26.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-18T09:08:26.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-18T09:08:26.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0be3c9d9-d998-4eae-b7e6-1a104e16bcb4 2023-10-18T09:08:26.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620106364479, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-18T09:08:26.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-18T09:08:26.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620106366111, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 270, "table_properties": {"data_size": 83196, "index_size": 236, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 11100, "raw_average_key_size": 48, "raw_value_size": 76920, "raw_average_value_size": 335, "num_data_blocks": 10, "num_entries": 229, "num_filter_entries": 229, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697620106, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0be3c9d9-d998-4eae-b7e6-1a104e16bcb4", "db_session_id": "7OZM614IZ00Y4QRXWBY3", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-18T09:08:26.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620106366246, "job": 1, "event": "recovery_finished"} 2023-10-18T09:08:26.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-18T09:08:26.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T09:08:26.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55de0c22c000 2023-10-18T09:08:26.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: DB pointer 0x55de0c218000 2023-10-18T09:08:26.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T09:08:26.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T09:08:26.758 INFO:journalctl@ceph.mon.a.smithi061.stdout: ** DB Stats ** 2023-10-18T09:08:26.758 INFO:journalctl@ceph.mon.a.smithi061.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:08:26.759 INFO:journalctl@ceph.mon.a.smithi061.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T09:08:26.759 INFO:journalctl@ceph.mon.a.smithi061.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:08:26.759 INFO:journalctl@ceph.mon.a.smithi061.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:08:26.759 INFO:journalctl@ceph.mon.a.smithi061.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: ** Compaction Stats [default] ** 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:08:26.760 INFO:journalctl@ceph.mon.a.smithi061.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T09:08:26.761 INFO:journalctl@ceph.mon.a.smithi061.stdout: L0 2/0 84.86 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:08:26.761 INFO:journalctl@ceph.mon.a.smithi061.stdout: Sum 2/0 84.86 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:08:26.761 INFO:journalctl@ceph.mon.a.smithi061.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:08:26.761 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.761 INFO:journalctl@ceph.mon.a.smithi061.stdout: ** Compaction Stats [default] ** 2023-10-18T09:08:26.762 INFO:journalctl@ceph.mon.a.smithi061.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:08:26.762 INFO:journalctl@ceph.mon.a.smithi061.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T09:08:26.762 INFO:journalctl@ceph.mon.a.smithi061.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:08:26.762 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.762 INFO:journalctl@ceph.mon.a.smithi061.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T09:08:26.763 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.763 INFO:journalctl@ceph.mon.a.smithi061.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:08:26.763 INFO:journalctl@ceph.mon.a.smithi061.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T09:08:26.763 INFO:journalctl@ceph.mon.a.smithi061.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T09:08:26.763 INFO:journalctl@ceph.mon.a.smithi061.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T09:08:26.764 INFO:journalctl@ceph.mon.a.smithi061.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T09:08:26.764 INFO:journalctl@ceph.mon.a.smithi061.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T09:08:26.764 INFO:journalctl@ceph.mon.a.smithi061.stdout: Cumulative compaction: 0.00 GB write, 14.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:08:26.764 INFO:journalctl@ceph.mon.a.smithi061.stdout: Interval compaction: 0.00 GB write, 14.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:08:26.764 INFO:journalctl@ceph.mon.a.smithi061.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T09:08:26.765 INFO:journalctl@ceph.mon.a.smithi061.stdout: Block cache BinnedLRUCache@0x55de0c18f1f0#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-10-18T09:08:26.765 INFO:journalctl@ceph.mon.a.smithi061.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2023-10-18T09:08:26.765 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.765 INFO:journalctl@ceph.mon.a.smithi061.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T09:08:26.765 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: starting mon.a rank 0 at public addrs [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] at bind addrs [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???) e1 preinit fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).mds e1 new map 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).mds e1 print_map 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout: e1 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T09:08:26.766 INFO:journalctl@ceph.mon.a.smithi061.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T09:08:26.767 INFO:journalctl@ceph.mon.a.smithi061.stdout: legacy client fscid: -1 2023-10-18T09:08:26.767 INFO:journalctl@ceph.mon.a.smithi061.stdout: 2023-10-18T09:08:26.767 INFO:journalctl@ceph.mon.a.smithi061.stdout: No filesystems configured 2023-10-18T09:08:26.767 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T09:08:26.767 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 systemd[1]: Started Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-18T09:08:26.768 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: monmap e1: 1 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:08:26.769 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: fsmap 2023-10-18T09:08:26.769 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: osdmap e1: 0 total, 0 up, 0 in 2023-10-18T09:08:26.769 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:26 smithi061 ceph-mon[98573]: mgrmap e1: no daemons active 2023-10-18T09:08:27.483 INFO:teuthology.orchestra.run.smithi061.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-18T09:08:27.510 INFO:teuthology.orchestra.run.smithi061.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:08:27.511 INFO:teuthology.orchestra.run.smithi061.stdout:Creating mgr... 2023-10-18T09:08:27.511 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-18T09:08:27.512 INFO:teuthology.orchestra.run.smithi061.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-18T09:08:27.891 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x 2023-10-18T09:08:27.891 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x.service: Unit ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x.service not loaded. 2023-10-18T09:08:28.109 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7.target.wants/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@.service. 2023-10-18T09:08:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1357572856' entity='client.admin' 2023-10-18T09:08:28.255 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:28 smithi061 systemd[1]: Starting Ceph mgr.x for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:08:29.220 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:28 smithi061 podman[98802]: 2023-10-18T09:08:29.334 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T09:08:29.335 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout disabled 2023-10-18T09:08:29.351 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T09:08:29.352 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:08:29.352 INFO:teuthology.orchestra.run.smithi061.stdout:firewalld.service is not enabled 2023-10-18T09:08:29.352 INFO:teuthology.orchestra.run.smithi061.stdout:Not possible to enable service . firewalld.service is not available 2023-10-18T09:08:29.370 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T09:08:29.371 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout disabled 2023-10-18T09:08:29.387 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T09:08:29.388 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:08:29.388 INFO:teuthology.orchestra.run.smithi061.stdout:firewalld.service is not enabled 2023-10-18T09:08:29.388 INFO:teuthology.orchestra.run.smithi061.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-18T09:08:29.389 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mgr to start... 2023-10-18T09:08:29.389 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mgr... 2023-10-18T09:08:29.491 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 bash[98802]: 99c5d242f278e688a84ec93e782bac6687aee6759968fab421ef7367877f286e 2023-10-18T09:08:29.491 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 systemd[1]: Started Ceph mgr.x for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:08:29.492 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:29.375+0000 7f6d0d745200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T09:08:29.746 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:29.488+0000 7f6d0d745200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T09:08:29.746 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:29.609+0000 7f6d0d745200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T09:08:30.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:29.748+0000 7f6d0d745200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T09:08:30.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:29.986+0000 7f6d0d745200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T09:08:30.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3872917992' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T09:08:30.572 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:30.575 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:08:30.575 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:08:30.575 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T09:08:30.576 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T09:08:30.576 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T09:08:30.576 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T09:08:30.576 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.576 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T09:08:30.577 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T09:08:30.577 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 0 2023-10-18T09:08:30.577 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:30.577 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T09:08:30.577 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T09:08:30.578 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:30.578 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-18T09:08:30.578 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T09:08:30.578 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:30.578 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T09:08:30.579 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T09:08:30.580 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T09:08:30.580 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T09:08:30.583 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T09:08:30.584 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T09:08:30.584 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.584 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T09:08:30.584 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T09:08:30.584 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T09:08:30.585 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T09:08:30.585 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T09:08:30.585 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T09:08:30.585 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T09:08:30.585 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T09:08:30.586 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T09:08:30.586 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.586 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T09:08:30.586 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:30.586 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T09:08:30.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T09:08:30.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T09:08:30.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-18T09:08:30.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:30.588 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T09:08:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T09:08:20.266489+0000", 2023-10-18T09:08:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:30.590 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:30.590 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T09:08:30.590 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:08:30.590 INFO:teuthology.orchestra.run.smithi061.stdout:mgr not available, waiting (1/15)... 2023-10-18T09:08:30.805 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:30.510+0000 7f6d0d745200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T09:08:30.805 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:30.682+0000 7f6d0d745200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T09:08:31.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:30.802+0000 7f6d0d745200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T09:08:31.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:30.931+0000 7f6d0d745200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T09:08:31.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:31.496+0000 7f6d0d745200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T09:08:32.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:32.416+0000 7f6d0d745200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T09:08:32.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:32.527+0000 7f6d0d745200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T09:08:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:33 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1631649443' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T09:08:33.928 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:33.570+0000 7f6d0d745200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:08:33.929 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:33.714+0000 7f6d0d745200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T09:08:33.973 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:33.973 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:08:33.974 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:08:33.974 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T09:08:33.974 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T09:08:33.974 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T09:08:33.974 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T09:08:33.975 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.975 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T09:08:33.975 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T09:08:33.975 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 0 2023-10-18T09:08:33.975 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:33.976 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T09:08:33.976 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T09:08:33.976 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:33.977 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-10-18T09:08:33.977 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T09:08:33.977 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:33.977 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T09:08:33.978 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T09:08:33.978 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.978 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T09:08:33.979 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:33.979 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T09:08:33.979 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T09:08:33.980 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T09:08:33.980 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T09:08:33.980 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T09:08:33.981 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T09:08:33.981 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.982 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T09:08:33.982 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T09:08:33.982 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T09:08:33.983 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T09:08:33.983 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T09:08:33.983 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T09:08:33.984 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T09:08:33.984 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T09:08:33.984 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T09:08:33.985 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.985 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T09:08:33.985 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:33.986 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T09:08:33.986 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T09:08:33.986 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.987 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T09:08:33.987 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-18T09:08:33.987 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T09:08:33.987 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T09:08:33.987 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T09:08:33.988 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T09:08:33.988 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T09:08:33.988 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:33.988 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:33.988 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.989 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T09:08:33.989 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:33.989 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T09:08:20.266489+0000", 2023-10-18T09:08:33.989 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:33.989 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:33.990 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T09:08:33.990 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:08:33.990 INFO:teuthology.orchestra.run.smithi061.stdout:mgr not available, waiting (2/15)... 2023-10-18T09:08:34.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:33.950+0000 7f6d0d745200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:08:34.631 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:34.338+0000 7f6d0d745200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T09:08:35.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:34.627+0000 7f6d0d745200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T09:08:35.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:34.830+0000 7f6d0d745200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T09:08:35.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:35.190+0000 7f6d0d745200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T09:08:36.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:36.169+0000 7f6d0d745200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T09:08:36.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:36.350+0000 7f6d0d745200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T09:08:37.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:36.611+0000 7f6d0d745200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T09:08:37.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:36.762+0000 7f6d0d745200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T09:08:37.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:37 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:37.053+0000 7f6d0d745200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T09:08:38.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:38.182+0000 7f6d0d745200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T09:08:38.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: Activating manager daemon x 2023-10-18T09:08:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: mgrmap e2: x(active, starting, since 0.00415444s) 2023-10-18T09:08:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T09:08:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T09:08:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T09:08:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:08:38.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T09:08:38.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: Manager daemon x is now available 2023-10-18T09:08:38.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T09:08:38.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T09:08:38.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' 2023-10-18T09:08:38.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' 2023-10-18T09:08:38.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:38 smithi061 ceph-mon[98573]: from='mgr.14100 172.21.15.61:0/1055308143' entity='mgr.x' 2023-10-18T09:08:40.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:40 smithi061 ceph-mon[98573]: mgrmap e3: x(active, since 1.01s) 2023-10-18T09:08:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:41 smithi061 ceph-mon[98573]: mgrmap e4: x(active, since 2s) 2023-10-18T09:08:42.268 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:42.268 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:08:42.268 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:08:42.268 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "health": { 2023-10-18T09:08:42.268 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-18T09:08:42.269 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-18T09:08:42.269 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-18T09:08:42.269 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.269 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-18T09:08:42.269 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-18T09:08:42.270 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 0 2023-10-18T09:08:42.270 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:42.270 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-18T09:08:42.270 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "a" 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-18T09:08:42.271 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-18T09:08:42.272 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.272 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-18T09:08:42.272 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:42.272 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-18T09:08:42.272 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-18T09:08:42.273 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-18T09:08:42.273 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-18T09:08:42.276 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-18T09:08:42.277 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-18T09:08:42.277 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.277 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-18T09:08:42.277 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-18T09:08:42.277 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-18T09:08:42.278 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-18T09:08:42.279 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.279 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-18T09:08:42.279 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:42.279 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-18T09:08:42.279 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-18T09:08:42.280 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.280 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-18T09:08:42.280 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T09:08:42.280 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-18T09:08:42.281 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-18T09:08:42.281 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-18T09:08:42.281 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-18T09:08:42.281 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "restful" 2023-10-18T09:08:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ], 2023-10-18T09:08:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-18T09:08:42.282 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-18T09:08:42.283 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "modified": "2023-10-18T09:08:20.266489+0000", 2023-10-18T09:08:42.283 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-18T09:08:42.283 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout }, 2023-10-18T09:08:42.283 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-18T09:08:42.283 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:08:42.284 INFO:teuthology.orchestra.run.smithi061.stdout:mgr is available 2023-10-18T09:08:42.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1997217177' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-18T09:08:43.549 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:43.549 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [global] 2023-10-18T09:08:43.550 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout fsid = c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:43.550 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-18T09:08:43.550 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-18T09:08:43.550 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-18T09:08:43.550 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout [osd] 2023-10-18T09:08:43.551 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-18T09:08:43.552 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-18T09:08:43.552 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-18T09:08:43.552 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-18T09:08:43.552 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-18T09:08:43.552 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling cephadm module... 2023-10-18T09:08:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1542934359' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-18T09:08:44.729 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1320721508' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-18T09:08:44.730 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: ignoring --setuser ceph since I am not root 2023-10-18T09:08:44.730 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: ignoring --setgroup ceph since I am not root 2023-10-18T09:08:45.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:44.726+0000 7fb0a18df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T09:08:45.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:44.836+0000 7fb0a18df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T09:08:45.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:44.959+0000 7fb0a18df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T09:08:45.462 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:45.090+0000 7fb0a18df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T09:08:45.463 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:45.331+0000 7fb0a18df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T09:08:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1320721508' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-18T09:08:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-mon[98573]: mgrmap e5: x(active, since 6s) 2023-10-18T09:08:46.122 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:45.813+0000 7fb0a18df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T09:08:46.122 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:45.998+0000 7fb0a18df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T09:08:46.306 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:08:46.306 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-18T09:08:46.306 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T09:08:46.307 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-18T09:08:46.307 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-18T09:08:46.307 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:08:46.307 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for the mgr to restart... 2023-10-18T09:08:46.307 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mgr epoch 5... 2023-10-18T09:08:46.463 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:46.120+0000 7fb0a18df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T09:08:46.463 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:46.250+0000 7fb0a18df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T09:08:46.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3069977383' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-18T09:08:47.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:46.830+0000 7fb0a18df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T09:08:48.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:47.717+0000 7fb0a18df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T09:08:48.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:47.828+0000 7fb0a18df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T09:08:49.213 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:48.845+0000 7fb0a18df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:08:49.213 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:48.972+0000 7fb0a18df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T09:08:49.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:49.210+0000 7fb0a18df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:08:49.885 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:49.596+0000 7fb0a18df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T09:08:50.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:49.882+0000 7fb0a18df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T09:08:50.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:50.084+0000 7fb0a18df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T09:08:50.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:50.444+0000 7fb0a18df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T09:08:51.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:51.416+0000 7fb0a18df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T09:08:51.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:51.598+0000 7fb0a18df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T09:08:52.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:51.858+0000 7fb0a18df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T09:08:52.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:52.010+0000 7fb0a18df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T09:08:52.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:52.301+0000 7fb0a18df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T09:08:53.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:08:53.422+0000 7fb0a18df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T09:08:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: Active manager daemon x restarted 2023-10-18T09:08:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: Activating manager daemon x 2023-10-18T09:08:53.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: osdmap e2: 0 total, 0 up, 0 in 2023-10-18T09:08:53.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: mgrmap e6: x(active, starting, since 0.0033951s) 2023-10-18T09:08:53.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:08:53.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T09:08:53.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T09:08:53.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T09:08:53.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T09:08:53.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: Manager daemon x is now available 2023-10-18T09:08:53.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:53.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:53.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:08:53.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:53 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:08:54.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: Found migration_current of "None". Setting to last migration. 2023-10-18T09:08:54.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:08:54.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T09:08:54.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T09:08:54.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:54.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:54.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:08:54.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:54 smithi061 ceph-mon[98573]: mgrmap e7: x(active, since 1.00628s) 2023-10-18T09:08:54.899 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:08:54.900 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-18T09:08:54.900 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-18T09:08:54.900 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:08:54.900 INFO:teuthology.orchestra.run.smithi061.stdout:mgr epoch 5 is available 2023-10-18T09:08:54.900 INFO:teuthology.orchestra.run.smithi061.stdout:Setting orchestrator backend to cephadm... 2023-10-18T09:08:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: [18/Oct/2023:09:08:54] ENGINE Bus STARTING 2023-10-18T09:08:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: [18/Oct/2023:09:08:54] ENGINE Serving on http://172.21.15.61:8765 2023-10-18T09:08:55.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: [18/Oct/2023:09:08:54] ENGINE Serving on https://172.21.15.61:7150 2023-10-18T09:08:55.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: [18/Oct/2023:09:08:54] ENGINE Bus STARTED 2023-10-18T09:08:55.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-18T09:08:55.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:55 smithi061 ceph-mon[98573]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-18T09:08:56.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:56 smithi061 ceph-mon[98573]: mgrmap e8: x(active, since 2s) 2023-10-18T09:08:56.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:56 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:56.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:56 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:08:57.456 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-18T09:08:57.456 INFO:teuthology.orchestra.run.smithi061.stdout:Generating ssh key... 2023-10-18T09:08:57.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:57 smithi061 ceph-mon[98573]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:08:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-mon[98573]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:08:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:58.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:08:58.754 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: Generating public/private rsa key pair. 2023-10-18T09:08:58.754 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: Your identification has been saved in /tmp/tmpn5725cln/key. 2023-10-18T09:08:58.754 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: Your public key has been saved in /tmp/tmpn5725cln/key.pub. 2023-10-18T09:08:58.755 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: The key fingerprint is: 2023-10-18T09:08:58.755 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: SHA256:cV5BwA0ZBgezawipcBwcnHi6xqiF/qgKdRMIyYl/TOY ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:08:58.755 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: The key's randomart image is: 2023-10-18T09:08:58.755 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: +---[RSA 3072]----+ 2023-10-18T09:08:58.755 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |+=+o ++*Bo | 2023-10-18T09:08:58.756 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |+==oo. =o .. | 2023-10-18T09:08:58.756 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |.++=+ o . . | 2023-10-18T09:08:58.756 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |.o..Eo . = . | 2023-10-18T09:08:58.756 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |ooo.o . S . | 2023-10-18T09:08:58.756 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |++.. . . | 2023-10-18T09:08:58.757 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |=. | 2023-10-18T09:08:58.757 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |o.. | 2023-10-18T09:08:58.757 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: |=... | 2023-10-18T09:08:58.757 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:08:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: +----[SHA256]-----+ 2023-10-18T09:08:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:59 smithi061 ceph-mon[98573]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:08:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:08:59 smithi061 ceph-mon[98573]: Generating ssh key... 2023-10-18T09:09:00.112 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:09:00.112 INFO:teuthology.orchestra.run.smithi061.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-18T09:09:00.113 INFO:teuthology.orchestra.run.smithi061.stdout:Adding key to root@localhost authorized_keys... 2023-10-18T09:09:00.115 INFO:teuthology.orchestra.run.smithi061.stdout:Adding host smithi061... 2023-10-18T09:09:00.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:00 smithi061 ceph-mon[98573]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:02.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:02 smithi061 ceph-mon[98573]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi061", "addr": "172.21.15.61", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:03.717 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:03 smithi061 ceph-mon[98573]: Deploying cephadm binary to smithi061 2023-10-18T09:09:04.799 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout Added host 'smithi061' with addr '172.21.15.61' 2023-10-18T09:09:04.800 INFO:teuthology.orchestra.run.smithi061.stdout:Deploying unmanaged mon service... 2023-10-18T09:09:05.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:05 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:05.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:05 smithi061 ceph-mon[98573]: Added host smithi061 2023-10-18T09:09:05.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:05 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:06.180 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-18T09:09:06.180 INFO:teuthology.orchestra.run.smithi061.stdout:Deploying unmanaged mgr service... 2023-10-18T09:09:06.802 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:06 smithi061 ceph-mon[98573]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:06.803 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:06 smithi061 ceph-mon[98573]: Saving service mon spec with placement count:5 2023-10-18T09:09:06.803 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:06 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:07.587 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-18T09:09:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:08 smithi061 ceph-mon[98573]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:08 smithi061 ceph-mon[98573]: Saving service mgr spec with placement count:2 2023-10-18T09:09:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:08 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:08 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:08 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:09.679 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:09 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3306155959' entity='client.admin' 2023-10-18T09:09:10.142 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling the dashboard module... 2023-10-18T09:09:11.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:10 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/35699157' entity='client.admin' 2023-10-18T09:09:12.001 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:11 smithi061 ceph-mon[98573]: from='mgr.14118 172.21.15.61:0/1948057233' entity='mgr.x' 2023-10-18T09:09:12.001 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:11 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1670087113' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-18T09:09:12.001 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:11 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: ignoring --setuser ceph since I am not root 2023-10-18T09:09:12.001 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:11 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: ignoring --setgroup ceph since I am not root 2023-10-18T09:09:12.002 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:11 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:11.998+0000 7fb19c2a1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T09:09:12.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:12.108+0000 7fb19c2a1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T09:09:12.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:12.231+0000 7fb19c2a1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T09:09:12.624 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:12.364+0000 7fb19c2a1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T09:09:13.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1670087113' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-18T09:09:13.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-mon[98573]: mgrmap e9: x(active, since 18s) 2023-10-18T09:09:13.004 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:12 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:12.631+0000 7fb19c2a1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T09:09:13.421 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:13 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:13.115+0000 7fb19c2a1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T09:09:13.422 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:13 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:13.296+0000 7fb19c2a1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T09:09:13.607 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:09:13.608 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-18T09:09:13.608 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-18T09:09:13.608 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-18T09:09:13.608 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-18T09:09:13.609 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:09:13.609 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for the mgr to restart... 2023-10-18T09:09:13.609 INFO:teuthology.orchestra.run.smithi061.stdout:Waiting for mgr epoch 9... 2023-10-18T09:09:13.698 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:13 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:13.418+0000 7fb19c2a1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T09:09:13.698 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:13 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:13.552+0000 7fb19c2a1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T09:09:13.957 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:13 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/139703079' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-18T09:09:14.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:14 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:14.138+0000 7fb19c2a1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T09:09:15.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:15 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:15.024+0000 7fb19c2a1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T09:09:15.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:15 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:15.134+0000 7fb19c2a1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T09:09:16.501 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:16.134+0000 7fb19c2a1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:09:16.502 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:16.260+0000 7fb19c2a1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T09:09:16.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:16.499+0000 7fb19c2a1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:09:17.173 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:16.882+0000 7fb19c2a1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T09:09:17.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:17.170+0000 7fb19c2a1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T09:09:17.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:17.373+0000 7fb19c2a1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T09:09:18.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:17.737+0000 7fb19c2a1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T09:09:19.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:18.713+0000 7fb19c2a1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T09:09:19.004 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:18.895+0000 7fb19c2a1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T09:09:19.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:19.158+0000 7fb19c2a1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T09:09:19.503 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:19.312+0000 7fb19c2a1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T09:09:20.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:19.605+0000 7fb19c2a1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T09:09:21.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:09:20.737+0000 7fb19c2a1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T09:09:21.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: Active manager daemon x restarted 2023-10-18T09:09:21.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: Activating manager daemon x 2023-10-18T09:09:21.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: osdmap e3: 0 total, 0 up, 0 in 2023-10-18T09:09:21.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: mgrmap e10: x(active, starting, since 0.00681165s) 2023-10-18T09:09:21.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:09:21.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: Manager daemon x is now available 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:21.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:22.050 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-18T09:09:22.050 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-18T09:09:22.050 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:21 smithi061 ceph-mon[98573]: mgrmap e11: x(active, since 1.00708s) 2023-10-18T09:09:22.403 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout { 2023-10-18T09:09:22.403 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-18T09:09:22.404 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-18T09:09:22.404 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout } 2023-10-18T09:09:22.404 INFO:teuthology.orchestra.run.smithi061.stdout:mgr epoch 9 is available 2023-10-18T09:09:22.404 INFO:teuthology.orchestra.run.smithi061.stdout:Generating a dashboard self-signed certificate... 2023-10-18T09:09:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:22 smithi061 ceph-mon[98573]: [18/Oct/2023:09:09:21] ENGINE Bus STARTING 2023-10-18T09:09:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:22 smithi061 ceph-mon[98573]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-18T09:09:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:22 smithi061 ceph-mon[98573]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-18T09:09:23.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:22 smithi061 ceph-mon[98573]: [18/Oct/2023:09:09:21] ENGINE Serving on https://172.21.15.61:7150 2023-10-18T09:09:23.893 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-18T09:09:23.893 INFO:teuthology.orchestra.run.smithi061.stdout:Creating initial admin user... 2023-10-18T09:09:24.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: [18/Oct/2023:09:09:21] ENGINE Serving on http://172.21.15.61:8765 2023-10-18T09:09:24.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: [18/Oct/2023:09:09:21] ENGINE Bus STARTED 2023-10-18T09:09:24.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: mgrmap e12: x(active, since 2s) 2023-10-18T09:09:24.194 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:24.194 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:24.194 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:25.148 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:24 smithi061 ceph-mon[98573]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:25.149 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:25.475 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BgOZYH1D1o/ct3gHHJm89u6evghgWDTJVBuZJOMPD3dEifFKa/FGi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1697620164, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-18T09:09:25.475 INFO:teuthology.orchestra.run.smithi061.stdout:Fetching dashboard port number... 2023-10-18T09:09:26.127 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:25 smithi061 ceph-mon[98573]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:26.127 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:26.854 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stdout 8443 2023-10-18T09:09:26.873 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-18T09:09:26.874 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout disabled 2023-10-18T09:09:26.890 INFO:teuthology.orchestra.run.smithi061.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-18T09:09:26.890 INFO:teuthology.orchestra.run.smithi061.stdout:systemctl: stdout inactive 2023-10-18T09:09:26.891 INFO:teuthology.orchestra.run.smithi061.stdout:firewalld.service is not enabled 2023-10-18T09:09:26.891 INFO:teuthology.orchestra.run.smithi061.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-18T09:09:26.893 INFO:teuthology.orchestra.run.smithi061.stdout:Ceph Dashboard is now available at: 2023-10-18T09:09:26.893 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:26.893 INFO:teuthology.orchestra.run.smithi061.stdout: URL: https://smithi061.front.sepia.ceph.com:8443/ 2023-10-18T09:09:26.894 INFO:teuthology.orchestra.run.smithi061.stdout: User: admin 2023-10-18T09:09:26.894 INFO:teuthology.orchestra.run.smithi061.stdout: Password: pv9qxawruh 2023-10-18T09:09:26.894 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:26.895 INFO:teuthology.orchestra.run.smithi061.stdout:Saving cluster configuration to /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config directory 2023-10-18T09:09:26.895 INFO:teuthology.orchestra.run.smithi061.stdout:Enabling autotune for osd_memory_target 2023-10-18T09:09:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:26 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3256141662' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-18T09:09:28.738 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:28 smithi061 ceph-mon[98573]: mgrmap e13: x(active, since 6s) 2023-10-18T09:09:29.749 INFO:teuthology.orchestra.run.smithi061.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-18T09:09:29.749 INFO:teuthology.orchestra.run.smithi061.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-18T09:09:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.750 INFO:teuthology.orchestra.run.smithi061.stdout:Or, if you are only running a single cluster on this host: 2023-10-18T09:09:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-18T09:09:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.751 INFO:teuthology.orchestra.run.smithi061.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-18T09:09:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: ceph telemetry on 2023-10-18T09:09:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.754 INFO:teuthology.orchestra.run.smithi061.stdout:For more information see: 2023-10-18T09:09:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-18T09:09:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:29.755 INFO:teuthology.orchestra.run.smithi061.stdout:Bootstrap complete. 2023-10-18T09:09:29.809 INFO:tasks.cephadm:Fetching config... 2023-10-18T09:09:29.809 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:09:29.809 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-18T09:09:29.855 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-18T09:09:29.855 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:09:29.856 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-18T09:09:29.943 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-18T09:09:29.943 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:09:29.943 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-18T09:09:30.055 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-18T09:09:30.056 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:09:30.056 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-18T09:09:30.139 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-18T09:09:30.139 DEBUG:teuthology.orchestra.run.smithi061:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T09:09:30.300 INFO:teuthology.orchestra.run.smithi061.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:09:30.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T09:09:30.378 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:09:30.391 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-18T09:09:30.436 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC60Nid3LEHstEFpxzx4bUYUQ+qGsrIn/4kO//MFLHb6q+9HYcI1x455CruwDYQI5gdt6HIYX0QS5wWaTy+YVe/RkBnGKYRGqe4B4J1Kh2JtlGmPhUNZZe6JtwFO8vwEvigALCGfWVZ9w/0Q1x+neERkxRnU1UvHy0wUWG4/KuB2FKlK45SFnTZhEt/qpZqA9Ou7kgY+HRkPuKQAO+KI9JzKs4I8JrbmGWqq1SIDJR4VTdzcd/eGpolcLKq1EW8vgTLnlODUQXFMx9VScmENzr/MDd0k+D5OE0TPQYj9FzfPce4OEll5xLs+w26ixIN6Y6WT7QsXkv/v1ijINsAIMY/TDIUXevA6uJlzb9r/+UDDk++PmUQhnWWwAJaAjy/Awnmd74a2PTkw1wD3olvH89rNwG6I397loZPMyHDZYE3HDx6eMpZHjBueI9acEDlpIps7ZU8QJHcrxGPkcX9CUV9KFikeobcviDYxnyvKqDJkjxIJAv29wOwO4iQC8OfJMU= ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:09:30.449 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-18T09:09:30.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1553572215' entity='client.admin' 2023-10-18T09:09:31.191 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:33.614 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:33.614 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:33.615 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi061", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:09:33.615 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:33.615 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:33.615 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:33.979 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-18T09:09:33.979 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-18T09:09:34.722 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:34.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1323115800' entity='client.admin' 2023-10-18T09:09:34.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:34 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:34.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:34 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:37.011 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2023-10-18T09:09:37.012 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:09:37.012 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2023-10-18T09:09:37.032 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:09:37.033 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:37.093 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2023-10-18T09:09:37.093 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi110 2023-10-18T09:09:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:37.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:09:37.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:09:37.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:37 smithi061 ceph-mon[98573]: Updating smithi061:/etc/ceph/ceph.conf 2023-10-18T09:09:37.865 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:38.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:38 smithi061 ceph-mon[98573]: Updating smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:09:38.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:38 smithi061 ceph-mon[98573]: Updating smithi061:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:40.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:39 smithi061 ceph-mon[98573]: Updating smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:09:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:41.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:40 smithi061 ceph-mon[98573]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:42.236 INFO:teuthology.orchestra.run.smithi061.stdout:Added host 'smithi110' with addr '172.21.15.110' 2023-10-18T09:09:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:41 smithi061 ceph-mon[98573]: Deploying cephadm binary to smithi110 2023-10-18T09:09:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:41 smithi061 ceph-mon[98573]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:42.763 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-18T09:09:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:43 smithi061 ceph-mon[98573]: Added host smithi110 2023-10-18T09:09:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:43.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:43.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:43 smithi061 ceph-mon[98573]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:44.471 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:44 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:45.268 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:45.269 INFO:teuthology.orchestra.run.smithi061.stdout:[{"addr": "172.21.15.61", "hostname": "smithi061", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2023-10-18T09:09:45.791 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2023-10-18T09:09:45.791 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:09:45.792 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2023-10-18T09:09:45.812 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:09:45.812 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:45.872 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2023-10-18T09:09:45.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi170 2023-10-18T09:09:45.935 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:45.935 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:45 smithi061 ceph-mon[98573]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:46.613 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:46.868 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:46 smithi061 ceph-mon[98573]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:09:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:47.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:09:47.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:09:47.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:09:49.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:48 smithi061 ceph-mon[98573]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:09:49.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:48 smithi061 ceph-mon[98573]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:09:49.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:49.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:49.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:50.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:49 smithi061 ceph-mon[98573]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:49 smithi061 ceph-mon[98573]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:09:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:49 smithi061 ceph-mon[98573]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:09:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:49 smithi061 ceph-mon[98573]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:50.841 INFO:teuthology.orchestra.run.smithi061.stdout:Added host 'smithi170' with addr '172.21.15.170' 2023-10-18T09:09:50.865 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:50 smithi061 ceph-mon[98573]: Deploying cephadm binary to smithi170 2023-10-18T09:09:51.404 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-18T09:09:52.149 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:51 smithi061 ceph-mon[98573]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:51 smithi061 ceph-mon[98573]: Added host smithi170 2023-10-18T09:09:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:09:52.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:53.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:53.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:52 smithi061 ceph-mon[98573]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:54.034 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:09:54.035 INFO:teuthology.orchestra.run.smithi061.stdout:[{"addr": "172.21.15.61", "hostname": "smithi061", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2023-10-18T09:09:54.462 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:54.537 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-18T09:09:54.537 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd crush tunables default 2023-10-18T09:09:55.278 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:09:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:55 smithi061 ceph-mon[98573]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:09:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:55 smithi061 ceph-mon[98573]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:57.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:09:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:09:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:09:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:09:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:09:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:09:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:09:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:09:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T09:09:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:adjusted tunables profile to default 2023-10-18T09:09:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:58 smithi061 ceph-mon[98573]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:09:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:58 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T09:09:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:58 smithi061 ceph-mon[98573]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:09:58.668 INFO:tasks.cephadm:Adding mon.a on smithi061 2023-10-18T09:09:58.668 INFO:tasks.cephadm:Adding mon.b on smithi110 2023-10-18T09:09:58.669 INFO:tasks.cephadm:Adding mon.c on smithi170 2023-10-18T09:09:58.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch apply mon '3;smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c' 2023-10-18T09:09:59.265 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:09:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:09:59 smithi061 ceph-mon[98573]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:00.674 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2023-10-18T09:10:01.134 DEBUG:teuthology.orchestra.run.smithi110:mon.b> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.b.service 2023-10-18T09:10:01.137 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.c.service 2023-10-18T09:10:01.139 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T09:10:01.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T09:10:01.170 INFO:journalctl@ceph.mon.c.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:10:01.173 INFO:journalctl@ceph.mon.b.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:10:02.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:02.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: Saving service mon spec with placement smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c;count:3 2023-10-18T09:10:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: Deploying daemon mon.c on smithi170 2023-10-18T09:10:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:01 smithi061 ceph-mon[98573]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:02.115 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:03 smithi170 systemd[1]: Starting Ceph mon.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:10:04.194 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:10:04.195 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","modified":"2023-10-18T09:08:15.555387Z","created":"2023-10-18T09:08:15.555387Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-18T09:10:04.196 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2023-10-18T09:10:04.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:03 smithi061 ceph-mon[98573]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 podman[135087]: 2023-10-18T09:10:04.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T09:10:04.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 7 2023-10-18T09:10:04.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pidfile_write: ignore empty --pid-file 2023-10-18T09:10:04.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: load: jerasure load: lrc 2023-10-18T09:10:04.720 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: RocksDB version: 7.9.2 2023-10-18T09:10:04.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Git sha 0 2023-10-18T09:10:04.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T09:10:04.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: DB SUMMARY 2023-10-18T09:10:04.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: DB Session ID: MWTQWDWAD2MY10JO895A 2023-10-18T09:10:04.721 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: CURRENT file: CURRENT 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.error_if_exists: 0 2023-10-18T09:10:04.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.create_if_missing: 0 2023-10-18T09:10:04.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.paranoid_checks: 1 2023-10-18T09:10:04.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T09:10:04.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T09:10:04.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T09:10:04.723 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.env: 0x55702a48fda0 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.info_log: 0x55702b86d660 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.statistics: (nil) 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.use_fsync: 0 2023-10-18T09:10:04.724 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_log_file_size: 0 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_fallocate: 1 2023-10-18T09:10:04.725 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T09:10:04.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T09:10:04.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.use_direct_reads: 0 2023-10-18T09:10:04.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T09:10:04.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T09:10:04.726 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.db_log_dir: 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.wal_dir: 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T09:10:04.727 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.write_buffer_manager: 0x55702b825360 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T09:10:04.728 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T09:10:04.729 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T09:10:04.730 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.unordered_write: 0 2023-10-18T09:10:04.730 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T09:10:04.730 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T09:10:04.730 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T09:10:04.730 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T09:10:04.731 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.row_cache: None 2023-10-18T09:10:04.731 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.wal_filter: None 2023-10-18T09:10:04.731 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T09:10:04.731 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T09:10:04.731 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.two_write_queues: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.wal_compression: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.atomic_flush: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T09:10:04.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.log_readahead_size: 0 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T09:10:04.733 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T09:10:04.735 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T09:10:04.736 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_background_jobs: 2 2023-10-18T09:10:04.736 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_background_compactions: -1 2023-10-18T09:10:04.736 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_subcompactions: 1 2023-10-18T09:10:04.736 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T09:10:04.736 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T09:10:04.737 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T09:10:04.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_open_files: -1 2023-10-18T09:10:04.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T09:10:04.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T09:10:04.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T09:10:04.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_background_flushes: -1 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Compression algorithms supported: 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kZSTD supported: 0 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kXpressCompression supported: 0 2023-10-18T09:10:04.739 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T09:10:04.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kZlibCompression supported: 1 2023-10-18T09:10:04.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kSnappyCompression supported: 1 2023-10-18T09:10:04.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kLZ4Compression supported: 1 2023-10-18T09:10:04.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: kBZip2Compression supported: 0 2023-10-18T09:10:04.740 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.merge_operator: 2023-10-18T09:10:04.741 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_filter: None 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55702b86d760) 2023-10-18T09:10:04.742 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2023-10-18T09:10:04.743 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x55702b8c31f0 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2023-10-18T09:10:04.744 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2023-10-18T09:10:04.745 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2023-10-18T09:10:04.745 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2023-10-18T09:10:04.745 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2023-10-18T09:10:04.745 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2023-10-18T09:10:04.745 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2023-10-18T09:10:04.746 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2023-10-18T09:10:04.747 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2023-10-18T09:10:04.748 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2023-10-18T09:10:04.749 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T09:10:04.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T09:10:04.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T09:10:04.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression: NoCompression 2023-10-18T09:10:04.749 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.num_levels: 7 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T09:10:04.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:10:04.751 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T09:10:04.752 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T09:10:04.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T09:10:04.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:10:04.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:10:04.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T09:10:04.753 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T09:10:04.754 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T09:10:04.755 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T09:10:04.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T09:10:04.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T09:10:04.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T09:10:04.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T09:10:04.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T09:10:04.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T09:10:04.758 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.table_properties_collectors: 2023-10-18T09:10:04.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.inplace_update_support: 0 2023-10-18T09:10:04.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T09:10:04.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T09:10:04.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T09:10:04.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T09:10:04.760 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.bloom_locality: 0 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.max_successive_merges: 0 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T09:10:04.761 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.ttl: 2592000 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enable_blob_files: false 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.min_blob_size: 0 2023-10-18T09:10:04.762 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T09:10:04.763 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 036d10c4-8b3b-4416-aa2b-95f51ce52508 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620204435109, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-18T09:10:04.764 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-18T09:10:04.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620204435524, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697620204, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "036d10c4-8b3b-4416-aa2b-95f51ce52508", "db_session_id": "MWTQWDWAD2MY10JO895A", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-18T09:10:04.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620204435591, "job": 1, "event": "recovery_finished"} 2023-10-18T09:10:04.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-18T09:10:04.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T09:10:04.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55702b960000 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: DB pointer 0x55702b94c000 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:10:04.766 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:10:04.767 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T09:10:04.768 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T09:10:04.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2023-10-18T09:10:04.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:10:04.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T09:10:04.769 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T09:10:04.770 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:10:04.771 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T09:10:04.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x55702b8c31f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-10-18T09:10:04.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-18T09:10:04.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.772 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T09:10:04.772 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-10-18T09:10:04.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-10-18T09:10:04.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:04.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(???) e0 preinit fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:04.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 bash[135087]: b42666a530eb1635eefe715f5027afa701a8780b62351df32d2112c84e229d55 2023-10-18T09:10:04.773 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).mds e1 new map 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).mds e1 print_map 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2023-10-18T09:10:04.774 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2023-10-18T09:10:04.775 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2023-10-18T09:10:04.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-18T09:10:04.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-18T09:10:04.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-18T09:10:04.775 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-18T09:10:04.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-18T09:10:04.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-18T09:10:04.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T09:10:04.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:04.776 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Added host smithi110 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.777 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:10:04.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:04.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:04.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:10:04.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:04.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:04.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:10:04.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:04.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Deploying cephadm binary to smithi170 2023-10-18T09:10:04.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Added host smithi170 2023-10-18T09:10:04.782 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:10:04.783 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:10:04.784 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.785 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T09:10:04.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:10:04.786 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Saving service mon spec with placement smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c;count:3 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.787 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:04.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:04.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:04.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:04.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:04.788 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:04.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: Deploying daemon mon.c on smithi170 2023-10-18T09:10:04.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:04.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 ceph-mon[135150]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-18T09:10:04.789 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:04 smithi170 systemd[1]: Started Ceph mon.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:10:05.735 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T09:10:05.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T09:10:06.308 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:10:09.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:09.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: mon.a calling monitor election 2023-10-18T09:10:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: mon.c calling monitor election 2023-10-18T09:10:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: monmap e2: 2 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: fsmap 2023-10-18T09:10:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: mgrmap e13: x(active, since 48s) 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: overall HEALTH_OK 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:09.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:09.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:10.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:10.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: mon.a calling monitor election 2023-10-18T09:10:10.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:10.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: mon.c calling monitor election 2023-10-18T09:10:10.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:10.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:10.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:10.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: monmap e2: 2 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:10.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: fsmap 2023-10-18T09:10:10.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:10.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: mgrmap e13: x(active, since 48s) 2023-10-18T09:10:10.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: overall HEALTH_OK 2023-10-18T09:10:10.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:10.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:10.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:10.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:10.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:10.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:10 smithi170 ceph-mon[135150]: Deploying daemon mon.b on smithi110 2023-10-18T09:10:10.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:10.944 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:10:10.945 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","modified":"2023-10-18T09:10:04.458666Z","created":"2023-10-18T09:08:15.555387Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-18T09:10:10.946 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2023-10-18T09:10:11.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:10 smithi061 ceph-mon[98573]: Deploying daemon mon.b on smithi110 2023-10-18T09:10:11.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:11.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:11 smithi170 ceph-mon[135150]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:11.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:11 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/1350195762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:11 smithi061 ceph-mon[98573]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:11 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/1350195762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 systemd[1]: Starting Ceph mon.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:10:12.270 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-18T09:10:12.270 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-18T09:10:12.588 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 podman[134770]: 2023-10-18T09:10:12.589 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: set uid:gid to 167:167 (ceph:ceph) 2023-10-18T09:10:12.589 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev), process ceph-mon, pid 7 2023-10-18T09:10:12.589 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pidfile_write: ignore empty --pid-file 2023-10-18T09:10:12.590 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: load: jerasure load: lrc 2023-10-18T09:10:12.590 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: RocksDB version: 7.9.2 2023-10-18T09:10:12.590 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Git sha 0 2023-10-18T09:10:12.590 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Compile date 2023-10-16 22:08:19 2023-10-18T09:10:12.590 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: DB SUMMARY 2023-10-18T09:10:12.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: DB Session ID: HUKZ9DCZBFWS6H7J5VOZ 2023-10-18T09:10:12.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: CURRENT file: CURRENT 2023-10-18T09:10:12.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: IDENTITY file: IDENTITY 2023-10-18T09:10:12.591 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-18T09:10:12.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-18T09:10:12.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-18T09:10:12.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.error_if_exists: 0 2023-10-18T09:10:12.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.create_if_missing: 0 2023-10-18T09:10:12.592 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.paranoid_checks: 1 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.env: 0x5584b66b3da0 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.fs: PosixFileSystem 2023-10-18T09:10:12.593 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.info_log: 0x5584b7d51660 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_file_opening_threads: 16 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.statistics: (nil) 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.use_fsync: 0 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_log_file_size: 0 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-18T09:10:12.594 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-18T09:10:12.595 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.keep_log_file_num: 1000 2023-10-18T09:10:12.595 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.recycle_log_file_num: 0 2023-10-18T09:10:12.595 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_fallocate: 1 2023-10-18T09:10:12.595 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_mmap_reads: 0 2023-10-18T09:10:12.595 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_mmap_writes: 0 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.use_direct_reads: 0 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.create_missing_column_families: 0 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.db_log_dir: 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.wal_dir: 2023-10-18T09:10:12.596 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-18T09:10:12.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-18T09:10:12.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-18T09:10:12.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-18T09:10:12.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-18T09:10:12.597 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.advise_random_on_open: 1 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.db_write_buffer_size: 0 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.write_buffer_manager: 0x5584b7d09360 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-18T09:10:12.598 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-18T09:10:12.599 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.rate_limiter: (nil) 2023-10-18T09:10:12.599 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-18T09:10:12.599 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.wal_recovery_mode: 2 2023-10-18T09:10:12.599 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enable_thread_tracking: 0 2023-10-18T09:10:12.600 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enable_pipelined_write: 0 2023-10-18T09:10:12.600 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.unordered_write: 0 2023-10-18T09:10:12.600 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-18T09:10:12.600 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-18T09:10:12.600 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-18T09:10:12.601 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-18T09:10:12.601 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.row_cache: None 2023-10-18T09:10:12.601 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.wal_filter: None 2023-10-18T09:10:12.601 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-18T09:10:12.601 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_ingest_behind: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.two_write_queues: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.manual_wal_flush: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.wal_compression: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.atomic_flush: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-18T09:10:12.602 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-18T09:10:12.603 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-18T09:10:12.603 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.log_readahead_size: 0 2023-10-18T09:10:12.603 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-18T09:10:12.603 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.best_efforts_recovery: 0 2023-10-18T09:10:12.603 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-18T09:10:12.604 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-18T09:10:12.604 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.allow_data_in_errors: 0 2023-10-18T09:10:12.604 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.db_host_id: __hostname__ 2023-10-18T09:10:12.605 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-18T09:10:12.605 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_background_jobs: 2 2023-10-18T09:10:12.606 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_background_compactions: -1 2023-10-18T09:10:12.606 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_subcompactions: 1 2023-10-18T09:10:12.606 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-18T09:10:12.606 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_total_wal_size: 0 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-18T09:10:12.607 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_open_files: -1 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bytes_per_sync: 0 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_readahead_size: 0 2023-10-18T09:10:12.608 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_background_flushes: -1 2023-10-18T09:10:12.609 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Compression algorithms supported: 2023-10-18T09:10:12.609 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-18T09:10:12.609 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kZSTD supported: 0 2023-10-18T09:10:12.609 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kXpressCompression supported: 0 2023-10-18T09:10:12.609 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kLZ4HCCompression supported: 1 2023-10-18T09:10:12.610 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kZlibCompression supported: 1 2023-10-18T09:10:12.610 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kSnappyCompression supported: 1 2023-10-18T09:10:12.610 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kLZ4Compression supported: 1 2023-10-18T09:10:12.610 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: kBZip2Compression supported: 0 2023-10-18T09:10:12.610 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-18T09:10:12.611 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-18T09:10:12.611 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-18T09:10:12.611 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-18T09:10:12.611 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-18T09:10:12.611 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.merge_operator: 2023-10-18T09:10:12.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_filter: None 2023-10-18T09:10:12.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_filter_factory: None 2023-10-18T09:10:12.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.sst_partitioner_factory: None 2023-10-18T09:10:12.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-18T09:10:12.612 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5584b7d51760) 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout: cache_index_and_filter_blocks: 1 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout: pin_top_level_index_and_filter: 1 2023-10-18T09:10:12.613 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_type: 0 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: data_block_index_type: 0 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_shortening: 1 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: checksum: 4 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: no_block_cache: 0 2023-10-18T09:10:12.614 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache: 0x5584b7da71f0 2023-10-18T09:10:12.615 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_name: BinnedLRUCache 2023-10-18T09:10:12.615 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_options: 2023-10-18T09:10:12.615 INFO:journalctl@ceph.mon.b.smithi110.stdout: capacity : 536870912 2023-10-18T09:10:12.615 INFO:journalctl@ceph.mon.b.smithi110.stdout: num_shard_bits : 4 2023-10-18T09:10:12.615 INFO:journalctl@ceph.mon.b.smithi110.stdout: strict_capacity_limit : 0 2023-10-18T09:10:12.616 INFO:journalctl@ceph.mon.b.smithi110.stdout: high_pri_pool_ratio: 0.000 2023-10-18T09:10:12.616 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_cache_compressed: (nil) 2023-10-18T09:10:12.616 INFO:journalctl@ceph.mon.b.smithi110.stdout: persistent_cache: (nil) 2023-10-18T09:10:12.616 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_size: 4096 2023-10-18T09:10:12.616 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_size_deviation: 10 2023-10-18T09:10:12.617 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_restart_interval: 16 2023-10-18T09:10:12.617 INFO:journalctl@ceph.mon.b.smithi110.stdout: index_block_restart_interval: 1 2023-10-18T09:10:12.617 INFO:journalctl@ceph.mon.b.smithi110.stdout: metadata_block_size: 4096 2023-10-18T09:10:12.617 INFO:journalctl@ceph.mon.b.smithi110.stdout: partition_filters: 0 2023-10-18T09:10:12.617 INFO:journalctl@ceph.mon.b.smithi110.stdout: use_delta_encoding: 1 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: filter_policy: bloomfilter 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: whole_key_filtering: 1 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: verify_compression: 0 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: read_amp_bytes_per_bit: 0 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: format_version: 5 2023-10-18T09:10:12.618 INFO:journalctl@ceph.mon.b.smithi110.stdout: enable_index_compression: 1 2023-10-18T09:10:12.619 INFO:journalctl@ceph.mon.b.smithi110.stdout: block_align: 0 2023-10-18T09:10:12.619 INFO:journalctl@ceph.mon.b.smithi110.stdout: max_auto_readahead_size: 262144 2023-10-18T09:10:12.619 INFO:journalctl@ceph.mon.b.smithi110.stdout: prepopulate_block_cache: 0 2023-10-18T09:10:12.619 INFO:journalctl@ceph.mon.b.smithi110.stdout: initial_auto_readahead_size: 8192 2023-10-18T09:10:12.619 INFO:journalctl@ceph.mon.b.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2023-10-18T09:10:12.620 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.write_buffer_size: 33554432 2023-10-18T09:10:12.620 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_write_buffer_number: 2 2023-10-18T09:10:12.620 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression: NoCompression 2023-10-18T09:10:12.620 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression: Disabled 2023-10-18T09:10:12.620 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.prefix_extractor: nullptr 2023-10-18T09:10:12.621 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-18T09:10:12.621 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.num_levels: 7 2023-10-18T09:10:12.621 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-18T09:10:12.621 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-18T09:10:12.621 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-18T09:10:12.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-18T09:10:12.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-18T09:10:12.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-18T09:10:12.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-18T09:10:12.622 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-18T09:10:12.623 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.level: 32767 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.strategy: 0 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-18T09:10:12.624 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.enabled: false 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.target_file_size_base: 67108864 2023-10-18T09:10:12.625 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-18T09:10:12.626 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-18T09:10:12.626 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-18T09:10:12.626 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-18T09:10:12.626 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 bash[134770]: a6933b532edd9dda9c11401dbc8f2204a153701c5ecc93ac9456bc5a0e13073c 2023-10-18T09:10:12.626 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-18T09:10:12.627 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-18T09:10:12.627 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-18T09:10:12.627 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-18T09:10:12.627 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-18T09:10:12.627 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.arena_block_size: 1048576 2023-10-18T09:10:12.628 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-18T09:10:12.629 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-18T09:10:12.629 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.disable_auto_compactions: 0 2023-10-18T09:10:12.629 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-18T09:10:12.629 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-18T09:10:12.629 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-18T09:10:12.630 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-18T09:10:12.630 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-18T09:10:12.630 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-18T09:10:12.630 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-18T09:10:12.630 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.table_properties_collectors: 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.inplace_update_support: 0 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-18T09:10:12.631 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-18T09:10:12.632 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-18T09:10:12.632 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-18T09:10:12.632 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.bloom_locality: 0 2023-10-18T09:10:12.632 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.max_successive_merges: 0 2023-10-18T09:10:12.632 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.paranoid_file_checks: 0 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.force_consistency_checks: 1 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.report_bg_io_stats: 0 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.ttl: 2592000 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-18T09:10:12.633 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enable_blob_files: false 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.min_blob_size: 0 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_file_size: 268435456 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-18T09:10:12.634 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-18T09:10:12.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-18T09:10:12.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-18T09:10:12.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-18T09:10:12.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.blob_file_starting_level: 0 2023-10-18T09:10:12.635 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ef9dc48e-dc2c-487d-b0ce-478fc0b8c7a0 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620212489863, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-18T09:10:12.636 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620212490414, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697620212, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ef9dc48e-dc2c-487d-b0ce-478fc0b8c7a0", "db_session_id": "HUKZ9DCZBFWS6H7J5VOZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-18T09:10:12.637 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697620212490504, "job": 1, "event": "recovery_finished"} 2023-10-18T09:10:12.637 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-18T09:10:12.637 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-18T09:10:12.637 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5584b7e44000 2023-10-18T09:10:12.637 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: DB pointer 0x5584b7e30000 2023-10-18T09:10:12.638 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-18T09:10:12.638 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-18T09:10:12.638 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** DB Stats ** 2023-10-18T09:10:12.638 INFO:journalctl@ceph.mon.b.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:10:12.638 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:12.639 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:12.639 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:10:12.639 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-18T09:10:12.639 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-18T09:10:12.639 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-18T09:10:12.640 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.640 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** Compaction Stats [default] ** 2023-10-18T09:10:12.640 INFO:journalctl@ceph.mon.b.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:10:12.640 INFO:journalctl@ceph.mon.b.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-18T09:10:12.640 INFO:journalctl@ceph.mon.b.smithi110.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:10:12.641 INFO:journalctl@ceph.mon.b.smithi110.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:10:12.641 INFO:journalctl@ceph.mon.b.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:10:12.641 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.641 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** Compaction Stats [default] ** 2023-10-18T09:10:12.641 INFO:journalctl@ceph.mon.b.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-18T09:10:12.642 INFO:journalctl@ceph.mon.b.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-18T09:10:12.642 INFO:journalctl@ceph.mon.b.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-18T09:10:12.642 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.642 INFO:journalctl@ceph.mon.b.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-18T09:10:12.642 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.643 INFO:journalctl@ceph.mon.b.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-18T09:10:12.643 INFO:journalctl@ceph.mon.b.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-18T09:10:12.643 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-18T09:10:12.643 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-18T09:10:12.643 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: Block cache BinnedLRUCache@0x5584b7da71f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-10-18T09:10:12.644 INFO:journalctl@ceph.mon.b.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-18T09:10:12.645 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.645 INFO:journalctl@ceph.mon.b.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-18T09:10:12.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-18T09:10:12.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: using public_addr v2:172.21.15.110:0/0 -> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2023-10-18T09:10:12.645 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: starting mon.b rank -1 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:12.646 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(???) e0 preinit fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:12.646 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).mds e1 new map 2023-10-18T09:10:12.646 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-18T09:10:12.646 INFO:journalctl@ceph.mon.b.smithi110.stdout: e1 2023-10-18T09:10:12.646 INFO:journalctl@ceph.mon.b.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T09:10:12.647 INFO:journalctl@ceph.mon.b.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T09:10:12.647 INFO:journalctl@ceph.mon.b.smithi110.stdout: legacy client fscid: -1 2023-10-18T09:10:12.647 INFO:journalctl@ceph.mon.b.smithi110.stdout: 2023-10-18T09:10:12.647 INFO:journalctl@ceph.mon.b.smithi110.stdout: No filesystems configured 2023-10-18T09:10:12.648 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-18T09:10:12.648 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-18T09:10:12.648 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-18T09:10:12.648 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-18T09:10:12.648 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-18T09:10:12.649 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-18T09:10:12.649 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-18T09:10:12.649 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:12.649 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:12.649 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-18T09:10:12.650 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.650 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Added host smithi110 2023-10-18T09:10:12.650 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.650 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.650 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.651 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:12.652 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.653 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:12.654 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:10:12.654 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:12.654 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.654 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Deploying cephadm binary to smithi170 2023-10-18T09:10:12.654 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Added host smithi170 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.655 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.657 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.657 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:10:12.657 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:12.657 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:12.657 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.659 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-18T09:10:12.659 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-18T09:10:12.659 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2750846879' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-18T09:10:12.659 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:12.659 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:12.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Saving service mon spec with placement smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c;count:3 2023-10-18T09:10:12.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.660 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Deploying daemon mon.c on smithi170 2023-10-18T09:10:12.661 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.662 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.662 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:12.662 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.662 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.a calling monitor election 2023-10-18T09:10:12.662 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.c calling monitor election 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.663 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: monmap e2: 2 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: fsmap 2023-10-18T09:10:12.664 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mgrmap e13: x(active, since 48s) 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: overall HEALTH_OK 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:12.665 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:12.666 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:12.666 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: Deploying daemon mon.b on smithi110 2023-10-18T09:10:12.666 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:12.666 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:12.666 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/1350195762' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:12.667 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 ceph-mon[134818]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-18T09:10:12.667 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:12 smithi110 systemd[1]: Started Ceph mon.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:10:12.837 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:10:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: mon.c calling monitor election 2023-10-18T09:10:17.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: mon.a calling monitor election 2023-10-18T09:10:17.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:17.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:17.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:17.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:17.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:17.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:17.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:17.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: fsmap 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: mgrmap e13: x(active, since 56s) 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T09:10:17.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: mon.b (rank 2) addr [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] is down (out of quorum) 2023-10-18T09:10:17.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:17.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:17.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:17.897 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:17.923 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-18T09:10:17.923 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","modified":"2023-10-18T09:10:12.513884Z","created":"2023-10-18T09:08:15.555387Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-18T09:10:17.925 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2023-10-18T09:10:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: mon.c calling monitor election 2023-10-18T09:10:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: mon.a calling monitor election 2023-10-18T09:10:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:18.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:18.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:18.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: fsmap 2023-10-18T09:10:18.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:18.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: mgrmap e13: x(active, since 56s) 2023-10-18T09:10:18.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T09:10:18.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T09:10:18.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T09:10:18.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: mon.b (rank 2) addr [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] is down (out of quorum) 2023-10-18T09:10:18.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:18.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:18.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:18.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:18.300 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-18T09:10:18.300 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-18T09:10:18.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:18.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:18 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/3756085043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:18.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:18.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:18.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:18 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/3756085043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:18.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.058 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:19.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:10:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: mon.c calling monitor election 2023-10-18T09:10:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: mon.a calling monitor election 2023-10-18T09:10:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:10:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:19.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:19.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: fsmap 2023-10-18T09:10:19.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:19.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: mgrmap e13: x(active, since 56s) 2023-10-18T09:10:19.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-18T09:10:19.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-18T09:10:19.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-18T09:10:19.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: mon.b (rank 2) addr [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] is down (out of quorum) 2023-10-18T09:10:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:19.967 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:19.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/3756085043' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-18T09:10:19.968 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:20.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mon.b calling monitor election 2023-10-18T09:10:20.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Updating smithi061:/etc/ceph/ceph.conf 2023-10-18T09:10:20.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mon.b calling monitor election 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mon.c calling monitor election 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mon.a calling monitor election 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T09:10:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:20.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: fsmap 2023-10-18T09:10:20.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:20.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: mgrmap e13: x(active, since 58s) 2023-10-18T09:10:20.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T09:10:20.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Cluster is now healthy 2023-10-18T09:10:20.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: overall HEALTH_OK 2023-10-18T09:10:20.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mon.b calling monitor election 2023-10-18T09:10:20.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Updating smithi061:/etc/ceph/ceph.conf 2023-10-18T09:10:20.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:10:20.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:10:20.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mon.b calling monitor election 2023-10-18T09:10:20.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mon.c calling monitor election 2023-10-18T09:10:20.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mon.a calling monitor election 2023-10-18T09:10:20.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: fsmap 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: mgrmap e13: x(active, since 58s) 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T09:10:20.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Cluster is now healthy 2023-10-18T09:10:20.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: overall HEALTH_OK 2023-10-18T09:10:20.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mon.b calling monitor election 2023-10-18T09:10:20.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Updating smithi061:/etc/ceph/ceph.conf 2023-10-18T09:10:20.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Updating smithi110:/etc/ceph/ceph.conf 2023-10-18T09:10:20.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Updating smithi170:/etc/ceph/ceph.conf 2023-10-18T09:10:20.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mon.b calling monitor election 2023-10-18T09:10:20.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mon.c calling monitor election 2023-10-18T09:10:20.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mon.a calling monitor election 2023-10-18T09:10:20.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-18T09:10:20.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: monmap e3: 3 mons at {a=[v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0],b=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-10-18T09:10:20.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: fsmap 2023-10-18T09:10:20.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: osdmap e4: 0 total, 0 up, 0 in 2023-10-18T09:10:20.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: mgrmap e13: x(active, since 58s) 2023-10-18T09:10:20.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-18T09:10:20.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Cluster is now healthy 2023-10-18T09:10:20.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: overall HEALTH_OK 2023-10-18T09:10:20.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Updating smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: Updating smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:20.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.896 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:20.970 INFO:teuthology.orchestra.run.smithi061.stdout:# minimal ceph.conf for c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:20.971 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2023-10-18T09:10:20.971 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:10:20.971 INFO:teuthology.orchestra.run.smithi061.stdout: mon_host = [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2023-10-18T09:10:21.447 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-18T09:10:21.447 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:10:21.448 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T09:10:21.525 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:10:21.525 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:21.635 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:10:21.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T09:10:21.673 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:10:21.673 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: Updating smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T09:10:21.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:21.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:21.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: Reconfiguring daemon mon.a on smithi061 2023-10-18T09:10:21.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:21 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2402209544' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.747 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:10:21.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2023-10-18T09:10:21.753 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:10:21.514+0000 7fb155929700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-18T09:10:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: Updating smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:21.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:21.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T09:10:21.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:21.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:21.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: Reconfiguring daemon mon.a on smithi061 2023-10-18T09:10:21.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:21 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2402209544' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.778 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: Updating smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:21.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: Reconfiguring mon.a (unknown last config time)... 2023-10-18T09:10:21.780 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:21.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:21.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: Reconfiguring daemon mon.a on smithi061 2023-10-18T09:10:21.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:21 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2402209544' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:21.784 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:10:21.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-18T09:10:21.857 INFO:tasks.cephadm:Adding mgr.x on smithi061 2023-10-18T09:10:21.857 INFO:tasks.cephadm:Adding mgr.y on smithi110 2023-10-18T09:10:21.858 INFO:tasks.cephadm:Adding mgr.z on smithi170 2023-10-18T09:10:21.858 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch apply mgr '3;smithi061=x;smithi110=y;smithi170=z' 2023-10-18T09:10:22.455 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:10:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: Reconfiguring mon.b (monmap changed)... 2023-10-18T09:10:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:23.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:23.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:23.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: Reconfiguring daemon mon.b on smithi110 2023-10-18T09:10:23.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:23 smithi110 ceph-mon[134818]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:23.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:23.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: Reconfiguring mon.b (monmap changed)... 2023-10-18T09:10:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: Reconfiguring daemon mon.b on smithi110 2023-10-18T09:10:23.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:23 smithi170 ceph-mon[135150]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:23.929 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2023-10-18T09:10:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: Reconfiguring mon.b (monmap changed)... 2023-10-18T09:10:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:24.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:24.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:24.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: Reconfiguring daemon mon.b on smithi110 2023-10-18T09:10:24.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:23 smithi061 ceph-mon[98573]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:24.403 DEBUG:teuthology.orchestra.run.smithi110:mgr.y> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.y.service 2023-10-18T09:10:24.406 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.z.service 2023-10-18T09:10:24.408 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2023-10-18T09:10:24.409 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set client client_mount_timeout 600 2023-10-18T09:10:24.414 INFO:tasks.cephadm:setting config [client] debug_client = 20 2023-10-18T09:10:24.414 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set client debug_client 20 2023-10-18T09:10:24.437 INFO:journalctl@ceph.mgr.z.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:10:24.438 INFO:journalctl@ceph.mgr.y.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:10:24.460 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2023-10-18T09:10:24.460 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set client debug_ms 1 2023-10-18T09:10:24.505 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2023-10-18T09:10:24.506 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set client rados_mon_op_timeout 900 2023-10-18T09:10:24.552 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2023-10-18T09:10:24.553 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set client rados_osd_op_timeout 900 2023-10-18T09:10:24.600 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2023-10-18T09:10:24.601 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_mds 20 2023-10-18T09:10:24.649 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2023-10-18T09:10:24.649 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_mds_balancer 20 2023-10-18T09:10:24.697 INFO:tasks.cephadm:setting config [mds] debug_ms = 1 2023-10-18T09:10:24.698 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds debug_ms 1 2023-10-18T09:10:24.731 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.731 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.731 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: Reconfiguring mon.c (monmap changed)... 2023-10-18T09:10:24.731 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:24.732 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:24.732 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:24.732 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: Reconfiguring daemon mon.c on smithi170 2023-10-18T09:10:24.732 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.739 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2023-10-18T09:10:24.739 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_fragment_size_max 10000 2023-10-18T09:10:24.790 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2023-10-18T09:10:24.790 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_merge_size 5 2023-10-18T09:10:24.845 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2023-10-18T09:10:24.846 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_split_bits 3 2023-10-18T09:10:24.899 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2023-10-18T09:10:24.899 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_bal_split_size 100 2023-10-18T09:10:24.956 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2023-10-18T09:10:24.957 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_debug_frag True 2023-10-18T09:10:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: Reconfiguring mon.c (monmap changed)... 2023-10-18T09:10:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:24.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:24.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:24.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: Reconfiguring daemon mon.c on smithi170 2023-10-18T09:10:24.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:25.009 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2023-10-18T09:10:25.010 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_debug_scatterstat True 2023-10-18T09:10:25.045 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:25.046 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:25.046 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: Reconfiguring mon.c (monmap changed)... 2023-10-18T09:10:25.046 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-18T09:10:25.046 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-18T09:10:25.047 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:25.047 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: Reconfiguring daemon mon.c on smithi170 2023-10-18T09:10:25.047 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:25.066 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2023-10-18T09:10:25.067 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_op_complaint_time 180 2023-10-18T09:10:25.136 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2023-10-18T09:10:25.136 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds mds_verify_scatter True 2023-10-18T09:10:25.193 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2023-10-18T09:10:25.193 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds osd_op_complaint_time 180 2023-10-18T09:10:25.267 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2023-10-18T09:10:25.267 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds rados_mon_op_timeout 900 2023-10-18T09:10:25.334 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2023-10-18T09:10:25.334 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mds rados_osd_op_timeout 900 2023-10-18T09:10:25.388 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2023-10-18T09:10:25.388 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set mon mon_op_complaint_time 120 2023-10-18T09:10:25.445 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2023-10-18T09:10:25.445 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config set osd osd_op_complaint_time 180 2023-10-18T09:10:25.509 DEBUG:tasks.cephadm:set 21 configs 2023-10-18T09:10:25.510 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi061') 2023-10-18T09:10:25.884 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:25.963 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:26.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi061=x;smithi110=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:26.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: Saving service mgr spec with placement smithi061=x;smithi110=y;smithi170=z;count:3 2023-10-18T09:10:26.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:26.007 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi061=x;smithi110=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: Saving service mgr spec with placement smithi061=x;smithi110=y;smithi170=z;count:3 2023-10-18T09:10:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:26.153 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:26.153 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi061=x;smithi110=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:10:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: Saving service mgr spec with placement smithi061=x;smithi110=y;smithi170=z;count:3 2023-10-18T09:10:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:26.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:26.432 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:26.781 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:26.789 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:27.493 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:27.495 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:27.619 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:27 smithi061 ceph-mon[98573]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:27.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:27 smithi110 ceph-mon[134818]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:27 smithi170 ceph-mon[135150]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:28.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:28.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:28.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:28.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:28.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:28.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:28 smithi170 ceph-mon[135150]: Deploying daemon mgr.z on smithi170 2023-10-18T09:10:28.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:28.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:28.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:28.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:28.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:28.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:28 smithi110 ceph-mon[134818]: Deploying daemon mgr.z on smithi170 2023-10-18T09:10:28.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:28.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:28.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:28.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:28.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:28.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:28.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:28 smithi061 ceph-mon[98573]: Deploying daemon mgr.z on smithi170 2023-10-18T09:10:29.143 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:28 smithi170 systemd[1]: Starting Ceph mgr.z for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:10:29.426 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-mon[135150]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:29.426 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 podman[136987]: 2023-10-18T09:10:29.661 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:29 smithi061 ceph-mon[98573]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:29 smithi110 ceph-mon[134818]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:29.726 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 bash[136987]: aa46dbb587f3c0adf3402db46d088d863d0f13b4fad6d3036d98a95c5428735f 2023-10-18T09:10:29.727 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-mgr[137027]: -- 172.21.15.170:0/546479427 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5579792d71e0 con 0x5579792ac800 2023-10-18T09:10:29.727 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 systemd[1]: Started Ceph mgr.z for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:10:29.727 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:29.554+0000 7f0501473200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T09:10:29.727 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:29.634+0000 7f0501473200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T09:10:29.727 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:29.723+0000 7f0501473200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T09:10:29.989 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:29.817+0000 7f0501473200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T09:10:30.350 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:29.986+0000 7f0501473200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T09:10:30.642 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:30.348+0000 7f0501473200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T09:10:30.642 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:30.480+0000 7f0501473200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T09:10:30.643 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:30.569+0000 7f0501473200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T09:10:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:30.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:30.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:30.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:30.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: Deploying daemon mgr.y on smithi110 2023-10-18T09:10:30.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:30.799 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.799 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.799 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.799 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:30.800 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:30.800 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:30.800 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:30.800 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: Deploying daemon mgr.y on smithi110 2023-10-18T09:10:30.801 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:30 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:30.880 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:30.890 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.890 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.890 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:30.890 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:30.891 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-18T09:10:30.891 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:30.891 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:30.891 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: Deploying daemon mgr.y on smithi110 2023-10-18T09:10:30.891 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:30 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:31.082 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:30.663+0000 7f0501473200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T09:10:31.212 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 systemd[1]: Starting Ceph mgr.y for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:10:31.392 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:31.079+0000 7f0501473200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T09:10:31.536 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 podman[135989]: 2023-10-18T09:10:31.556 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi061') 2023-10-18T09:10:31.788 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:31 smithi061 ceph-mon[98573]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:31.830 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:31 smithi170 ceph-mon[135150]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:31.831 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:31.746+0000 7f0501473200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T09:10:31.837 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-mon[134818]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:31.837 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 bash[135989]: fa63f2f8dfd55c64369c8cd4d0f1218738d5d694d5e01d6816a7130a3435a8a7 2023-10-18T09:10:31.838 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-mgr[136029]: -- 172.21.15.110:0/3951338111 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5632198f31e0 con 0x5632198c8c00 2023-10-18T09:10:31.838 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 systemd[1]: Started Ceph mgr.y for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:10:31.838 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:31.665+0000 7f157cadc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-18T09:10:31.838 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:31.745+0000 7f157cadc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-18T09:10:31.838 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:31.834+0000 7f157cadc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-18T09:10:32.122 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:31.934+0000 7f157cadc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-18T09:10:32.142 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:31.827+0000 7f0501473200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T09:10:32.462 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:32.119+0000 7f157cadc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-18T09:10:32.839 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.839 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.839 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.840 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.840 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:32.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:32.504+0000 7f157cadc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-18T09:10:32.840 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:32.639+0000 7f157cadc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-18T09:10:32.841 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:32.740+0000 7f157cadc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-18T09:10:32.841 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:32.836+0000 7f157cadc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-18T09:10:32.932 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:32.644+0000 7f0501473200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:10:32.932 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:32.741+0000 7f0501473200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T09:10:32.932 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.933 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:32.936 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.936 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.936 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.937 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:32.937 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:33.231 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:32.929+0000 7f0501473200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:10:33.624 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:33.228+0000 7f0501473200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T09:10:33.625 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:33.473+0000 7f0501473200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T09:10:33.625 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:33.621+0000 7f0501473200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T09:10:33.711 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:33.280+0000 7f157cadc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-18T09:10:33.889 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:33.890 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1453837366' entity='client.admin' 2023-10-18T09:10:33.890 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:33.890 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:33.890 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:33.890 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:33.891 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:33.891 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:33.891 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:33.891 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:33.891 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:33.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:33.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:33.948 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:34.131 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:34.132 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1453837366' entity='client.admin' 2023-10-18T09:10:34.132 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:34.132 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.133 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.133 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.133 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.134 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:34.134 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:34.135 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.135 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:34.135 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:34.136 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:34.142 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:33.886+0000 7f0501473200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T09:10:34.212 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:33.955+0000 7f157cadc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-18T09:10:34.212 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:34.036+0000 7f157cadc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-18T09:10:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1453837366' entity='client.admin' 2023-10-18T09:10:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:34.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-18T09:10:34.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-18T09:10:34.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:34.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:34 smithi170 ceph-mon[135150]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T09:10:34.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:34 smithi170 ceph-mon[135150]: Reconfiguring daemon mgr.x on smithi061 2023-10-18T09:10:34.893 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:34.587+0000 7f0501473200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T09:10:34.893 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:34.717+0000 7f0501473200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T09:10:35.057 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:34 smithi061 ceph-mon[98573]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T09:10:35.058 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:34 smithi061 ceph-mon[98573]: Reconfiguring daemon mgr.x on smithi061 2023-10-18T09:10:35.068 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:34.803+0000 7f157cadc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:10:35.068 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:34.893+0000 7f157cadc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-18T09:10:35.068 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:34 smithi110 ceph-mon[134818]: Reconfiguring mgr.x (unknown last config time)... 2023-10-18T09:10:35.069 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:34 smithi110 ceph-mon[134818]: Reconfiguring daemon mgr.x on smithi061 2023-10-18T09:10:35.243 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:34.911+0000 7f0501473200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T09:10:35.244 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:35.024+0000 7f0501473200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T09:10:35.344 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:35.065+0000 7f157cadc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-18T09:10:35.712 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:35.341+0000 7f157cadc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-18T09:10:35.712 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:35.565+0000 7f157cadc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-18T09:10:35.802 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:35 smithi061 ceph-mon[98573]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:35.803 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:35 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:35.981 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:35.712+0000 7f157cadc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-18T09:10:35.982 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-mon[134818]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:35.982 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:35.982 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:36.104 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:35.240+0000 7f0501473200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T09:10:36.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:35 smithi170 ceph-mon[135150]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:36.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:35 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:36.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:35 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:36.114 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:35 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:36.218 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:36.283 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:36.392 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z[137003]: 2023-10-18T09:10:36.101+0000 7f0501473200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T09:10:36.462 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:35.979+0000 7f157cadc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-18T09:10:36.962 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:36.675+0000 7f157cadc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-18T09:10:36.962 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:36.806+0000 7f157cadc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-18T09:10:36.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:36.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: Standby manager daemon z started 2023-10-18T09:10:36.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T09:10:36.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:36.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T09:10:36.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:36.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:36 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:37.001 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:37.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: Standby manager daemon z started 2023-10-18T09:10:37.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T09:10:37.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:37.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T09:10:37.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:37.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:36 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:37.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: Standby manager daemon z started 2023-10-18T09:10:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-18T09:10:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-18T09:10:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.170:0/2793333517' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:37.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:36 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:37.332 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:37.002+0000 7f157cadc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-18T09:10:37.332 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:37.114+0000 7f157cadc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-18T09:10:37.333 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:37.330+0000 7f157cadc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-18T09:10:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:37 smithi170 ceph-mon[135150]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:37 smithi170 ceph-mon[135150]: mgrmap e14: x(active, since 76s), standbys: z 2023-10-18T09:10:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T09:10:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:37 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1420580016' entity='client.admin' 2023-10-18T09:10:38.185 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-mon[134818]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:38.186 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-mon[134818]: mgrmap e14: x(active, since 76s), standbys: z 2023-10-18T09:10:38.186 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T09:10:38.186 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:37 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1420580016' entity='client.admin' 2023-10-18T09:10:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:37 smithi061 ceph-mon[98573]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:37 smithi061 ceph-mon[98573]: mgrmap e14: x(active, since 76s), standbys: z 2023-10-18T09:10:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-18T09:10:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:37 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1420580016' entity='client.admin' 2023-10-18T09:10:38.462 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y[136005]: 2023-10-18T09:10:38.183+0000 7f157cadc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-18T09:10:39.091 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:39.092 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: Standby manager daemon y started 2023-10-18T09:10:39.092 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T09:10:39.092 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:39.093 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T09:10:39.093 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:39.093 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:38 smithi061 ceph-mon[98573]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: Standby manager daemon y started 2023-10-18T09:10:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T09:10:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T09:10:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:38 smithi110 ceph-mon[134818]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: Standby manager daemon y started 2023-10-18T09:10:39.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-18T09:10:39.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-18T09:10:39.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-18T09:10:39.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: from='mgr.? 172.21.15.110:0/374589247' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-18T09:10:39.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:38 smithi170 ceph-mon[135150]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:39.489 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:39.496 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:39.547 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:39.713 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:39.881 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:39.921 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi061') 2023-10-18T09:10:39.982 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.046+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe32c0a1470 msgr2=0x7fe32c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.046+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe32c0a1470 0x7fe32c0a3930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe344074550 tx=0x7fe338009260 comp rx=0 tx=0).stop 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe344073250 msgr2=0x7fe34419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe344073250 0x7fe34419ccc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe334000c00 tx=0x7fe33400f690 comp rx=0 tx=0).stop 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 shutdown_connections 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe32c0a1470 0x7fe32c0a3930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe34410ce20 0x7fe3441a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe344073250 0x7fe34419ccc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 --2- 172.21.15.61:0/1907101261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe344074820 0x7fe34419d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 >> 172.21.15.61:0/1907101261 conn(0x7fe3440fbe00 msgr2=0x7fe3441001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 shutdown_connections 2023-10-18T09:10:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.047+0000 7fe349d53700 1 -- 172.21.15.61:0/1907101261 wait complete. 2023-10-18T09:10:40.072 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:39 smithi061 ceph-mon[98573]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-10-18T09:10:40.073 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T09:10:40.073 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:39 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:39 smithi110 ceph-mon[134818]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-10-18T09:10:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:39 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T09:10:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:39 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:40.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:39 smithi170 ceph-mon[135150]: mgrmap e15: x(active, since 78s), standbys: z, y 2023-10-18T09:10:40.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:39 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-18T09:10:40.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:39 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:40.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.604+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3824024849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 msgr2=0x7fc0ac106040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.604+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3824024849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac106040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc0980098f0 tx=0x7fc098009c00 comp rx=0 tx=0).stop 2023-10-18T09:10:40.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3824024849 shutdown_connections 2023-10-18T09:10:40.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3824024849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 0x7fc0ac108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3824024849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3824024849 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac106040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3824024849 >> 172.21.15.61:0/3824024849 conn(0x7fc0ac0facf0 msgr2=0x7fc0ac0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:40.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3824024849 shutdown_connections 2023-10-18T09:10:40.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.605+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3824024849 wait complete. 2023-10-18T09:10:40.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 Processor -- start 2023-10-18T09:10:40.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 -- start start 2023-10-18T09:10:40.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:40.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:40.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 0x7fc0ac19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:40.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac1a0570 con 0x7fc0ac103c50 2023-10-18T09:10:40.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac1a06b0 con 0x7fc0ac106580 2023-10-18T09:10:40.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.606+0000 7fc0b2fd3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac1a0840 con 0x7fc0ac101320 2023-10-18T09:10:40.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0b0d6f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:40.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:40.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0b0d6f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46572/0 (socket says 172.21.15.61:46572) 2023-10-18T09:10:40.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0b0d6f700 1 -- 172.21.15.61:0/3959191742 learned_addr learned my addr 172.21.15.61:0/3959191742 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0b1570700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 0x7fc0ac19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 -- 172.21.15.61:0/3959191742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 msgr2=0x7fc0ac198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 -- 172.21.15.61:0/3959191742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 msgr2=0x7fc0ac19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 0x7fc0ac19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 -- 172.21.15.61:0/3959191742 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098009520 con 0x7fc0ac103c50 2023-10-18T09:10:40.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0b0d6f700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:40.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0abfff700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac198f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:40.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.607+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09801c070 con 0x7fc0ac103c50 2023-10-18T09:10:40.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.608+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc09800bb90 con 0x7fc0ac103c50 2023-10-18T09:10:40.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.608+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac1a0ac0 con 0x7fc0ac103c50 2023-10-18T09:10:40.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.608+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098003aa0 con 0x7fc0ac103c50 2023-10-18T09:10:40.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.608+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac1a0f30 con 0x7fc0ac103c50 2023-10-18T09:10:40.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.609+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac04e4c0 con 0x7fc0ac103c50 2023-10-18T09:10:40.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.611+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fc098003c60 con 0x7fc0ac103c50 2023-10-18T09:10:40.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.613+0000 7fc0a9ffb700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 0x7fc09c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:40.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.613+0000 7fc0b0d6f700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 0x7fc09c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:40.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.613+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc0980cbb90 con 0x7fc0ac103c50 2023-10-18T09:10:40.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.614+0000 7fc0b0d6f700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 0x7fc09c0a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0011550 tx=0x7fc0a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:40.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.617+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc09805b100 con 0x7fc0ac103c50 2023-10-18T09:10:40.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.840+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7fc0ac10c0a0 con 0x7fc0ac103c50 2023-10-18T09:10:40.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.844+0000 7fc0a9ffb700 1 -- 172.21.15.61:0/3959191742 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v20)=0 v20) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc098096eb0 con 0x7fc0ac103c50 2023-10-18T09:10:40.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.846+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 msgr2=0x7fc09c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.846+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 0x7fc09c0a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0011550 tx=0x7fc0a0009260 comp rx=0 tx=0).stop 2023-10-18T09:10:40.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.846+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 msgr2=0x7fc0ac198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:40.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.846+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac198f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800b790 comp rx=0 tx=0).stop 2023-10-18T09:10:40.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.846+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 shutdown_connections 2023-10-18T09:10:40.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc09c0a1470 0x7fc09c0a3930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0ac106580 0x7fc0ac19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0ac101320 0x7fc0ac198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 --2- 172.21.15.61:0/3959191742 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0ac103c50 0x7fc0ac198f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:40.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 >> 172.21.15.61:0/3959191742 conn(0x7fc0ac0facf0 msgr2=0x7fc0ac0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:40.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 shutdown_connections 2023-10-18T09:10:40.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:40.847+0000 7fc0b2fd3700 1 -- 172.21.15.61:0/3959191742 wait complete. 2023-10-18T09:10:41.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:41 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:41.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:41 smithi061 ceph-mon[98573]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:41.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3959191742' entity='client.admin' 2023-10-18T09:10:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:41 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:41 smithi170 ceph-mon[135150]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:41.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3959191742' entity='client.admin' 2023-10-18T09:10:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:41 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:41 smithi110 ceph-mon[134818]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3959191742' entity='client.admin' 2023-10-18T09:10:41.933 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi061') 2023-10-18T09:10:41.933 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi061') 2023-10-18T09:10:41.934 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi061') 2023-10-18T09:10:41.934 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi061') 2023-10-18T09:10:42.654 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:42.686 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi061') 2023-10-18T09:10:42.686 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi061') 2023-10-18T09:10:43.828 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.829 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.829 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:43.829 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:43.829 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:43.829 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.830 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:43 smithi061 ceph-mon[98573]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:43.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:43.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:43.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:43.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:43 smithi170 ceph-mon[135150]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:43.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:10:43.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:10:43.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:10:43.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:10:43.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:43 smithi110 ceph-mon[134818]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 -- 172.21.15.61:0/1864541230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1023e0 msgr2=0x7fa55c108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 --2- 172.21.15.61:0/1864541230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1023e0 0x7fa55c108c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa55400b0d0 tx=0x7fa55400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 -- 172.21.15.61:0/1864541230 shutdown_connections 2023-10-18T09:10:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 --2- 172.21.15.61:0/1864541230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1023e0 0x7fa55c108c90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 --2- 172.21.15.61:0/1864541230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c100840 0x7fa55c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 --2- 172.21.15.61:0/1864541230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 0x7fa55c101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 -- 172.21.15.61:0/1864541230 >> 172.21.15.61:0/1864541230 conn(0x7fa55c0fbe00 msgr2=0x7fa55c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 -- 172.21.15.61:0/1864541230 shutdown_connections 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.114+0000 7fa5622ad700 1 -- 172.21.15.61:0/1864541230 wait complete. 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.115+0000 7fa5622ad700 1 Processor -- start 2023-10-18T09:10:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.115+0000 7fa5622ad700 1 -- start start 2023-10-18T09:10:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 0x7fa55c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 0x7fa55c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19e3a0 con 0x7fa55c100840 2023-10-18T09:10:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19e4e0 con 0x7fa55c101a40 2023-10-18T09:10:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa5622ad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c19e670 con 0x7fa55c1023e0 2023-10-18T09:10:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:46176/0 (socket says 172.21.15.61:46176) 2023-10-18T09:10:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 -- 172.21.15.61:0/561779673 learned_addr learned my addr 172.21.15.61:0/561779673 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 0x7fa55c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa56084a700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 0x7fa55c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 -- 172.21.15.61:0/561779673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 msgr2=0x7fa55c19ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 0x7fa55c19ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 -- 172.21.15.61:0/561779673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 msgr2=0x7fa55c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 0x7fa55c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa55bfff700 1 -- 172.21.15.61:0/561779673 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554009d00 con 0x7fa55c100840 2023-10-18T09:10:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.116+0000 7fa56084a700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 0x7fa55c19ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:10:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa55bfff700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9a0 tx=0x7fa54c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c009790 con 0x7fa55c100840 2023-10-18T09:10:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c19e950 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c0044e0 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c003d20 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.117+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c19ee50 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.118+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c107c00 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.120+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fa54c003e80 con 0x7fa55c100840 2023-10-18T09:10:44.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.122+0000 7fa5597fa700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 0x7fa5440a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.122+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa54c014070 con 0x7fa55c100840 2023-10-18T09:10:44.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.122+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 0x7fa5440a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.122+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 0x7fa5440a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa550009c60 tx=0x7fa550009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:44.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.124+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa54c096690 con 0x7fa55c100840 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.340+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7fa55c10c0a0 con 0x7fa55c100840 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.344+0000 7fa5597fa700 1 -- 172.21.15.61:0/561779673 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v21)=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa54c099cb0 con 0x7fa55c100840 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 msgr2=0x7fa5440a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 0x7fa5440a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa550009c60 tx=0x7fa550009260 comp rx=0 tx=0).stop 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 msgr2=0x7fa55c1968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00e9a0 tx=0x7fa54c00ecb0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 shutdown_connections 2023-10-18T09:10:44.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a13f0 0x7fa5440a38b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1023e0 0x7fa55c19ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100840 0x7fa55c1968b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 --2- 172.21.15.61:0/561779673 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101a40 0x7fa55c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 >> 172.21.15.61:0/561779673 conn(0x7fa55c0fbe00 msgr2=0x7fa55c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:44.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 shutdown_connections 2023-10-18T09:10:44.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.347+0000 7fa5622ad700 1 -- 172.21.15.61:0/561779673 wait complete. 2023-10-18T09:10:44.772 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi061') 2023-10-18T09:10:44.773 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi061') 2023-10-18T09:10:44.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 -- 172.21.15.61:0/1211580792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e84101a40 msgr2=0x7f3e84101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 --2- 172.21.15.61:0/1211580792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e84101a40 0x7f3e84101ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e780099c0 tx=0x7f3e78009cd0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 -- 172.21.15.61:0/1211580792 shutdown_connections 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 --2- 172.21.15.61:0/1211580792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e841023e0 0x7f3e84108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 --2- 172.21.15.61:0/1211580792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 0x7f3e84100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 --2- 172.21.15.61:0/1211580792 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e84101a40 0x7f3e84101ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 -- 172.21.15.61:0/1211580792 >> 172.21.15.61:0/1211580792 conn(0x7f3e840fbe00 msgr2=0x7f3e840fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 -- 172.21.15.61:0/1211580792 shutdown_connections 2023-10-18T09:10:44.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.958+0000 7f3e88c36700 1 -- 172.21.15.61:0/1211580792 wait complete. 2023-10-18T09:10:44.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.959+0000 7f3e88c36700 1 Processor -- start 2023-10-18T09:10:44.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 -- start start 2023-10-18T09:10:44.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 0x7f3e841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e84101a40 0x7f3e84198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a04f0 con 0x7f3e841023e0 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a0630 con 0x7f3e84100840 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e88c36700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841a07c0 con 0x7f3e84101a40 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e82d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e82d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:46200/0 (socket says 172.21.15.61:46200) 2023-10-18T09:10:44.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e82d9d700 1 -- 172.21.15.61:0/2991047428 learned_addr learned my addr 172.21.15.61:0/2991047428 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:44.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.960+0000 7f3e81d9b700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e84101a40 0x7f3e84198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 -- 172.21.15.61:0/2991047428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e84101a40 msgr2=0x7f3e84198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e84101a40 0x7f3e84198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 -- 172.21.15.61:0/2991047428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 msgr2=0x7f3e841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e8259c700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 0x7f3e841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 0x7f3e841989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 -- 172.21.15.61:0/2991047428 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780095f0 con 0x7f3e841023e0 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e82d9d700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c007f10 tx=0x7f3e7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c017070 con 0x7f3e841023e0 2023-10-18T09:10:44.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7c00f040 con 0x7f3e841023e0 2023-10-18T09:10:44.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.961+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e841a0aa0 con 0x7f3e841023e0 2023-10-18T09:10:44.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.962+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c005740 con 0x7f3e841023e0 2023-10-18T09:10:44.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.962+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e841a0fa0 con 0x7f3e841023e0 2023-10-18T09:10:44.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.963+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8404e4c0 con 0x7f3e841023e0 2023-10-18T09:10:44.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.965+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f3e7c0058a0 con 0x7f3e841023e0 2023-10-18T09:10:44.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.967+0000 7f3e737fe700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 0x7f3e6c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:44.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.968+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e7c0ca830 con 0x7f3e841023e0 2023-10-18T09:10:44.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.968+0000 7f3e8259c700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 0x7f3e6c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:44.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.969+0000 7f3e8259c700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 0x7f3e6c0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e841018a0 tx=0x7f3e74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:44.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:44.972+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e7c095ac0 con 0x7f3e841023e0 2023-10-18T09:10:44.977 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:45.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.272+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) v1 -- 0x7f3e8410c090 con 0x7f3e841023e0 2023-10-18T09:10:45.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.277+0000 7f3e737fe700 1 -- 172.21.15.61:0/2991047428 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v22)=0 v22) v1 ==== 116+0+0 (secure 0 0 0) 0x7f3e7c0990e0 con 0x7f3e841023e0 2023-10-18T09:10:45.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 msgr2=0x7f3e6c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:45.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 0x7f3e6c0a3930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e841018a0 tx=0x7f3e74009260 comp rx=0 tx=0).stop 2023-10-18T09:10:45.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 msgr2=0x7f3e8419ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:45.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c007f10 tx=0x7f3e7c00d3b0 comp rx=0 tx=0).stop 2023-10-18T09:10:45.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 shutdown_connections 2023-10-18T09:10:45.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e6c0a1470 0x7f3e6c0a3930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:45.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e841023e0 0x7f3e8419ff20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:45.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e84100840 0x7f3e841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:45.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 --2- 172.21.15.61:0/2991047428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e84101a40 0x7f3e84198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:45.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 >> 172.21.15.61:0/2991047428 conn(0x7f3e840fbe00 msgr2=0x7f3e841072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:45.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 shutdown_connections 2023-10-18T09:10:45.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:45.295+0000 7f3e88c36700 1 -- 172.21.15.61:0/2991047428 wait complete. 2023-10-18T09:10:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/561779673' entity='client.admin' 2023-10-18T09:10:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:45 smithi061 ceph-mon[98573]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:45.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2991047428' entity='client.admin' 2023-10-18T09:10:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/561779673' entity='client.admin' 2023-10-18T09:10:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:45 smithi170 ceph-mon[135150]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2991047428' entity='client.admin' 2023-10-18T09:10:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/561779673' entity='client.admin' 2023-10-18T09:10:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:45 smithi110 ceph-mon[134818]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2991047428' entity='client.admin' 2023-10-18T09:10:47.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 -- 172.21.15.61:0/1306139847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 msgr2=0x7fa01c108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 --2- 172.21.15.61:0/1306139847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c108990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa01400b0a0 tx=0x7fa01400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 -- 172.21.15.61:0/1306139847 shutdown_connections 2023-10-18T09:10:47.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 --2- 172.21.15.61:0/1306139847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c108990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 --2- 172.21.15.61:0/1306139847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 --2- 172.21.15.61:0/1306139847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 -- 172.21.15.61:0/1306139847 >> 172.21.15.61:0/1306139847 conn(0x7fa01c0facf0 msgr2=0x7fa01c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:47.632 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:47 smithi061 ceph-mon[98573]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:47.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 -- 172.21.15.61:0/1306139847 shutdown_connections 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.627+0000 7fa022c6a700 1 -- 172.21.15.61:0/1306139847 wait complete. 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.628+0000 7fa022c6a700 1 Processor -- start 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.628+0000 7fa022c6a700 1 -- start start 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c10f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c116380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c116950 con 0x7fa01c101320 2023-10-18T09:10:47.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c116ac0 con 0x7fa01c106580 2023-10-18T09:10:47.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa022c6a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c116c80 con 0x7fa01c103c50 2023-10-18T09:10:47.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:46218/0 (socket says 172.21.15.61:46218) 2023-10-18T09:10:47.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 -- 172.21.15.61:0/2986235942 learned_addr learned my addr 172.21.15.61:0/2986235942 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:47.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa01bfff700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c10f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa021207700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c116380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 -- 172.21.15.61:0/2986235942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 msgr2=0x7fa01c10f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c10f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.629+0000 7fa020a06700 1 -- 172.21.15.61:0/2986235942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 msgr2=0x7fa01c116380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa020a06700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c116380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa020a06700 1 -- 172.21.15.61:0/2986235942 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014009d00 con 0x7fa01c101320 2023-10-18T09:10:47.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa01bfff700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa021207700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c116380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa020a06700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9a0 tx=0x7fa01000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010009790 con 0x7fa01c101320 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa010010430 con 0x7fa01c101320 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010018680 con 0x7fa01c101320 2023-10-18T09:10:47.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c116f60 con 0x7fa01c101320 2023-10-18T09:10:47.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.630+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c1adc70 con 0x7fa01c101320 2023-10-18T09:10:47.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.632+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c0fc870 con 0x7fa01c101320 2023-10-18T09:10:47.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.634+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fa0100105a0 con 0x7fa01c101320 2023-10-18T09:10:47.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.635+0000 7fa019ffb700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 0x7fa00c0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.636+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa010014070 con 0x7fa01c101320 2023-10-18T09:10:47.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.636+0000 7fa01bfff700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 0x7fa00c0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.636+0000 7fa01bfff700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 0x7fa00c0a3930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa008011490 tx=0x7fa008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:47.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.638+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa010096720 con 0x7fa01c101320 2023-10-18T09:10:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:47 smithi110 ceph-mon[134818]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:47.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 -- 172.21.15.61:0/3800251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 msgr2=0x7f1f540736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/3800251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f540736d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009990 tx=0x7f1f40009ca0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 -- 172.21.15.61:0/3800251538 shutdown_connections 2023-10-18T09:10:47.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/3800251538 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54108710 0x7f1f5410aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/3800251538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54074820 0x7f1f54072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/3800251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f540736d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 -- 172.21.15.61:0/3800251538 >> 172.21.15.61:0/3800251538 conn(0x7f1f540fbe00 msgr2=0x7f1f540fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:47.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 -- 172.21.15.61:0/3800251538 shutdown_connections 2023-10-18T09:10:47.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.838+0000 7f1f5ac23700 1 -- 172.21.15.61:0/3800251538 wait complete. 2023-10-18T09:10:47.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.839+0000 7f1f5ac23700 1 Processor -- start 2023-10-18T09:10:47.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 -- start start 2023-10-18T09:10:47.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54074820 0x7f1f54198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54108710 0x7f1f5419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f541a0500 con 0x7f1f54074820 2023-10-18T09:10:47.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f541a0640 con 0x7f1f54108710 2023-10-18T09:10:47.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f5ac23700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f541a07d0 con 0x7f1f54073250 2023-10-18T09:10:47.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f589bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f53fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54074820 0x7f1f54198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f589bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46670/0 (socket says 172.21.15.61:46670) 2023-10-18T09:10:47.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f589bf700 1 -- 172.21.15.61:0/1654076248 learned_addr learned my addr 172.21.15.61:0/1654076248 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:47.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.840+0000 7f1f591c0700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54108710 0x7f1f5419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 -- 172.21.15.61:0/1654076248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54108710 msgr2=0x7f1f5419ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54108710 0x7f1f5419ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 -- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54074820 msgr2=0x7f1f54198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54074820 0x7f1f54198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 -- 172.21.15.61:0/1654076248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400095f0 con 0x7f1f54073250 2023-10-18T09:10:47.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f589bf700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e9a0 tx=0x7f1f4800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:47.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.841+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f48009790 con 0x7f1f54073250 2023-10-18T09:10:47.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.842+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4800c810 con 0x7f1f54073250 2023-10-18T09:10:47.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.842+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f480054c0 con 0x7f1f54073250 2023-10-18T09:10:47.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.842+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f541a0ab0 con 0x7f1f54073250 2023-10-18T09:10:47.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.842+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f541a0f60 con 0x7f1f54073250 2023-10-18T09:10:47.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.843+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f541074d0 con 0x7f1f54073250 2023-10-18T09:10:47.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.846+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f1f4800cc80 con 0x7f1f54073250 2023-10-18T09:10:47.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.848+0000 7f1f51ffb700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 0x7f1f440a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:47.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.848+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f48014070 con 0x7f1f54073250 2023-10-18T09:10:47.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.848+0000 7f1f53fff700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 0x7f1f440a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:47.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.849+0000 7f1f53fff700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 0x7f1f440a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f40000c00 tx=0x7f1f40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:47.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.852+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f480964a0 con 0x7f1f54073250 2023-10-18T09:10:47.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.861+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7fa01c04e4c0 con 0x7fa01c101320 2023-10-18T09:10:47.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.865+0000 7fa019ffb700 1 -- 172.21.15.61:0/2986235942 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v23)=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fa010026070 con 0x7fa01c101320 2023-10-18T09:10:47.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 msgr2=0x7fa00c0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 0x7fa00c0a3930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa008011490 tx=0x7fa008009260 comp rx=0 tx=0).stop 2023-10-18T09:10:47.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 msgr2=0x7fa01c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:47.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e9a0 tx=0x7fa01000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 shutdown_connections 2023-10-18T09:10:47.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa00c0a1470 0x7fa00c0a3930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa01c106580 0x7fa01c116380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa01c101320 0x7fa01c10ee10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 --2- 172.21.15.61:0/2986235942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa01c103c50 0x7fa01c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:47.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 >> 172.21.15.61:0/2986235942 conn(0x7fa01c0facf0 msgr2=0x7fa01c101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:47.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 shutdown_connections 2023-10-18T09:10:47.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:47.868+0000 7fa022c6a700 1 -- 172.21.15.61:0/2986235942 wait complete. 2023-10-18T09:10:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:47 smithi170 ceph-mon[135150]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:48.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.087+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f1f5404e4c0 con 0x7f1f54073250 2023-10-18T09:10:48.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.092+0000 7f1f51ffb700 1 -- 172.21.15.61:0/1654076248 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1f48099ac0 con 0x7f1f54073250 2023-10-18T09:10:48.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 msgr2=0x7f1f440a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:48.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 0x7f1f440a3930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f40000c00 tx=0x7f1f40005fb0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 msgr2=0x7f1f541989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:48.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e9a0 tx=0x7f1f4800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 shutdown_connections 2023-10-18T09:10:48.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f440a1470 0x7f1f440a3930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f54108710 0x7f1f5419ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f54073250 0x7f1f541989c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 --2- 172.21.15.61:0/1654076248 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f54074820 0x7f1f54198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:48.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 >> 172.21.15.61:0/1654076248 conn(0x7f1f540fbe00 msgr2=0x7f1f54104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:48.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 shutdown_connections 2023-10-18T09:10:48.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:48.095+0000 7f1f5ac23700 1 -- 172.21.15.61:0/1654076248 wait complete. 2023-10-18T09:10:49.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:48 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2986235942' entity='client.admin' 2023-10-18T09:10:49.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:48 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:49.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:48 smithi170 ceph-mon[135150]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:49.206 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:48 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2986235942' entity='client.admin' 2023-10-18T09:10:49.206 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:48 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:49.206 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:48 smithi061 ceph-mon[98573]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:49.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:48 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2986235942' entity='client.admin' 2023-10-18T09:10:49.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:48 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:49.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:48 smithi110 ceph-mon[134818]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:49.748 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi061') 2023-10-18T09:10:49.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 -- 172.21.15.61:0/1929010511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb458100430 msgr2=0x7fb4581008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:49.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 --2- 172.21.15.61:0/1929010511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb458100430 0x7fb4581008b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb45000b0a0 tx=0x7fb45000b3b0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 -- 172.21.15.61:0/1929010511 shutdown_connections 2023-10-18T09:10:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 --2- 172.21.15.61:0/1929010511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb458100430 0x7fb4581008b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 --2- 172.21.15.61:0/1929010511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4580ff110 0x7fb4580ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 --2- 172.21.15.61:0/1929010511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 0x7fb4580ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 -- 172.21.15.61:0/1929010511 >> 172.21.15.61:0/1929010511 conn(0x7fb4580facf0 msgr2=0x7fb4580fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 -- 172.21.15.61:0/1929010511 shutdown_connections 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.719+0000 7fb46026e700 1 -- 172.21.15.61:0/1929010511 wait complete. 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.720+0000 7fb46026e700 1 Processor -- start 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.720+0000 7fb46026e700 1 -- start start 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 0x7fb458199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:49.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 0x7fb45819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:49.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a05b0 con 0x7fb4580ff110 2023-10-18T09:10:49.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a06f0 con 0x7fb458100430 2023-10-18T09:10:49.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb46026e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a0880 con 0x7fb4580ffa70 2023-10-18T09:10:49.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e00a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:49.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e00a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55016/0 (socket says 172.21.15.61:55016) 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e00a700 1 -- 172.21.15.61:0/2369289032 learned_addr learned my addr 172.21.15.61:0/2369289032 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 0x7fb45819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45d809700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 0x7fb458199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 -- 172.21.15.61:0/2369289032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 msgr2=0x7fb458199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 0x7fb458199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 -- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 msgr2=0x7fb458198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e80b700 1 -- 172.21.15.61:0/2369289032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009d00 con 0x7fb458100430 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.721+0000 7fb45e00a700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb45e80b700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 0x7fb45819ffe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb450007b90 tx=0x7fb45000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000e040 con 0x7fb458100430 2023-10-18T09:10:49.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4500094b0 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45001b6c0 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581a0b00 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.722+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581a0ec0 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.723+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb458109ef0 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.725+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fb45001d470 con 0x7fb458100430 2023-10-18T09:10:49.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.727+0000 7fb44f7fe700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 0x7fb4440a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:49.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.727+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb4500cc340 con 0x7fb458100430 2023-10-18T09:10:49.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.728+0000 7fb45e00a700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 0x7fb4440a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:49.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.729+0000 7fb45e00a700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 0x7fb4440a38b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb454011550 tx=0x7fb454009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:49.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.731+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb450097660 con 0x7fb458100430 2023-10-18T09:10:49.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.966+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7fb45804e4c0 con 0x7fb458100430 2023-10-18T09:10:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.972+0000 7fb44f7fe700 1 -- 172.21.15.61:0/2369289032 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb45009ac80 con 0x7fb458100430 2023-10-18T09:10:49.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 msgr2=0x7fb4440a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:49.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 0x7fb4440a38b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb454011550 tx=0x7fb454009260 comp rx=0 tx=0).stop 2023-10-18T09:10:49.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 msgr2=0x7fb45819ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:49.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 0x7fb45819ffe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb450007b90 tx=0x7fb45000bce0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 shutdown_connections 2023-10-18T09:10:49.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4440a13f0 0x7fb4440a38b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb458100430 0x7fb45819ffe0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4580ff110 0x7fb458198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 --2- 172.21.15.61:0/2369289032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4580ffa70 0x7fb458199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:49.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 >> 172.21.15.61:0/2369289032 conn(0x7fb4580facf0 msgr2=0x7fb4580fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:49.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 shutdown_connections 2023-10-18T09:10:49.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:49.974+0000 7fb46026e700 1 -- 172.21.15.61:0/2369289032 wait complete. 2023-10-18T09:10:50.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.046+0000 7fde37c57700 1 -- 172.21.15.61:0/2046946125 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde30101a40 msgr2=0x7fde30101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.046+0000 7fde37c57700 1 --2- 172.21.15.61:0/2046946125 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde30101a40 0x7fde30101ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde200099c0 tx=0x7fde20009cd0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 -- 172.21.15.61:0/2046946125 shutdown_connections 2023-10-18T09:10:50.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 --2- 172.21.15.61:0/2046946125 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde301023e0 0x7fde30108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 --2- 172.21.15.61:0/2046946125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 0x7fde30100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 --2- 172.21.15.61:0/2046946125 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde30101a40 0x7fde30101ea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 -- 172.21.15.61:0/2046946125 >> 172.21.15.61:0/2046946125 conn(0x7fde300fbe00 msgr2=0x7fde300fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:50.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 -- 172.21.15.61:0/2046946125 shutdown_connections 2023-10-18T09:10:50.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.047+0000 7fde37c57700 1 -- 172.21.15.61:0/2046946125 wait complete. 2023-10-18T09:10:50.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 Processor -- start 2023-10-18T09:10:50.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 -- start start 2023-10-18T09:10:50.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 0x7fde30198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 0x7fde30198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301a04b0 con 0x7fde301023e0 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301a05f0 con 0x7fde30101a40 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.048+0000 7fde37c57700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde301a0780 con 0x7fde30100840 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde351f2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 0x7fde30198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde359f3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 0x7fde30198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55024/0 (socket says 172.21.15.61:55024) 2023-10-18T09:10:50.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 -- 172.21.15.61:0/1638246182 learned_addr learned my addr 172.21.15.61:0/1638246182 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:50.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 -- 172.21.15.61:0/1638246182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 msgr2=0x7fde30198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 0x7fde30198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 -- 172.21.15.61:0/1638246182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 msgr2=0x7fde30198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 0x7fde30198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 -- 172.21.15.61:0/1638246182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200095f0 con 0x7fde301023e0 2023-10-18T09:10:50.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde351f2700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 0x7fde30198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:50.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.049+0000 7fde361f4700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde2800ef60 tx=0x7fde2800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:50.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.050+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28018070 con 0x7fde301023e0 2023-10-18T09:10:50.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.050+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde2800f040 con 0x7fde301023e0 2023-10-18T09:10:50.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.050+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde301a0a00 con 0x7fde301023e0 2023-10-18T09:10:50.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.050+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800ccc0 con 0x7fde301023e0 2023-10-18T09:10:50.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.050+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde301a0eb0 con 0x7fde301023e0 2023-10-18T09:10:50.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.051+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3004e4c0 con 0x7fde301023e0 2023-10-18T09:10:50.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.053+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fde28003a00 con 0x7fde301023e0 2023-10-18T09:10:50.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.055+0000 7fde26ffd700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 0x7fde1c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.055+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fde280ca680 con 0x7fde301023e0 2023-10-18T09:10:50.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.055+0000 7fde359f3700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 0x7fde1c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.056+0000 7fde359f3700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 0x7fde1c0a38b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde301018a0 tx=0x7fde2c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:50.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.059+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde28095a40 con 0x7fde301023e0 2023-10-18T09:10:50.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.277+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) v1 -- 0x7fde3010c090 con 0x7fde301023e0 2023-10-18T09:10:50.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.281+0000 7fde26ffd700 1 -- 172.21.15.61:0/1638246182 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v26)=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde28099060 con 0x7fde301023e0 2023-10-18T09:10:50.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 msgr2=0x7fde1c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 0x7fde1c0a38b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde301018a0 tx=0x7fde2c007400 comp rx=0 tx=0).stop 2023-10-18T09:10:50.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 msgr2=0x7fde3019fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde2800ef60 tx=0x7fde2800ef90 comp rx=0 tx=0).stop 2023-10-18T09:10:50.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 shutdown_connections 2023-10-18T09:10:50.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde1c0a13f0 0x7fde1c0a38b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde301023e0 0x7fde3019fee0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde30100840 0x7fde30198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 --2- 172.21.15.61:0/1638246182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde30101a40 0x7fde30198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 >> 172.21.15.61:0/1638246182 conn(0x7fde300fbe00 msgr2=0x7fde30072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:50.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 shutdown_connections 2023-10-18T09:10:50.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.284+0000 7fde37c57700 1 -- 172.21.15.61:0/1638246182 wait complete. 2023-10-18T09:10:50.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 -- 172.21.15.61:0/3853630179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4101a20 msgr2=0x7f9bb4101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/3853630179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4101a20 0x7f9bb4101e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00098f0 tx=0x7f9ba0009c00 comp rx=0 tx=0).stop 2023-10-18T09:10:50.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 -- 172.21.15.61:0/3853630179 shutdown_connections 2023-10-18T09:10:50.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/3853630179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 0x7f9bb4108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/3853630179 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4100820 0x7f9bb4100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/3853630179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4101a20 0x7f9bb4101e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 -- 172.21.15.61:0/3853630179 >> 172.21.15.61:0/3853630179 conn(0x7f9bb40fbdc0 msgr2=0x7f9bb40fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:50.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 -- 172.21.15.61:0/3853630179 shutdown_connections 2023-10-18T09:10:50.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.537+0000 7f9bbaba2700 1 -- 172.21.15.61:0/3853630179 wait complete. 2023-10-18T09:10:50.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.538+0000 7f9bbaba2700 1 Processor -- start 2023-10-18T09:10:50.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.538+0000 7f9bbaba2700 1 -- start start 2023-10-18T09:10:50.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.538+0000 7f9bbaba2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4100820 0x7f9bb41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.538+0000 7f9bbaba2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bbaba2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 0x7f9bb419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bbaba2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb41a04f0 con 0x7f9bb4101a20 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bbaba2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb41a0630 con 0x7f9bb41023c0 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bbaba2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb41a07c0 con 0x7f9bb4100820 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb893e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4100820 0x7f9bb41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55032/0 (socket says 172.21.15.61:55032) 2023-10-18T09:10:50.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 -- 172.21.15.61:0/1577458418 learned_addr learned my addr 172.21.15.61:0/1577458418 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:50.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb913f700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 0x7f9bb419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 -- 172.21.15.61:0/1577458418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4100820 msgr2=0x7f9bb41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4100820 0x7f9bb41989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 -- 172.21.15.61:0/1577458418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 msgr2=0x7f9bb419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 0x7f9bb419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.539+0000 7f9bb3fff700 1 -- 172.21.15.61:0/1577458418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80095f0 con 0x7f9bb4101a20 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bb3fff700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000fba0 tx=0x7f9ba000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:50.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba001c070 con 0x7f9bb4101a20 2023-10-18T09:10:50.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba0009520 con 0x7f9bb4101a20 2023-10-18T09:10:50.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ba0005500 con 0x7f9bb4101a20 2023-10-18T09:10:50.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0003bf0 con 0x7f9bb4101a20 2023-10-18T09:10:50.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.540+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb41a0cc0 con 0x7f9bb4101a20 2023-10-18T09:10:50.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.541+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb404e4c0 con 0x7f9bb4101a20 2023-10-18T09:10:50.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.543+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f9ba0003d50 con 0x7f9bb4101a20 2023-10-18T09:10:50.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.544+0000 7f9bb1ffb700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 0x7f9ba40a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:50.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.544+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ba00cbd10 con 0x7f9bb4101a20 2023-10-18T09:10:50.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.545+0000 7f9bb893e700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 0x7f9ba40a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:50.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.545+0000 7f9bb893e700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 0x7f9ba40a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4101880 tx=0x7f9ba8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:50.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.548+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9ba005b100 con 0x7f9bb4101a20 2023-10-18T09:10:50.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.808+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f9bb410c290 con 0x7f9bb4101a20 2023-10-18T09:10:50.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.811+0000 7f9bb1ffb700 1 -- 172.21.15.61:0/1577458418 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v27)=0 v27) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9ba00d2080 con 0x7f9bb4101a20 2023-10-18T09:10:50.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 msgr2=0x7f9ba40a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 0x7f9ba40a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4101880 tx=0x7f9ba8009310 comp rx=0 tx=0).stop 2023-10-18T09:10:50.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 msgr2=0x7f9bb4198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:50.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000fba0 tx=0x7f9ba000b560 comp rx=0 tx=0).stop 2023-10-18T09:10:50.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 shutdown_connections 2023-10-18T09:10:50.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9ba40a13f0 0x7f9ba40a38b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9bb41023c0 0x7f9bb419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bb4100820 0x7f9bb41989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 --2- 172.21.15.61:0/1577458418 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9bb4101a20 0x7f9bb4198ef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:50.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 >> 172.21.15.61:0/1577458418 conn(0x7f9bb40fbdc0 msgr2=0x7f9bb4107260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:50.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.814+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 shutdown_connections 2023-10-18T09:10:50.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:50.815+0000 7f9bbaba2700 1 -- 172.21.15.61:0/1577458418 wait complete. 2023-10-18T09:10:51.101 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:50 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:51.102 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:50 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1638246182' entity='client.admin' 2023-10-18T09:10:51.102 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:50 smithi061 ceph-mon[98573]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:51.102 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:50 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1577458418' entity='client.admin' 2023-10-18T09:10:51.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 -- 172.21.15.61:0/952157120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9958103a20 msgr2=0x7f9958105e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 --2- 172.21.15.61:0/952157120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9958103a20 0x7f9958105e10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9948009990 tx=0x7f9948009ca0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 -- 172.21.15.61:0/952157120 shutdown_connections 2023-10-18T09:10:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 --2- 172.21.15.61:0/952157120 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 0x7f99581087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 --2- 172.21.15.61:0/952157120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99581010f0 0x7f99581034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 --2- 172.21.15.61:0/952157120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9958103a20 0x7f9958105e10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 -- 172.21.15.61:0/952157120 >> 172.21.15.61:0/952157120 conn(0x7f99580fab00 msgr2=0x7f99580fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:51.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 -- 172.21.15.61:0/952157120 shutdown_connections 2023-10-18T09:10:51.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.119+0000 7f995f4ba700 1 -- 172.21.15.61:0/952157120 wait complete. 2023-10-18T09:10:51.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.120+0000 7f995f4ba700 1 Processor -- start 2023-10-18T09:10:51.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.120+0000 7f995f4ba700 1 -- start start 2023-10-18T09:10:51.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 0x7f9958196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 0x7f995819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819e0c0 con 0x7f9958106350 2023-10-18T09:10:51.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819e200 con 0x7f99581010f0 2023-10-18T09:10:51.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995f4ba700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995819e390 con 0x7f9958103a20 2023-10-18T09:10:51.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:51332/0 (socket says 172.21.15.61:51332) 2023-10-18T09:10:51.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 -- 172.21.15.61:0/1990039329 learned_addr learned my addr 172.21.15.61:0/1990039329 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:51.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995ecb9700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 0x7f995819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995dcb7700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 0x7f9958196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 -- 172.21.15.61:0/1990039329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 msgr2=0x7f9958196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 0x7f9958196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 -- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 msgr2=0x7f995819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 0x7f995819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995e4b8700 1 -- 172.21.15.61:0/1990039329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480095f0 con 0x7f99581010f0 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.121+0000 7f995dcb7700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 0x7f9958196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:10:51.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f995e4b8700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9a0 tx=0x7f995400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:51.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954009790 con 0x7f99581010f0 2023-10-18T09:10:51.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995400cbd0 con 0x7f99581010f0 2023-10-18T09:10:51.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995819e670 con 0x7f99581010f0 2023-10-18T09:10:51.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954018680 con 0x7f99581010f0 2023-10-18T09:10:51.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.122+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995819ebc0 con 0x7f99581010f0 2023-10-18T09:10:51.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.123+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99580fc680 con 0x7f99581010f0 2023-10-18T09:10:51.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.125+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f995400cd40 con 0x7f99581010f0 2023-10-18T09:10:51.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 -- 172.21.15.61:0/1473070664 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc074820 msgr2=0x7f02cc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 --2- 172.21.15.61:0/1473070664 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc074820 0x7f02cc072c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009990 tx=0x7f02bc009ca0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 -- 172.21.15.61:0/1473070664 shutdown_connections 2023-10-18T09:10:51.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 --2- 172.21.15.61:0/1473070664 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc10ce20 0x7f02cc10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 --2- 172.21.15.61:0/1473070664 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc074820 0x7f02cc072c80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 --2- 172.21.15.61:0/1473070664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc073250 0x7f02cc0736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 -- 172.21.15.61:0/1473070664 >> 172.21.15.61:0/1473070664 conn(0x7f02cc0fbe00 msgr2=0x7f02cc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:51.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 -- 172.21.15.61:0/1473070664 shutdown_connections 2023-10-18T09:10:51.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.055+0000 7f02d197d700 1 -- 172.21.15.61:0/1473070664 wait complete. 2023-10-18T09:10:51.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.056+0000 7f02d197d700 1 Processor -- start 2023-10-18T09:10:51.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.056+0000 7f02d197d700 1 -- start start 2023-10-18T09:10:51.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc10ce20 0x7f02cc1a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc1a4890 con 0x7f02cc073250 2023-10-18T09:10:51.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc1a49d0 con 0x7f02cc074820 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02d197d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc1a4b90 con 0x7f02cc10ce20 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02ca7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55044/0 (socket says 172.21.15.61:55044) 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02ca7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:51308/0 (socket says 172.21.15.61:51308) 2023-10-18T09:10:51.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 -- 172.21.15.61:0/759547149 learned_addr learned my addr 172.21.15.61:0/759547149 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:51.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02cb7fe700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc10ce20 0x7f02cc1a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 -- 172.21.15.61:0/759547149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc10ce20 msgr2=0x7f02cc1a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc10ce20 0x7f02cc1a42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 -- 172.21.15.61:0/759547149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 msgr2=0x7f02cc19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02caffd700 1 -- 172.21.15.61:0/759547149 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0095f0 con 0x7f02cc073250 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.057+0000 7f02ca7fc700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02caffd700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00b680 tx=0x7f02bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc01d070 con 0x7f02cc073250 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02bc021410 con 0x7f02cc073250 2023-10-18T09:10:51.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc1a4e10 con 0x7f02cc073250 2023-10-18T09:10:51.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc017650 con 0x7f02cc073250 2023-10-18T09:10:51.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.058+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc1a51c0 con 0x7f02cc073250 2023-10-18T09:10:51.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.060+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc103080 con 0x7f02cc073250 2023-10-18T09:10:51.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.061+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f02bc021580 con 0x7f02cc073250 2023-10-18T09:10:51.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.062+0000 7f02d097b700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 0x7f02b40a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.063+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02bc0cc390 con 0x7f02cc073250 2023-10-18T09:10:51.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.063+0000 7f02ca7fc700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 0x7f02b40a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.063+0000 7f02ca7fc700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 0x7f02b40a3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02cc074550 tx=0x7f02c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:51.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.066+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f02bc0976b0 con 0x7f02cc073250 2023-10-18T09:10:51.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.127+0000 7f994f7fe700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 0x7f99440a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:51.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.127+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9954014070 con 0x7f99581010f0 2023-10-18T09:10:51.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.127+0000 7f995dcb7700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 0x7f99440a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:51.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.127+0000 7f995dcb7700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 0x7f99440a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f994800b2e0 tx=0x7f9948005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:51.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.131+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9954052100 con 0x7f99581010f0 2023-10-18T09:10:51.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.285+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f02cc02cf10 con 0x7f02cc073250 2023-10-18T09:10:51.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.289+0000 7f02d097b700 1 -- 172.21.15.61:0/759547149 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v28)=0 v28) v1 ==== 119+0+0 (secure 0 0 0) 0x7f02bc09acd0 con 0x7f02cc073250 2023-10-18T09:10:51.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 msgr2=0x7f02b40a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 0x7f02b40a3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02cc074550 tx=0x7f02c0009260 comp rx=0 tx=0).stop 2023-10-18T09:10:51.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 msgr2=0x7f02cc19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00b680 tx=0x7f02bc004730 comp rx=0 tx=0).stop 2023-10-18T09:10:51.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 shutdown_connections 2023-10-18T09:10:51.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02b40a1470 0x7f02b40a3930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02cc10ce20 0x7f02cc1a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f02cc073250 0x7f02cc19cd90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 --2- 172.21.15.61:0/759547149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02cc074820 0x7f02cc19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.291+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 >> 172.21.15.61:0/759547149 conn(0x7f02cc0fbe00 msgr2=0x7f02cc1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:51.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.292+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 shutdown_connections 2023-10-18T09:10:51.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.292+0000 7f02d197d700 1 -- 172.21.15.61:0/759547149 wait complete. 2023-10-18T09:10:51.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.366+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f995804e4c0 con 0x7f99581010f0 2023-10-18T09:10:51.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.371+0000 7f994f7fe700 1 -- 172.21.15.61:0/1990039329 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v29)=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9954096520 con 0x7f99581010f0 2023-10-18T09:10:51.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 msgr2=0x7f99440a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 0x7f99440a3930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f994800b2e0 tx=0x7f9948005e80 comp rx=0 tx=0).stop 2023-10-18T09:10:51.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 msgr2=0x7f9958196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:51.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f995400e9a0 tx=0x7f995400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 shutdown_connections 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f99440a1470 0x7f99440a3930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9958106350 0x7f995819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99581010f0 0x7f9958196580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 --2- 172.21.15.61:0/1990039329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9958103a20 0x7f9958196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:51.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 >> 172.21.15.61:0/1990039329 conn(0x7f99580fab00 msgr2=0x7f9958101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:51.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 shutdown_connections 2023-10-18T09:10:51.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:51.373+0000 7f995f4ba700 1 -- 172.21.15.61:0/1990039329 wait complete. 2023-10-18T09:10:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:50 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:50 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1638246182' entity='client.admin' 2023-10-18T09:10:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:50 smithi170 ceph-mon[135150]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:50 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1577458418' entity='client.admin' 2023-10-18T09:10:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:50 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:50 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1638246182' entity='client.admin' 2023-10-18T09:10:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:50 smithi110 ceph-mon[134818]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:50 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1577458418' entity='client.admin' 2023-10-18T09:10:52.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 -- 172.21.15.61:0/3117586222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c103c40 msgr2=0x7f5c4c1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:52.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3117586222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c103c40 0x7f5c4c1040a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c400099c0 tx=0x7f5c40009cd0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 -- 172.21.15.61:0/3117586222 shutdown_connections 2023-10-18T09:10:52.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3117586222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c10ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3117586222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c102a40 0x7f5c4c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3117586222 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c103c40 0x7f5c4c1040a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 -- 172.21.15.61:0/3117586222 >> 172.21.15.61:0/3117586222 conn(0x7f5c4c0fe000 msgr2=0x7f5c4c100420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:52.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 -- 172.21.15.61:0/3117586222 shutdown_connections 2023-10-18T09:10:52.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.473+0000 7f5c51da8700 1 -- 172.21.15.61:0/3117586222 wait complete. 2023-10-18T09:10:52.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.474+0000 7f5c51da8700 1 Processor -- start 2023-10-18T09:10:52.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.474+0000 7f5c51da8700 1 -- start start 2023-10-18T09:10:52.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:52.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c103c40 0x7f5c4c198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:52.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:52.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1a04e0 con 0x7f5c4c1045e0 2023-10-18T09:10:52.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1a0620 con 0x7f5c4c102a40 2023-10-18T09:10:52.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c51da8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c1a07b0 con 0x7f5c4c103c40 2023-10-18T09:10:52.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:52.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:52.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:51356/0 (socket says 172.21.15.61:51356) 2023-10-18T09:10:52.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 -- 172.21.15.61:0/3030156458 learned_addr learned my addr 172.21.15.61:0/3030156458 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:10:52.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4affd700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c103c40 0x7f5c4c198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:52.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 -- 172.21.15.61:0/3030156458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c103c40 msgr2=0x7f5c4c198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:52.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c103c40 0x7f5c4c198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 -- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 msgr2=0x7f5c4c19ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:52.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c19ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4b7fe700 1 -- 172.21.15.61:0/3030156458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c400095f0 con 0x7f5c4c102a40 2023-10-18T09:10:52.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.475+0000 7f5c4bfff700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:10:52.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c4b7fe700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e950 tx=0x7f5c3c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:52.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c009770 con 0x7f5c4c102a40 2023-10-18T09:10:52.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3c00c7e0 con 0x7f5c4c102a40 2023-10-18T09:10:52.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c1a0a90 con 0x7f5c4c102a40 2023-10-18T09:10:52.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003c80 con 0x7f5c4c102a40 2023-10-18T09:10:52.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.476+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c1a0f90 con 0x7f5c4c102a40 2023-10-18T09:10:52.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.477+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c109ed0 con 0x7f5c4c102a40 2023-10-18T09:10:52.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.479+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5c3c00c950 con 0x7f5c4c102a40 2023-10-18T09:10:52.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.480+0000 7f5c48ff9700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 0x7f5c340a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:10:52.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.480+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c3c014070 con 0x7f5c4c102a40 2023-10-18T09:10:52.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.481+0000 7f5c4affd700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 0x7f5c340a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:10:52.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.481+0000 7f5c4affd700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 0x7f5c340a39b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c40003730 tx=0x7f5c4000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:10:52.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.484+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5c3c096450 con 0x7f5c4c102a40 2023-10-18T09:10:52.543 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:52 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/759547149' entity='client.admin' 2023-10-18T09:10:52.543 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:52 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:52.628 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.628 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.628 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.629 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.629 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.629 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.630 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:52 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/759547149' entity='client.admin' 2023-10-18T09:10:52.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:52 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:52.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.706+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f5c4c04e4c0 con 0x7f5c4c102a40 2023-10-18T09:10:52.711 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:52 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/759547149' entity='client.admin' 2023-10-18T09:10:52.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:52 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:52.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.712+0000 7f5c48ff9700 1 -- 172.21.15.61:0/3030156458 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5c3c099a70 con 0x7f5c4c102a40 2023-10-18T09:10:52.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.714+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 msgr2=0x7f5c340a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:52.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.714+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 0x7f5c340a39b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c40003730 tx=0x7f5c4000b540 comp rx=0 tx=0).stop 2023-10-18T09:10:52.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.714+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 msgr2=0x7f5c4c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:10:52.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.714+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e950 tx=0x7f5c3c00ec60 comp rx=0 tx=0).stop 2023-10-18T09:10:52.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 shutdown_connections 2023-10-18T09:10:52.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5c340a14f0 0x7f5c340a39b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5c4c1045e0 0x7f5c4c19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c4c102a40 0x7f5c4c1989a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 --2- 172.21.15.61:0/3030156458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c4c103c40 0x7f5c4c198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:10:52.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 >> 172.21.15.61:0/3030156458 conn(0x7f5c4c0fe000 msgr2=0x7f5c4c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:10:52.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 shutdown_connections 2023-10-18T09:10:52.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:10:52.715+0000 7f5c51da8700 1 -- 172.21.15.61:0/3030156458 wait complete. 2023-10-18T09:10:52.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi061') 2023-10-18T09:10:52.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'c578e0ac-6d95-11ee-8db6-212e2dc638e7', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi061') 2023-10-18T09:10:53.511 INFO:tasks.cephadm:Deploying OSDs... 2023-10-18T09:10:53.512 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:10:53.512 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T09:10:53.557 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T09:10:53.557 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_1 2023-10-18T09:10:53.642 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T09:10:53.642 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:53.642 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-18T09:10:53.642 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:53.642 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:53.643 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-10-18 09:09:31.133059109 +0000 2023-10-18T09:10:53.643 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-10-18 09:03:39.109752934 +0000 2023-10-18T09:10:53.643 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-10-18 09:03:39.109752934 +0000 2023-10-18T09:10:53.643 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-10-18 09:03:39.082753377 +0000 2023-10-18T09:10:53.643 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T09:10:53.752 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-10-18T09:10:53.753 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-10-18T09:10:53.753 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000295936 s, 1.7 MB/s 2023-10-18T09:10:53.755 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T09:10:53.801 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_2 2023-10-18T09:10:53.886 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T09:10:53.887 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:53.887 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-18T09:10:53.887 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:53.887 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:53.887 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-10-18 09:09:31.200058025 +0000 2023-10-18T09:10:53.888 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-10-18 09:03:39.965738866 +0000 2023-10-18T09:10:53.888 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-10-18 09:03:39.965738866 +0000 2023-10-18T09:10:53.888 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-10-18 09:03:39.938739310 +0000 2023-10-18T09:10:53.888 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T09:10:53.988 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:53 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' 2023-10-18T09:10:53.988 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:53 smithi061 ceph-mon[98573]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:53.994 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-10-18T09:10:53.994 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-10-18T09:10:53.995 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000292568 s, 1.8 MB/s 2023-10-18T09:10:53.997 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T09:10:54.043 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_3 2023-10-18T09:10:54.127 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T09:10:54.127 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:54.128 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-18T09:10:54.128 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:54.128 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:54.128 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-10-18 09:09:31.273056843 +0000 2023-10-18T09:10:54.128 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-10-18 09:03:40.858724190 +0000 2023-10-18T09:10:54.129 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-10-18 09:03:40.858724190 +0000 2023-10-18T09:10:54.129 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-10-18 09:03:40.831724634 +0000 2023-10-18T09:10:54.129 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T09:10:54.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:53 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' 2023-10-18T09:10:54.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:53 smithi170 ceph-mon[135150]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:53 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' 2023-10-18T09:10:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:53 smithi110 ceph-mon[134818]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:54.233 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-10-18T09:10:54.234 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-10-18T09:10:54.234 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000301356 s, 1.7 MB/s 2023-10-18T09:10:54.236 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T09:10:54.282 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_4 2023-10-18T09:10:54.326 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T09:10:54.327 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:54.327 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-18T09:10:54.327 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:54.327 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:54.327 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-10-18 09:09:31.340055759 +0000 2023-10-18T09:10:54.328 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-10-18 09:03:41.694710452 +0000 2023-10-18T09:10:54.328 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-10-18 09:03:41.694710452 +0000 2023-10-18T09:10:54.328 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-10-18 09:03:41.668710879 +0000 2023-10-18T09:10:54.328 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T09:10:54.434 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-10-18T09:10:54.434 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-10-18T09:10:54.434 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000307066 s, 1.7 MB/s 2023-10-18T09:10:54.436 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T09:10:54.482 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2023-10-18T09:10:54.483 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T09:10:54.504 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T09:10:54.504 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2023-10-18T09:10:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T09:10:54.564 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:54.565 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 392407 Links: 1 2023-10-18T09:10:54.565 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:54.565 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:54.565 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-10-18 09:10:30.964656546 +0000 2023-10-18T09:10:54.565 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-10-18 09:00:46.529037322 +0000 2023-10-18T09:10:54.566 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-10-18 09:00:46.529037322 +0000 2023-10-18T09:10:54.566 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-10-18T09:10:54.566 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T09:10:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-10-18T09:10:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-10-18T09:10:54.636 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000132471 s, 3.9 MB/s 2023-10-18T09:10:54.637 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T09:10:54.698 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2023-10-18T09:10:54.757 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T09:10:54.758 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:54.758 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 394046 Links: 1 2023-10-18T09:10:54.758 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:54.758 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:54.758 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-10-18 09:10:30.964656546 +0000 2023-10-18T09:10:54.759 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-10-18 09:00:47.056028510 +0000 2023-10-18T09:10:54.759 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-10-18 09:00:47.056028510 +0000 2023-10-18T09:10:54.759 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-10-18T09:10:54.759 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T09:10:54.826 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-10-18T09:10:54.826 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-10-18T09:10:54.827 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000116304 s, 4.4 MB/s 2023-10-18T09:10:54.828 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T09:10:54.887 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2023-10-18T09:10:54.944 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T09:10:54.945 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:54.945 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 389519 Links: 1 2023-10-18T09:10:54.945 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:54.945 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:54.946 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-10-18 09:10:30.964656546 +0000 2023-10-18T09:10:54.946 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-10-18 09:00:47.549020266 +0000 2023-10-18T09:10:54.946 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-10-18 09:00:47.549020266 +0000 2023-10-18T09:10:54.946 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-10-18T09:10:54.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T09:10:55.011 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-10-18T09:10:55.011 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-10-18T09:10:55.011 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.9009e-05 s, 5.2 MB/s 2023-10-18T09:10:55.012 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T09:10:55.070 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2023-10-18T09:10:55.127 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T09:10:55.127 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:55.127 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 6h/6d Inode: 389640 Links: 1 2023-10-18T09:10:55.128 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:55.128 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:55.128 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2023-10-18 09:10:30.965656529 +0000 2023-10-18T09:10:55.128 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2023-10-18 09:00:48.039012072 +0000 2023-10-18T09:10:55.128 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2023-10-18 09:00:48.039012072 +0000 2023-10-18T09:10:55.129 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2023-10-18T09:10:55.129 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T09:10:55.192 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2023-10-18T09:10:55.193 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2023-10-18T09:10:55.193 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000115692 s, 4.4 MB/s 2023-10-18T09:10:55.194 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T09:10:55.252 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-18T09:10:55.252 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2023-10-18T09:10:55.273 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-18T09:10:55.273 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2023-10-18T09:10:55.335 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-18T09:10:55.335 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:55.335 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 392760 Links: 1 2023-10-18T09:10:55.336 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:55.336 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:55.336 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-18 09:10:28.848595143 +0000 2023-10-18T09:10:55.336 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-18 09:00:47.654373948 +0000 2023-10-18T09:10:55.336 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-18 09:00:47.654373948 +0000 2023-10-18T09:10:55.337 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-18T09:10:55.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-18T09:10:55.407 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-18T09:10:55.407 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-18T09:10:55.408 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000134664 s, 3.8 MB/s 2023-10-18T09:10:55.408 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-18T09:10:55.469 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2023-10-18T09:10:55.528 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-18T09:10:55.529 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:55.529 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 395325 Links: 1 2023-10-18T09:10:55.529 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:55.529 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:55.530 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-18 09:10:28.848595143 +0000 2023-10-18T09:10:55.530 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-18 09:00:48.156362254 +0000 2023-10-18T09:10:55.530 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-18 09:00:48.156362254 +0000 2023-10-18T09:10:55.530 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-18T09:10:55.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-18T09:10:55.597 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-18T09:10:55.597 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-18T09:10:55.597 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000146399 s, 3.5 MB/s 2023-10-18T09:10:55.598 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-18T09:10:55.658 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2023-10-18T09:10:55.763 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-18T09:10:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:55.764 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 393471 Links: 1 2023-10-18T09:10:55.764 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:55.765 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:55.765 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-18 09:10:28.848595143 +0000 2023-10-18T09:10:55.765 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-18 09:00:48.653350677 +0000 2023-10-18T09:10:55.766 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-18 09:00:48.653350677 +0000 2023-10-18T09:10:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-18T09:10:55.766 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-18T09:10:55.831 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-18T09:10:55.831 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-18T09:10:55.831 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000121594 s, 4.2 MB/s 2023-10-18T09:10:55.832 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-18T09:10:55.891 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2023-10-18T09:10:55.949 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-18T09:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-18T09:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 395421 Links: 1 2023-10-18T09:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-18T09:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-18T09:10:55.950 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-18 09:10:28.848595143 +0000 2023-10-18T09:10:55.951 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-18 09:00:49.149339124 +0000 2023-10-18T09:10:55.951 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-18 09:00:49.149339124 +0000 2023-10-18T09:10:55.951 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-18T09:10:55.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-18T09:10:56.015 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-18T09:10:56.016 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-18T09:10:56.016 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00010158 s, 5.0 MB/s 2023-10-18T09:10:56.017 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-18T09:10:56.074 INFO:tasks.cephadm:Deploying osd.0 on smithi061 with /dev/vg_nvme/lv_4... 2023-10-18T09:10:56.075 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T09:10:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:55 smithi170 ceph-mon[135150]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:56.143 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:55 smithi061 ceph-mon[98573]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:55 smithi110 ceph-mon[134818]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:56.827 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:10:57.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:57 smithi110 ceph-mon[134818]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:57.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:57 smithi061 ceph-mon[98573]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:57.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:57 smithi170 ceph-mon[135150]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:10:58.969 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T09:10:58.969 INFO:teuthology.orchestra.run.smithi061.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T09:10:58.970 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 10+0 records in 2023-10-18T09:10:58.970 INFO:teuthology.orchestra.run.smithi061.stderr:10+0 records out 2023-10-18T09:10:58.970 INFO:teuthology.orchestra.run.smithi061.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.030844 s, 340 MB/s 2023-10-18T09:10:58.970 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 2023-10-18T09:10:58.971 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping successful for: 2023-10-18T09:10:59.528 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi061:vg_nvme/lv_4 2023-10-18T09:11:00.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:10:59 smithi170 ceph-mon[135150]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:00.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:10:59 smithi110 ceph-mon[134818]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:00.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:10:59 smithi061 ceph-mon[98573]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:00.273 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:11:01.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 -- 172.21.15.61:0/99756997 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548074820 msgr2=0x7f2548072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:01.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 --2- 172.21.15.61:0/99756997 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548074820 0x7f2548072c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2538009990 tx=0x7f2538009ca0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 -- 172.21.15.61:0/99756997 shutdown_connections 2023-10-18T09:11:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 --2- 172.21.15.61:0/99756997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 0x7f254810aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 --2- 172.21.15.61:0/99756997 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548074820 0x7f2548072c80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 --2- 172.21.15.61:0/99756997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548073250 0x7f25480736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 -- 172.21.15.61:0/99756997 >> 172.21.15.61:0/99756997 conn(0x7f25480fbe00 msgr2=0x7f25480fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:11:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 -- 172.21.15.61:0/99756997 shutdown_connections 2023-10-18T09:11:01.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.845+0000 7f254e4ef700 1 -- 172.21.15.61:0/99756997 wait complete. 2023-10-18T09:11:01.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 Processor -- start 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 -- start start 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548074820 0x7f2548198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 0x7f254819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25481a04c0 con 0x7f2548073250 2023-10-18T09:11:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25481a0600 con 0x7f2548108710 2023-10-18T09:11:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.847+0000 7f254e4ef700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25481a0790 con 0x7f2548074820 2023-10-18T09:11:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:47504/0 (socket says 172.21.15.61:47504) 2023-10-18T09:11:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 -- 172.21.15.61:0/699568633 learned_addr learned my addr 172.21.15.61:0/699568633 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:11:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f254ca8c700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 0x7f254819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:01.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f25477fe700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548074820 0x7f2548198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:01.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 -- 172.21.15.61:0/699568633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548074820 msgr2=0x7f2548198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:01.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548074820 0x7f2548198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 -- 172.21.15.61:0/699568633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 msgr2=0x7f254819fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 0x7f254819fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 -- 172.21.15.61:0/699568633 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380095f0 con 0x7f2548073250 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.848+0000 7f2547fff700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f25380095c0 tx=0x7f2538005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.849+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253801d070 con 0x7f2548073250 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.849+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2538021410 con 0x7f2548073250 2023-10-18T09:11:01.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.849+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538017650 con 0x7f2548073250 2023-10-18T09:11:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.849+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25481a0a10 con 0x7f2548073250 2023-10-18T09:11:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.849+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25481a0dd0 con 0x7f2548073250 2023-10-18T09:11:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25481074d0 con 0x7f2548073250 2023-10-18T09:11:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.852+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2538021580 con 0x7f2548073250 2023-10-18T09:11:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.853+0000 7f25457fa700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 0x7f25300a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.853+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f25380ccbd0 con 0x7f2548073250 2023-10-18T09:11:01.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.853+0000 7f25477fe700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 0x7f25300a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:01.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.854+0000 7f25477fe700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 0x7f25300a38b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2548074550 tx=0x7f253c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:01.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:01.856+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2538097ef0 con 0x7f2548073250 2023-10-18T09:11:02.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:01 smithi061 ceph-mon[98573]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:02.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:02.075+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2548060c30 con 0x7f25300a13f0 2023-10-18T09:11:02.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:01 smithi170 ceph-mon[135150]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:01 smithi110 ceph-mon[134818]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:03.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:03.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:03.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:03.197 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:03.198 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:03.198 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:03.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:03.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:03.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:04.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:03 smithi170 ceph-mon[135150]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:04.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:03 smithi170 ceph-mon[135150]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:04.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:03 smithi061 ceph-mon[98573]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:04.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:03 smithi061 ceph-mon[98573]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:04.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:03 smithi110 ceph-mon[134818]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:04.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:03 smithi110 ceph-mon[134818]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:05 smithi170 ceph-mon[135150]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:05 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]: dispatch 2023-10-18T09:11:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:05 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]': finished 2023-10-18T09:11:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:05 smithi170 ceph-mon[135150]: osdmap e5: 1 total, 0 up, 1 in 2023-10-18T09:11:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:05 smithi110 ceph-mon[134818]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:05 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]: dispatch 2023-10-18T09:11:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:05 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]': finished 2023-10-18T09:11:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:05 smithi110 ceph-mon[134818]: osdmap e5: 1 total, 0 up, 1 in 2023-10-18T09:11:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:05 smithi061 ceph-mon[98573]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:05 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]: dispatch 2023-10-18T09:11:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:05 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2620489737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a451299-d658-4830-9f80-9ba0776f9122"}]': finished 2023-10-18T09:11:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:05 smithi061 ceph-mon[98573]: osdmap e5: 1 total, 0 up, 1 in 2023-10-18T09:11:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:07.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:06 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/19068469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:06 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/19068469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:07.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:06 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/19068469' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:07 smithi170 ceph-mon[135150]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:07 smithi110 ceph-mon[134818]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:07 smithi061 ceph-mon[98573]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:10.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:09 smithi061 ceph-mon[98573]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:10.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:09 smithi170 ceph-mon[135150]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:09 smithi110 ceph-mon[134818]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:11 smithi061 ceph-mon[98573]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:11 smithi170 ceph-mon[135150]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:11 smithi110 ceph-mon[134818]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:13.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T09:11:13.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:13.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T09:11:13.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-18T09:11:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:14.001 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:13 smithi061 ceph-mon[98573]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:14.002 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:13 smithi061 ceph-mon[98573]: Deploying daemon osd.0 on smithi061 2023-10-18T09:11:14.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:13 smithi170 ceph-mon[135150]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:14.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:13 smithi170 ceph-mon[135150]: Deploying daemon osd.0 on smithi061 2023-10-18T09:11:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:13 smithi110 ceph-mon[134818]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:13 smithi110 ceph-mon[134818]: Deploying daemon osd.0 on smithi061 2023-10-18T09:11:16.117 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:15 smithi061 ceph-mon[98573]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:15 smithi170 ceph-mon[135150]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:15 smithi110 ceph-mon[134818]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:17.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:17 smithi061 ceph-mon[98573]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:17.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:17 smithi110 ceph-mon[134818]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:17 smithi170 ceph-mon[135150]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:20.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:19 smithi061 ceph-mon[98573]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:20.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:20.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:19 smithi170 ceph-mon[135150]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:20.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:19 smithi110 ceph-mon[134818]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:22.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:21 smithi061 ceph-mon[98573]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:21 smithi170 ceph-mon[135150]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:21 smithi110 ceph-mon[134818]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:22.850 INFO:teuthology.orchestra.run.smithi061.stdout:Created osd(s) 0 on host 'smithi061' 2023-10-18T09:11:22.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.847+0000 7f25457fa700 1 -- 172.21.15.61:0/699568633 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2548060c30 con 0x7f25300a13f0 2023-10-18T09:11:22.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 msgr2=0x7f25300a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:22.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 0x7f25300a38b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2548074550 tx=0x7f253c00b410 comp rx=0 tx=0).stop 2023-10-18T09:11:22.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 msgr2=0x7f25481989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:22.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f25380095c0 tx=0x7f2538005740 comp rx=0 tx=0).stop 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 shutdown_connections 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f25300a13f0 0x7f25300a38b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2548108710 0x7f254819fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2548073250 0x7f25481989d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 --2- 172.21.15.61:0/699568633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2548074820 0x7f2548198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:22.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 >> 172.21.15.61:0/699568633 conn(0x7f25480fbe00 msgr2=0x7f2548104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:11:22.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 shutdown_connections 2023-10-18T09:11:22.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:11:22.850+0000 7f254e4ef700 1 -- 172.21.15.61:0/699568633 wait complete. 2023-10-18T09:11:23.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:23.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:23.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:23.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:23.418 DEBUG:teuthology.orchestra.run.smithi061:osd.0> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.0.service 2023-10-18T09:11:23.422 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_4... 2023-10-18T09:11:23.423 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T09:11:23.506 INFO:journalctl@ceph.osd.0.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:11:23.988 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:11:24.238 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:23 smithi110 ceph-mon[134818]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T09:11:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:23 smithi061 ceph-mon[98573]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T09:11:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:23 smithi170 ceph-mon[135150]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-18T09:11:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:24 smithi061 ceph-mon[98573]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T09:11:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:24 smithi061 ceph-mon[98573]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T09:11:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:24 smithi061 ceph-mon[98573]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:11:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:24 smithi061 ceph-mon[98573]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:25.255 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0[117726]: 2023-10-18T09:11:25.000+0000 7fc5b0b3f700 -1 osd.0 0 waiting for initial osdmap 2023-10-18T09:11:25.256 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0[117726]: 2023-10-18T09:11:25.004+0000 7fc5aa930700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:11:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:24 smithi170 ceph-mon[135150]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T09:11:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:24 smithi170 ceph-mon[135150]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T09:11:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:24 smithi170 ceph-mon[135150]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:11:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:24 smithi170 ceph-mon[135150]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:24 smithi110 ceph-mon[134818]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-18T09:11:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:24 smithi110 ceph-mon[134818]: osdmap e6: 1 total, 0 up, 1 in 2023-10-18T09:11:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:24 smithi110 ceph-mon[134818]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:11:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:24 smithi110 ceph-mon[134818]: pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T09:11:25.542 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T09:11:25.543 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2023-10-18T09:11:25.543 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2023-10-18T09:11:25.543 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00817138 s, 1.3 GB/s 2023-10-18T09:11:25.544 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2023-10-18T09:11:25.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2023-10-18T09:11:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:25 smithi110 ceph-mon[134818]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:11:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:25 smithi110 ceph-mon[134818]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T09:11:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-mon[98573]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:11:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-mon[98573]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T09:11:26.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:25 smithi170 ceph-mon[135150]: from='osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:11:26.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:25 smithi170 ceph-mon[135150]: osdmap e7: 1 total, 0 up, 1 in 2023-10-18T09:11:26.415 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:26.496 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:11:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:11:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:11:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] boot 2023-10-18T09:11:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: osdmap e8: 1 total, 1 up, 1 in 2023-10-18T09:11:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:27.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi061 2023-10-18T09:11:27.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:27.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 17271M 2023-10-18T09:11:27.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:27.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:27.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:11:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:11:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] boot 2023-10-18T09:11:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: osdmap e8: 1 total, 1 up, 1 in 2023-10-18T09:11:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi061 2023-10-18T09:11:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:27.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 17271M 2023-10-18T09:11:27.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:27.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:27.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:11:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:11:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: osd.0 [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] boot 2023-10-18T09:11:27.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: osdmap e8: 1 total, 1 up, 1 in 2023-10-18T09:11:27.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-18T09:11:27.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-18T09:11:27.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi061 2023-10-18T09:11:27.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:27.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 17271M 2023-10-18T09:11:27.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:27.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:27.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:27.736 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.731+0000 7f6a7b709700 1 -- 172.21.15.110:0/1015870501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 msgr2=0x7f6a74107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:27.737 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.731+0000 7f6a7b709700 1 --2- 172.21.15.110:0/1015870501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a74107b80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00b0d0 tx=0x7f6a6c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.737 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- 172.21.15.110:0/1015870501 shutdown_connections 2023-10-18T09:11:27.737 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- 172.21.15.110:0/1015870501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a74107b80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.737 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- 172.21.15.110:0/1015870501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a740ff730 0x7f6a740ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.737 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- 172.21.15.110:0/1015870501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a74100930 0x7f6a74100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.738 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- 172.21.15.110:0/1015870501 >> 172.21.15.110:0/1015870501 conn(0x7f6a740facf0 msgr2=0x7f6a740fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:11:27.738 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- 172.21.15.110:0/1015870501 shutdown_connections 2023-10-18T09:11:27.738 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- 172.21.15.110:0/1015870501 wait complete. 2023-10-18T09:11:27.738 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 Processor -- start 2023-10-18T09:11:27.738 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- start start 2023-10-18T09:11:27.739 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a740ff730 0x7f6a741967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:27.739 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a74100930 0x7f6a74196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:27.739 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:27.739 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7419e310 con 0x7f6a74100930 2023-10-18T09:11:27.739 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7419e450 con 0x7f6a741012d0 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.732+0000 7f6a7b709700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7419e5e0 con 0x7f6a740ff730 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60148/0 (socket says 172.21.15.110:60148) 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 -- 172.21.15.110:0/845390526 learned_addr learned my addr 172.21.15.110:0/845390526 (peer_addr_for_me v2:172.21.15.110:0/0) 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a794a5700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a740ff730 0x7f6a741967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:27.740 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a78ca4700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a74100930 0x7f6a74196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:27.741 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 -- 172.21.15.110:0/845390526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a740ff730 msgr2=0x7f6a741967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:27.741 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a740ff730 0x7f6a741967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.741 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 -- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a74100930 msgr2=0x7f6a74196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:27.741 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a74100930 0x7f6a74196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:27.741 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 -- 172.21.15.110:0/845390526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009d00 con 0x7f6a741012d0 2023-10-18T09:11:27.742 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a79ca6700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c007bd0 tx=0x7f6a6c009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:27.742 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00e040 con 0x7f6a741012d0 2023-10-18T09:11:27.742 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c01dd50 con 0x7f6a741012d0 2023-10-18T09:11:27.742 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0123f0 con 0x7f6a741012d0 2023-10-18T09:11:27.742 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7419e860 con 0x7f6a741012d0 2023-10-18T09:11:27.743 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.733+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7419ecd0 con 0x7f6a741012d0 2023-10-18T09:11:27.744 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.734+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7404e4c0 con 0x7f6a741012d0 2023-10-18T09:11:27.744 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.735+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f6a6c01d890 con 0x7f6a741012d0 2023-10-18T09:11:27.744 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.736+0000 7f6a6a7fc700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 0x7f6a600a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:27.744 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.736+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f6a6c0cd800 con 0x7f6a741012d0 2023-10-18T09:11:27.744 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.736+0000 7f6a794a5700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 0x7f6a600a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:27.745 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.736+0000 7f6a794a5700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 0x7f6a600a39b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a74100790 tx=0x7f6a7000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:27.745 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.739+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a6c0988c0 con 0x7f6a741012d0 2023-10-18T09:11:27.936 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:27.933+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a740688a0 con 0x7f6a600a14f0 2023-10-18T09:11:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:27 smithi061 ceph-mon[98573]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T09:11:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:27 smithi170 ceph-mon[135150]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T09:11:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:27 smithi110 ceph-mon[134818]: osdmap e9: 1 total, 1 up, 1 in 2023-10-18T09:11:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:28 smithi061 ceph-mon[98573]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:28 smithi061 ceph-mon[98573]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:28 smithi170 ceph-mon[135150]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:28 smithi170 ceph-mon[135150]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:28 smithi110 ceph-mon[134818]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:28 smithi110 ceph-mon[134818]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:30.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:30 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/1897420426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:30 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:30 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]': finished 2023-10-18T09:11:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:30 smithi170 ceph-mon[135150]: osdmap e10: 2 total, 1 up, 2 in 2023-10-18T09:11:30.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:30.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:30 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/1897420426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:30 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:30 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]': finished 2023-10-18T09:11:30.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:30 smithi110 ceph-mon[134818]: osdmap e10: 2 total, 1 up, 2 in 2023-10-18T09:11:30.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:30.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/1897420426' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:30 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]: dispatch 2023-10-18T09:11:30.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:30 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5535c5e8-804e-4d87-915d-00862ef1b386"}]': finished 2023-10-18T09:11:30.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:30 smithi061 ceph-mon[98573]: osdmap e10: 2 total, 1 up, 2 in 2023-10-18T09:11:30.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:31 smithi170 ceph-mon[135150]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:31 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/4062050709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:31 smithi110 ceph-mon[134818]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:31 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/4062050709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:31 smithi061 ceph-mon[98573]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:31 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/4062050709' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:34.098 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:33 smithi110 ceph-mon[134818]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:33 smithi170 ceph-mon[135150]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:33 smithi061 ceph-mon[98573]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:36.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:35 smithi170 ceph-mon[135150]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:35 smithi110 ceph-mon[134818]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:36.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:35 smithi061 ceph-mon[98573]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:37 smithi110 ceph-mon[134818]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T09:11:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:37.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:37 smithi110 ceph-mon[134818]: Deploying daemon osd.1 on smithi110 2023-10-18T09:11:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:37 smithi061 ceph-mon[98573]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T09:11:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:37.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:37 smithi061 ceph-mon[98573]: Deploying daemon osd.1 on smithi110 2023-10-18T09:11:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:37 smithi170 ceph-mon[135150]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:37.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-18T09:11:37.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:37.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:37 smithi170 ceph-mon[135150]: Deploying daemon osd.1 on smithi110 2023-10-18T09:11:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:39 smithi170 ceph-mon[135150]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:39 smithi110 ceph-mon[134818]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:40.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:39 smithi061 ceph-mon[98573]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:42.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:41 smithi170 ceph-mon[135150]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:41 smithi110 ceph-mon[134818]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:41 smithi061 ceph-mon[98573]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:44.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:44.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:44.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.205 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.203+0000 7f6a6a7fc700 1 -- 172.21.15.110:0/845390526 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6a740688a0 con 0x7f6a600a14f0 2023-10-18T09:11:44.205 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2023-10-18T09:11:44.207 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 msgr2=0x7f6a600a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:44.207 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 0x7f6a600a39b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a74100790 tx=0x7f6a7000b410 comp rx=0 tx=0).stop 2023-10-18T09:11:44.207 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 msgr2=0x7f6a7419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:44.208 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c007bd0 tx=0x7f6a6c009440 comp rx=0 tx=0).stop 2023-10-18T09:11:44.208 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 shutdown_connections 2023-10-18T09:11:44.208 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6a600a14f0 0x7f6a600a39b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:44.208 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a741012d0 0x7f6a7419dd40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:44.208 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a740ff730 0x7f6a741967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:44.209 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 --2- 172.21.15.110:0/845390526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6a74100930 0x7f6a74196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:44.209 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 >> 172.21.15.110:0/845390526 conn(0x7f6a740facf0 msgr2=0x7f6a74068190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:11:44.209 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 shutdown_connections 2023-10-18T09:11:44.209 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:11:44.205+0000 7f6a7b709700 1 -- 172.21.15.110:0/845390526 wait complete. 2023-10-18T09:11:44.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:44.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:44.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:44.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:44.686 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.1.service 2023-10-18T09:11:44.689 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2023-10-18T09:11:44.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-18T09:11:44.718 INFO:journalctl@ceph.osd.1.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:11:45.254 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:11:45.429 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:45 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.429 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:45 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.429 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:45 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:45.429 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:45 smithi110 ceph-mon[134818]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:45.430 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:11:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:11:45.335+0000 7f29128c8640 -1 osd.1 0 log_to_monitors true 2023-10-18T09:11:45.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:45 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:45 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:45 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:45.472 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:45 smithi170 ceph-mon[135150]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:45.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:11:45.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:45 smithi061 ceph-mon[98573]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:46 smithi110 ceph-mon[134818]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:46 smithi110 ceph-mon[134818]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:46 smithi061 ceph-mon[98573]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:46 smithi061 ceph-mon[98573]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:46 smithi170 ceph-mon[135150]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:46 smithi170 ceph-mon[135150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-18T09:11:46.768 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-18T09:11:46.769 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-18T09:11:46.769 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-10-18T09:11:46.769 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-10-18T09:11:46.769 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00790529 s, 1.3 GB/s 2023-10-18T09:11:46.769 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-10-18T09:11:47.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2023-10-18T09:11:47.462 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:11:47.188+0000 7f2907f3f700 -1 osd.1 0 waiting for initial osdmap 2023-10-18T09:11:47.462 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:11:47.193+0000 7f290052d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:11:47.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T09:11:47.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T09:11:47.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:47.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:47.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:47.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:47 smithi110 ceph-mon[134818]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:11:47.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T09:11:47.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T09:11:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:47.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:47 smithi061 ceph-mon[98573]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:11:47.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-18T09:11:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: osdmap e11: 2 total, 1 up, 2 in 2023-10-18T09:11:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:11:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:11:47.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:47 smithi170 ceph-mon[135150]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:11:47.765 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:11:48.239 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi110 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi110 to 17168M 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:48.240 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:48.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:48.241 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:48 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi110 2023-10-18T09:11:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi110 to 17168M 2023-10-18T09:11:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T09:11:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:48.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:48.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:48 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi110 2023-10-18T09:11:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi110 to 17168M 2023-10-18T09:11:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: osdmap e12: 2 total, 1 up, 2 in 2023-10-18T09:11:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:48.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:11:48.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:48 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:11:49.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 -- 172.21.15.170:0/1398514402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28100850 msgr2=0x7fea28100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:49.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1398514402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28100850 0x7fea28100c70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fea18009990 tx=0x7fea18009ca0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 -- 172.21.15.170:0/1398514402 shutdown_connections 2023-10-18T09:11:49.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1398514402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 0x7fea28108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1398514402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28100850 0x7fea28100c70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1398514402 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28101a50 0x7fea28101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.010+0000 7fea2df7c700 1 -- 172.21.15.170:0/1398514402 >> 172.21.15.170:0/1398514402 conn(0x7fea280fbe10 msgr2=0x7fea280fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:11:49.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.011+0000 7fea2df7c700 1 -- 172.21.15.170:0/1398514402 shutdown_connections 2023-10-18T09:11:49.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.011+0000 7fea2df7c700 1 -- 172.21.15.170:0/1398514402 wait complete. 2023-10-18T09:11:49.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.011+0000 7fea2df7c700 1 Processor -- start 2023-10-18T09:11:49.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 -- start start 2023-10-18T09:11:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28100850 0x7fea28196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 0x7fea2819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2819e380 con 0x7fea28100850 2023-10-18T09:11:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2819e4c0 con 0x7fea281023f0 2023-10-18T09:11:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea2df7c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2819e650 con 0x7fea28101a50 2023-10-18T09:11:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54230/0 (socket says 172.21.15.170:54230) 2023-10-18T09:11:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 -- 172.21.15.170:0/1731717030 learned_addr learned my addr 172.21.15.170:0/1731717030 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-18T09:11:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea277fe700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28100850 0x7fea28196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 -- 172.21.15.170:0/1731717030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 msgr2=0x7fea2819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea27fff700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 0x7fea2819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 0x7fea2819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 -- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28100850 msgr2=0x7fea28196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:11:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28100850 0x7fea28196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:11:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 -- 172.21.15.170:0/1731717030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180095f0 con 0x7fea28101a50 2023-10-18T09:11:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea26ffd700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b560 tx=0x7fea1c00b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c011830 con 0x7fea28101a50 2023-10-18T09:11:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea1c011e50 con 0x7fea28101a50 2023-10-18T09:11:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.012+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00f2e0 con 0x7fea28101a50 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.013+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2819e930 con 0x7fea28101a50 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.013+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2819ee30 con 0x7fea28101a50 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.013+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28107d50 con 0x7fea28101a50 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.014+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fea1c011990 con 0x7fea28101a50 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.015+0000 7fea24ff9700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 0x7fea100a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:11:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.015+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fea1c0caf30 con 0x7fea28101a50 2023-10-18T09:11:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.015+0000 7fea277fe700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 0x7fea100a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:11:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.016+0000 7fea277fe700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 0x7fea100a3930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea180038c0 tx=0x7fea18005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:11:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.020+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fea1c095de0 con 0x7fea28101a50 2023-10-18T09:11:49.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:11:49.218+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fea2802d020 con 0x7fea100a1470 2023-10-18T09:11:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:11:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:11:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] boot 2023-10-18T09:11:49.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: osdmap e13: 2 total, 2 up, 2 in 2023-10-18T09:11:49.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:49 smithi110 ceph-mon[134818]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:11:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:11:49.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] boot 2023-10-18T09:11:49.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: osdmap e13: 2 total, 2 up, 2 in 2023-10-18T09:11:49.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:49 smithi061 ceph-mon[98573]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:11:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:11:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: osd.1 [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] boot 2023-10-18T09:11:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: osdmap e13: 2 total, 2 up, 2 in 2023-10-18T09:11:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-18T09:11:49.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:49 smithi170 ceph-mon[135150]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-18T09:11:50.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:50 smithi170 ceph-mon[135150]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T09:11:50.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:50 smithi170 ceph-mon[135150]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:50.331 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:50.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:50.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:50.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:50 smithi110 ceph-mon[134818]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T09:11:50.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:50 smithi110 ceph-mon[134818]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:50.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:50.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:50.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:50.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:50 smithi061 ceph-mon[98573]: osdmap e14: 2 total, 2 up, 2 in 2023-10-18T09:11:50.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:50 smithi061 ceph-mon[98573]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:11:50.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:11:50.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:11:50.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:51.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:51 smithi061 ceph-mon[98573]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:51 smithi170 ceph-mon[135150]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:51 smithi110 ceph-mon[134818]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:52.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/3000365042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]': finished 2023-10-18T09:11:52.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: osdmap e15: 3 total, 2 up, 3 in 2023-10-18T09:11:52.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:11:52.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:52 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/1009124420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:52.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/3000365042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]': finished 2023-10-18T09:11:52.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: osdmap e15: 3 total, 2 up, 3 in 2023-10-18T09:11:52.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:11:52.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:52 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/1009124420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:52.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/3000365042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]: dispatch 2023-10-18T09:11:52.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "222ac61b-a988-44c1-b86e-b31756c1f539"}]': finished 2023-10-18T09:11:52.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: osdmap e15: 3 total, 2 up, 3 in 2023-10-18T09:11:52.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:11:52.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:52 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/1009124420' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:11:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:53 smithi110 ceph-mon[134818]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:53 smithi061 ceph-mon[98573]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:53.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:53 smithi170 ceph-mon[135150]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:55 smithi170 ceph-mon[135150]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:55 smithi110 ceph-mon[134818]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:55 smithi061 ceph-mon[98573]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:57.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:57 smithi170 ceph-mon[135150]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:57.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:57 smithi110 ceph-mon[134818]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:57.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:57 smithi061 ceph-mon[98573]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:58.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T09:11:58.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T09:11:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:58.818 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-18T09:11:58.819 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:11:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:59 smithi110 ceph-mon[134818]: Deploying daemon osd.2 on smithi170 2023-10-18T09:11:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:11:59 smithi110 ceph-mon[134818]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:59.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:59 smithi170 ceph-mon[135150]: Deploying daemon osd.2 on smithi170 2023-10-18T09:11:59.722 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:11:59 smithi170 ceph-mon[135150]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:11:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:59 smithi061 ceph-mon[98573]: Deploying daemon osd.2 on smithi170 2023-10-18T09:11:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:11:59 smithi061 ceph-mon[98573]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:02.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:01 smithi170 ceph-mon[135150]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:01 smithi110 ceph-mon[134818]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:02.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:01 smithi061 ceph-mon[98573]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:03.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:03.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:03 smithi170 ceph-mon[135150]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:03.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:03 smithi110 ceph-mon[134818]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:03 smithi061 ceph-mon[98573]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:05.836 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2023-10-18T09:12:05.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.832+0000 7fea24ff9700 1 -- 172.21.15.170:0/1731717030 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fea2802d020 con 0x7fea100a1470 2023-10-18T09:12:05.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 msgr2=0x7fea100a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:05.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 0x7fea100a3930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fea180038c0 tx=0x7fea18005fb0 comp rx=0 tx=0).stop 2023-10-18T09:12:05.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 msgr2=0x7fea28196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:05.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b560 tx=0x7fea1c00b920 comp rx=0 tx=0).stop 2023-10-18T09:12:05.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 shutdown_connections 2023-10-18T09:12:05.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fea100a1470 0x7fea100a3930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:05.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea281023f0 0x7fea2819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:05.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.835+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fea28100850 0x7fea28196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:05.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.836+0000 7fea2df7c700 1 --2- 172.21.15.170:0/1731717030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea28101a50 0x7fea28196d80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:05.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.836+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 >> 172.21.15.170:0/1731717030 conn(0x7fea280fbe10 msgr2=0x7fea28104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:05.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.836+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 shutdown_connections 2023-10-18T09:12:05.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:05.836+0000 7fea2df7c700 1 -- 172.21.15.170:0/1731717030 wait complete. 2023-10-18T09:12:05.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:05.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:05.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:05.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:05.995 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:05.996 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:05.996 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:05.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:05.997 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:06.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:06.256 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.2.service 2023-10-18T09:12:06.259 INFO:tasks.cephadm:Deploying osd.3 on smithi061 with /dev/vg_nvme/lv_3... 2023-10-18T09:12:06.259 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T09:12:06.292 INFO:journalctl@ceph.osd.2.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:12:06.822 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:12:06.560+0000 7fecb2f1e640 -1 osd.2 0 log_to_monitors true 2023-10-18T09:12:07.016 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:12:07.619 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:07 smithi061 ceph-mon[98573]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.620 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:07 smithi061 ceph-mon[98573]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.620 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:07 smithi061 ceph-mon[98573]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:07.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:07 smithi110 ceph-mon[134818]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:07 smithi110 ceph-mon[134818]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:07 smithi110 ceph-mon[134818]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:07.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:07 smithi170 ceph-mon[135150]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:07 smithi170 ceph-mon[135150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-18T09:12:07.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:07 smithi170 ceph-mon[135150]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:08 smithi110 ceph-mon[134818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T09:12:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:08 smithi110 ceph-mon[134818]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:08 smithi110 ceph-mon[134818]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T09:12:08.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:08 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:08.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:08 smithi110 ceph-mon[134818]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:08.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:08 smithi061 ceph-mon[98573]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T09:12:08.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:08 smithi061 ceph-mon[98573]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:08.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:08 smithi061 ceph-mon[98573]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T09:12:08.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:08 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:08.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:08 smithi061 ceph-mon[98573]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:08.857 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:12:08.435+0000 7feca8595700 -1 osd.2 0 waiting for initial osdmap 2023-10-18T09:12:08.858 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:12:08.440+0000 7feca2386700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:12:08.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-mon[135150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-18T09:12:08.858 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-mon[135150]: from='osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:08.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-mon[135150]: osdmap e16: 3 total, 2 up, 3 in 2023-10-18T09:12:08.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:08.859 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:08 smithi170 ceph-mon[135150]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:12:09.063 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T09:12:09.063 INFO:teuthology.orchestra.run.smithi061.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T09:12:09.063 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 10+0 records in 2023-10-18T09:12:09.064 INFO:teuthology.orchestra.run.smithi061.stderr:10+0 records out 2023-10-18T09:12:09.064 INFO:teuthology.orchestra.run.smithi061.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0571902 s, 183 MB/s 2023-10-18T09:12:09.064 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping successful for: 2023-10-18T09:12:09.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi061:vg_nvme/lv_3 2023-10-18T09:12:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:12:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T09:12:09.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:09.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:09.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:12:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T09:12:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:09.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:09.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:09.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:12:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: osdmap e17: 3 total, 2 up, 3 in 2023-10-18T09:12:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-18T09:12:09.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:09.895 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:10.368 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:12:10.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:12:10.708 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:12:10.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi170 2023-10-18T09:12:10.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi170 to 17168M 2023-10-18T09:12:10.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] boot 2023-10-18T09:12:10.709 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: osdmap e18: 3 total, 3 up, 3 in 2023-10-18T09:12:10.710 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:10.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:12:10.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:12:10.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi170 2023-10-18T09:12:10.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi170 to 17168M 2023-10-18T09:12:10.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] boot 2023-10-18T09:12:10.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: osdmap e18: 3 total, 3 up, 3 in 2023-10-18T09:12:10.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:10.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:12:10.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:12:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi170 2023-10-18T09:12:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi170 to 17168M 2023-10-18T09:12:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: osd.2 [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] boot 2023-10-18T09:12:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: osdmap e18: 3 total, 3 up, 3 in 2023-10-18T09:12:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-18T09:12:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:11 smithi061 ceph-mon[98573]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T09:12:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:11 smithi061 ceph-mon[98573]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:11 smithi110 ceph-mon[134818]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T09:12:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:11 smithi110 ceph-mon[134818]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:11.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:11 smithi170 ceph-mon[135150]: osdmap e19: 3 total, 3 up, 3 in 2023-10-18T09:12:11.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:11 smithi170 ceph-mon[135150]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:11.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:11.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 -- 172.21.15.61:0/2337820000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884100840 msgr2=0x7f0884100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:11.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2337820000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884100840 0x7f0884100c60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08800099c0 tx=0x7f0880009cd0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 -- 172.21.15.61:0/2337820000 shutdown_connections 2023-10-18T09:12:11.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2337820000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08841023e0 0x7f0884108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2337820000 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884100840 0x7f0884100c60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2337820000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0884101a40 0x7f0884101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 -- 172.21.15.61:0/2337820000 >> 172.21.15.61:0/2337820000 conn(0x7f08840fbe00 msgr2=0x7f08840fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:11.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 -- 172.21.15.61:0/2337820000 shutdown_connections 2023-10-18T09:12:11.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.942+0000 7f088c7b6700 1 -- 172.21.15.61:0/2337820000 wait complete. 2023-10-18T09:12:11.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.943+0000 7f088c7b6700 1 Processor -- start 2023-10-18T09:12:11.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.943+0000 7f088c7b6700 1 -- start start 2023-10-18T09:12:11.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 0x7f08841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:11.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:11.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 0x7f088419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:11.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08841a04f0 con 0x7f0884101a40 2023-10-18T09:12:11.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08841a0630 con 0x7f08841023e0 2023-10-18T09:12:11.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088c7b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08841a07c0 con 0x7f0884100840 2023-10-18T09:12:11.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f0889d51700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:11.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f0889d51700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:43564/0 (socket says 172.21.15.61:43564) 2023-10-18T09:12:11.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f0889d51700 1 -- 172.21.15.61:0/2376143069 learned_addr learned my addr 172.21.15.61:0/2376143069 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:12:11.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088a552700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 0x7f08841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:11.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.944+0000 7f088ad53700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 0x7f088419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:11.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 -- 172.21.15.61:0/2376143069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 msgr2=0x7f08841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:11.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 0x7f08841989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 -- 172.21.15.61:0/2376143069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 msgr2=0x7f088419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:11.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 0x7f088419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:11.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 -- 172.21.15.61:0/2376143069 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800095f0 con 0x7f0884101a40 2023-10-18T09:12:11.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f088ad53700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 0x7f088419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:12:11.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f088a552700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 0x7f08841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:12:11.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f0889d51700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f087400e9d0 tx=0x7f087400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:11.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874009790 con 0x7f0884101a40 2023-10-18T09:12:11.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087400cbd0 con 0x7f0884101a40 2023-10-18T09:12:11.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08841a0aa0 con 0x7f0884101a40 2023-10-18T09:12:11.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874018680 con 0x7f0884101a40 2023-10-18T09:12:11.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.945+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08841a0fa0 con 0x7f0884101a40 2023-10-18T09:12:11.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.946+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088404e4c0 con 0x7f0884101a40 2023-10-18T09:12:11.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.949+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 15) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f087400cd40 con 0x7f0884101a40 2023-10-18T09:12:11.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.950+0000 7f087b7fe700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 0x7f08700a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:11.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.950+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f0874014070 con 0x7f0884101a40 2023-10-18T09:12:11.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.950+0000 7f088a552700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 0x7f08700a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:11.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.951+0000 7f088a552700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 0x7f08700a38b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0880003730 tx=0x7f088000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:11.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:11.953+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f08740968a0 con 0x7f0884101a40 2023-10-18T09:12:12.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:12.173+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0884108980 con 0x7f08700a13f0 2023-10-18T09:12:12.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:12.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T09:12:12.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:12.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:12.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:12.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:12.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:12.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T09:12:12.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:12.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:12.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:12.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:12.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:12.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: osdmap e20: 3 total, 3 up, 3 in 2023-10-18T09:12:12.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:12:12.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:12.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:12.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:13.471 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123292]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:13.471 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:13.471 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123292]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:13.472 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123292]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:13.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:13.478+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0874096c70 con 0x7f0884101a40 2023-10-18T09:12:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:13 smithi061 ceph-mon[98573]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:13 smithi061 ceph-mon[98573]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T09:12:13.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:13 smithi061 ceph-mon[98573]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:13.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:13 smithi170 ceph-mon[135150]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:13.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:13.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:13 smithi170 ceph-mon[135150]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T09:12:13.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:13 smithi170 ceph-mon[135150]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:13.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:13 smithi110 ceph-mon[134818]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:13.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:12:13.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:13 smithi110 ceph-mon[134818]: osdmap e21: 3 total, 3 up, 3 in 2023-10-18T09:12:13.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:13 smithi110 ceph-mon[134818]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:14.253 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123343]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:14.254 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:14.254 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123343]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:14.254 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:12:13 smithi061 sudo[123343]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:14.314 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144620]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:14.315 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:14.315 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144620]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:14.315 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144620]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:14.712 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144623]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:14.712 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:14.712 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144623]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:14.713 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:12:14 smithi110 sudo[144623]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:14.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:14 smithi110 ceph-mon[134818]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-18T09:12:14.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:14 smithi110 ceph-mon[134818]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T09:12:14.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:14 smithi061 ceph-mon[98573]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-18T09:12:14.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:14 smithi061 ceph-mon[98573]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T09:12:14.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:14 smithi170 ceph-mon[135150]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-18T09:12:14.839 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:14 smithi170 ceph-mon[135150]: osdmap e22: 3 total, 3 up, 3 in 2023-10-18T09:12:14.841 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.841 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:14.841 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146343]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:14.842 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146343]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:14.842 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146343]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:14.842 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146343]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:15.142 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146346]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:15.142 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146346]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:15.143 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146346]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:15.143 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:12:14 smithi170 sudo[146346]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123433]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:15.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:15.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123433]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:15.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123433]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:15 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:16.213 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:16.213 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:15 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:16.213 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146349]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:16.213 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146349]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:16.214 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146349]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:16.214 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146349]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123437]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123437]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123437]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 sudo[123437]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:15 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:16.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146352]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:16.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146352]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:16.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146352]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:16.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 sudo[146352]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:16.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144626]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-18T09:12:16.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:16.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144626]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:16.948 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144626]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144629]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-18T09:12:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-18T09:12:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144629]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-18T09:12:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 sudo[144629]: pam_unix(sudo:session): session closed for user root 2023-10-18T09:12:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3462839921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]': finished 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/600993847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.217 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:16 smithi110 ceph-mon[134818]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3462839921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]': finished 2023-10-18T09:12:17.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T09:12:17.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:17.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/600993847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:17.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:16 smithi061 ceph-mon[98573]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3462839921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]: dispatch 2023-10-18T09:12:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c79af743-8b1a-477c-a7ce-f4845ca1a61a"}]': finished 2023-10-18T09:12:17.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: osdmap e23: 4 total, 3 up, 4 in 2023-10-18T09:12:17.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:17.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:17.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-18T09:12:17.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-18T09:12:17.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-18T09:12:17.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-18T09:12:17.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/600993847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:17.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:17.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:16 smithi170 ceph-mon[135150]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-18T09:12:20.102 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:19 smithi061 ceph-mon[98573]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:20.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:19 smithi170 ceph-mon[135150]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:19 smithi110 ceph-mon[134818]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:22.129 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:21 smithi061 ceph-mon[98573]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-18T09:12:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:21 smithi170 ceph-mon[135150]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-18T09:12:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:21 smithi110 ceph-mon[134818]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-18T09:12:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:23 smithi170 ceph-mon[135150]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-18T09:12:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T09:12:24.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:23 smithi110 ceph-mon[134818]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-18T09:12:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T09:12:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:23 smithi061 ceph-mon[98573]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-18T09:12:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-18T09:12:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:24 smithi170 ceph-mon[135150]: Deploying daemon osd.3 on smithi061 2023-10-18T09:12:25.165 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:24 smithi061 ceph-mon[98573]: Deploying daemon osd.3 on smithi061 2023-10-18T09:12:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:24 smithi110 ceph-mon[134818]: Deploying daemon osd.3 on smithi061 2023-10-18T09:12:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:25 smithi170 ceph-mon[135150]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-18T09:12:26.175 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:25 smithi061 ceph-mon[98573]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-18T09:12:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:25 smithi110 ceph-mon[134818]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-18T09:12:27.436 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:27 smithi061 ceph-mon[98573]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2023-10-18T09:12:27.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:27 smithi110 ceph-mon[134818]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2023-10-18T09:12:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:27 smithi170 ceph-mon[135150]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 121 KiB/s wr, 8 op/s 2023-10-18T09:12:29.932 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:29 smithi061 ceph-mon[98573]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:29 smithi170 ceph-mon[135150]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:29 smithi110 ceph-mon[134818]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:31.279 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.279 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.280 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:31.280 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:31 smithi061 ceph-mon[98573]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:31 smithi170 ceph-mon[135150]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:31 smithi110 ceph-mon[134818]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-18T09:12:33.547 INFO:teuthology.orchestra.run.smithi061.stdout:Created osd(s) 3 on host 'smithi061' 2023-10-18T09:12:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.544+0000 7f087b7fe700 1 -- 172.21.15.61:0/2376143069 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0884108980 con 0x7f08700a13f0 2023-10-18T09:12:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 msgr2=0x7f08700a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 0x7f08700a38b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0880003730 tx=0x7f088000b540 comp rx=0 tx=0).stop 2023-10-18T09:12:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 msgr2=0x7f0884198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f087400e9d0 tx=0x7f087400ece0 comp rx=0 tx=0).stop 2023-10-18T09:12:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 shutdown_connections 2023-10-18T09:12:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f08700a13f0 0x7f08700a38b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08841023e0 0x7f088419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0884100840 0x7f08841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 --2- 172.21.15.61:0/2376143069 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0884101a40 0x7f0884198ef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 >> 172.21.15.61:0/2376143069 conn(0x7f08840fbe00 msgr2=0x7f0884107260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 shutdown_connections 2023-10-18T09:12:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:12:33.547+0000 7f088c7b6700 1 -- 172.21.15.61:0/2376143069 wait complete. 2023-10-18T09:12:34.108 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:34.108 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.108 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.109 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:34.109 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:34.109 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.109 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.109 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.110 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:34.111 DEBUG:teuthology.orchestra.run.smithi061:osd.3> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.3.service 2023-10-18T09:12:34.116 INFO:tasks.cephadm:Deploying osd.4 on smithi110 with /dev/vg_nvme/lv_3... 2023-10-18T09:12:34.116 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T09:12:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:34.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:34.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:34.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:34.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:34.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:34.210 INFO:journalctl@ceph.osd.3.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:12:34.694 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:12:34.983 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:34 smithi061 ceph-mon[98573]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T09:12:35.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:34 smithi170 ceph-mon[135150]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T09:12:35.154 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:34 smithi110 ceph-mon[134818]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-18T09:12:36.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-mon[98573]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:36.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-mon[98573]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T09:12:36.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-mon[98573]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T09:12:36.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-mon[98573]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:12:36.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:36.005 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:12:35.859+0000 7ff4cf5a5700 -1 osd.3 0 waiting for initial osdmap 2023-10-18T09:12:36.005 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:12:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:12:35.865+0000 7ff4c9396700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:12:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T09:12:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T09:12:36.129 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2023-10-18T09:12:36.129 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2023-10-18T09:12:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124342 s, 843 MB/s 2023-10-18T09:12:36.130 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2023-10-18T09:12:36.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:35 smithi170 ceph-mon[135150]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:36.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:35 smithi170 ceph-mon[135150]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T09:12:36.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:35 smithi170 ceph-mon[135150]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T09:12:36.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:35 smithi170 ceph-mon[135150]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:12:36.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:36.166 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:35 smithi110 ceph-mon[134818]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:36.166 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:35 smithi110 ceph-mon[134818]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-18T09:12:36.166 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:35 smithi110 ceph-mon[134818]: osdmap e24: 4 total, 3 up, 4 in 2023-10-18T09:12:36.166 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:35 smithi110 ceph-mon[134818]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:12:36.167 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:36.454 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2023-10-18T09:12:37.050 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:12:37.119 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:12:37.120 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T09:12:37.120 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.120 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.120 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] boot 2023-10-18T09:12:37.120 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T09:12:37.121 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:12:37.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T09:12:37.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] boot 2023-10-18T09:12:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T09:12:37.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: from='osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:12:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: osdmap e25: 4 total, 3 up, 4 in 2023-10-18T09:12:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: osd.3 [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] boot 2023-10-18T09:12:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: osdmap e26: 4 total, 4 up, 4 in 2023-10-18T09:12:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-18T09:12:38.107 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:12:38.107 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:12:38.108 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:38.108 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T09:12:38.108 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.108 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.108 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:38.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:38.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:12:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:12:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T09:12:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:38.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:38.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.249 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 -- 172.21.15.110:0/945754966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d01023c0 msgr2=0x7fd5d0108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:38.250 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 --2- 172.21.15.110:0/945754966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d01023c0 0x7fd5d0108c70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009fc0 tx=0x7fd5c800b330 comp rx=0 tx=0).stop 2023-10-18T09:12:38.250 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 -- 172.21.15.110:0/945754966 shutdown_connections 2023-10-18T09:12:38.250 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 --2- 172.21.15.110:0/945754966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d01023c0 0x7fd5d0108c70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:38.250 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 --2- 172.21.15.110:0/945754966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0100820 0x7fd5d0100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:38.251 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 --2- 172.21.15.110:0/945754966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d0101a20 0x7fd5d0101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:38.251 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 -- 172.21.15.110:0/945754966 >> 172.21.15.110:0/945754966 conn(0x7fd5d00fbe00 msgr2=0x7fd5d00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:38.251 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 -- 172.21.15.110:0/945754966 shutdown_connections 2023-10-18T09:12:38.251 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.247+0000 7fd5d5405700 1 -- 172.21.15.110:0/945754966 wait complete. 2023-10-18T09:12:38.254 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 Processor -- start 2023-10-18T09:12:38.254 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 -- start start 2023-10-18T09:12:38.254 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:38.254 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0101a20 0x7fd5d0196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 0x7fd5d019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d019e370 con 0x7fd5d01023c0 2023-10-18T09:12:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d019e4b0 con 0x7fd5d0100820 2023-10-18T09:12:38.255 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5d5405700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d019e640 con 0x7fd5d0101a20 2023-10-18T09:12:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5ceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5ceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60216/0 (socket says 172.21.15.110:60216) 2023-10-18T09:12:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5ceffd700 1 -- 172.21.15.110:0/2087123883 learned_addr learned my addr 172.21.15.110:0/2087123883 (peer_addr_for_me v2:172.21.15.110:0/0) 2023-10-18T09:12:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5ce7fc700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0101a20 0x7fd5d0196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:38.256 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.248+0000 7fd5cf7fe700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 0x7fd5d019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:38.257 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 -- 172.21.15.110:0/2087123883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0101a20 msgr2=0x7fd5d0196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:38.257 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0101a20 0x7fd5d0196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:38.257 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 -- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 msgr2=0x7fd5d019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:38.257 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 0x7fd5d019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 -- 172.21.15.110:0/2087123883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c8009c40 con 0x7fd5d0100820 2023-10-18T09:12:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5cf7fe700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 0x7fd5d019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:12:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5ceffd700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9f0 tx=0x7fd5c400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004220 con 0x7fd5d0100820 2023-10-18T09:12:38.258 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5c4004380 con 0x7fd5d0100820 2023-10-18T09:12:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d019e920 con 0x7fd5d0100820 2023-10-18T09:12:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4003c70 con 0x7fd5d0100820 2023-10-18T09:12:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.249+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d019ee20 con 0x7fd5d0100820 2023-10-18T09:12:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.250+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d0107cb0 con 0x7fd5d0100820 2023-10-18T09:12:38.259 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.251+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd5c400c7c0 con 0x7fd5d0100820 2023-10-18T09:12:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.251+0000 7fd5b7fff700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 0x7fd5b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.252+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fd5c4014070 con 0x7fd5d0100820 2023-10-18T09:12:38.260 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.252+0000 7fd5ce7fc700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 0x7fd5b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:38.261 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:12:38.261 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:12:38.261 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-18T09:12:38.261 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: osdmap e27: 4 total, 4 up, 4 in 2023-10-18T09:12:38.262 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.262 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.262 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.262 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:38.262 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.263 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:38.263 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:38.263 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:38.263 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.252+0000 7fd5ce7fc700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 0x7fd5b80a3940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005950 tx=0x7fd5c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:38.264 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.255+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd5c40967e0 con 0x7fd5d0100820 2023-10-18T09:12:38.455 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:38.452+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5d002cf80 con 0x7fd5b80a1480 2023-10-18T09:12:39.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi061 2023-10-18T09:12:39.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 8635M 2023-10-18T09:12:39.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T09:12:39.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:39.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:39.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi061 2023-10-18T09:12:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 8635M 2023-10-18T09:12:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T09:12:39.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:39.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:39.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:39.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi061 2023-10-18T09:12:39.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 8635M 2023-10-18T09:12:39.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: osdmap e28: 4 total, 4 up, 4 in 2023-10-18T09:12:39.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:39.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:39.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:39 smithi170 ceph-mon[135150]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:39 smithi170 ceph-mon[135150]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:40.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:39 smithi170 ceph-mon[135150]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T09:12:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:39 smithi110 ceph-mon[134818]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:39 smithi110 ceph-mon[134818]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:39 smithi110 ceph-mon[134818]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T09:12:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:39 smithi061 ceph-mon[98573]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:12:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:39 smithi061 ceph-mon[98573]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:40.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:39 smithi061 ceph-mon[98573]: osdmap e29: 4 total, 4 up, 4 in 2023-10-18T09:12:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T09:12:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:12:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/3344283276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]': finished 2023-10-18T09:12:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T09:12:41.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:41.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/1931342558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T09:12:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:12:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/3344283276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]': finished 2023-10-18T09:12:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T09:12:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:41.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/1931342558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:41.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: osdmap e30: 4 total, 4 up, 4 in 2023-10-18T09:12:41.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:12:41.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/3344283276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]: dispatch 2023-10-18T09:12:41.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e542f6db-5212-4e76-b5d8-05029c585fd7"}]': finished 2023-10-18T09:12:41.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: osdmap e31: 5 total, 4 up, 5 in 2023-10-18T09:12:41.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:41.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/1931342558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:12:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:43 smithi170 ceph-mon[135150]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-18T09:12:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:43 smithi110 ceph-mon[134818]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-18T09:12:44.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:43 smithi061 ceph-mon[98573]: pgmap v122: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2023-10-18T09:12:46.122 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:45 smithi110 ceph-mon[134818]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:12:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:45 smithi170 ceph-mon[135150]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:12:46.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:45 smithi061 ceph-mon[98573]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:12:47.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:47 smithi110 ceph-mon[134818]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:47.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T09:12:47.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:47 smithi061 ceph-mon[98573]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T09:12:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:47 smithi170 ceph-mon[135150]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-18T09:12:47.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:48.710 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:48 smithi110 ceph-mon[134818]: Deploying daemon osd.4 on smithi110 2023-10-18T09:12:48.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:48 smithi061 ceph-mon[98573]: Deploying daemon osd.4 on smithi110 2023-10-18T09:12:48.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:48 smithi170 ceph-mon[135150]: Deploying daemon osd.4 on smithi110 2023-10-18T09:12:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:49 smithi110 ceph-mon[134818]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:49 smithi061 ceph-mon[98573]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:49.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:49 smithi170 ceph-mon[135150]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:52.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:51 smithi170 ceph-mon[135150]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:52.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:51 smithi110 ceph-mon[134818]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:51 smithi061 ceph-mon[98573]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:52.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:54.084 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:54.085 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.085 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.085 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:54.085 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:54.085 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.116 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.114+0000 7fd5b7fff700 1 -- 172.21.15.110:0/2087123883 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd5d002cf80 con 0x7fd5b80a1480 2023-10-18T09:12:54.117 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 4 on host 'smithi110' 2023-10-18T09:12:54.119 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 msgr2=0x7fd5b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:54.119 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 0x7fd5b80a3940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0005950 tx=0x7fd5c000b410 comp rx=0 tx=0).stop 2023-10-18T09:12:54.119 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 msgr2=0x7fd5d0196830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:54.119 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e9f0 tx=0x7fd5c400ed00 comp rx=0 tx=0).stop 2023-10-18T09:12:54.120 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 shutdown_connections 2023-10-18T09:12:54.120 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5b80a1480 0x7fd5b80a3940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:54.120 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd5d01023c0 0x7fd5d019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:54.120 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5d0100820 0x7fd5d0196830 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:54.120 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 --2- 172.21.15.110:0/2087123883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5d0101a20 0x7fd5d0196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:54.121 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.116+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 >> 172.21.15.110:0/2087123883 conn(0x7fd5d00fbe00 msgr2=0x7fd5d0104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:54.121 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.117+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 shutdown_connections 2023-10-18T09:12:54.121 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:12:54.117+0000 7fd5d5405700 1 -- 172.21.15.110:0/2087123883 wait complete. 2023-10-18T09:12:54.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:54.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:54.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:54.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:54.540 DEBUG:teuthology.orchestra.run.smithi110:osd.4> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.4.service 2023-10-18T09:12:54.542 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2023-10-18T09:12:54.543 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-18T09:12:54.571 INFO:journalctl@ceph.osd.4.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:12:55.124 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:12:55.361 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.361 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.361 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:55.362 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:55 smithi110 ceph-mon[134818]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:55.380 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.380 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.381 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:55.381 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:55 smithi170 ceph-mon[135150]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:12:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:55 smithi061 ceph-mon[98573]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:55.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:12:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:12:55.416+0000 7f132fd0b640 -1 osd.4 0 log_to_monitors true 2023-10-18T09:12:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:56 smithi170 ceph-mon[135150]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:56 smithi170 ceph-mon[135150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:56 smithi110 ceph-mon[134818]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:56 smithi110 ceph-mon[134818]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:56 smithi061 ceph-mon[98573]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:56 smithi061 ceph-mon[98573]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-18T09:12:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-18T09:12:56.589 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-18T09:12:56.590 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-10-18T09:12:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-10-18T09:12:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00838097 s, 1.3 GB/s 2023-10-18T09:12:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-10-18T09:12:56.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2023-10-18T09:12:57.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T09:12:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T09:12:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:57 smithi170 ceph-mon[135150]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:57.462 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:12:57.126+0000 7f1325382700 -1 osd.4 0 waiting for initial osdmap 2023-10-18T09:12:57.462 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:12:57.133+0000 7f131d16f700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:12:57.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T09:12:57.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T09:12:57.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:57.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:57 smithi110 ceph-mon[134818]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-18T09:12:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: from='osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: osdmap e32: 5 total, 4 up, 5 in 2023-10-18T09:12:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:12:57.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:57 smithi061 ceph-mon[98573]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:57.547 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:12:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:12:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T09:12:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi110 2023-10-18T09:12:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi110 to 8584M 2023-10-18T09:12:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:58.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:58.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:12:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T09:12:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi110 2023-10-18T09:12:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi110 to 8584M 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:58.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:12:58.551 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: osdmap e33: 5 total, 4 up, 5 in 2023-10-18T09:12:58.552 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.552 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:58.552 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi110 2023-10-18T09:12:58.552 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.552 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi110 to 8584M 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:58.553 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:12:58.554 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:12:58.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 -- 172.21.15.170:0/709996268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 msgr2=0x7fc678107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 --2- 172.21.15.170:0/709996268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc678107b60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b0a0 tx=0x7fc67000b3b0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 -- 172.21.15.170:0/709996268 shutdown_connections 2023-10-18T09:12:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 --2- 172.21.15.170:0/709996268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc678107b60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 --2- 172.21.15.170:0/709996268 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc6780ff710 0x7fc6780ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 --2- 172.21.15.170:0/709996268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc678100910 0x7fc678100d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 -- 172.21.15.170:0/709996268 >> 172.21.15.170:0/709996268 conn(0x7fc6780facb0 msgr2=0x7fc6780fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:12:58.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 -- 172.21.15.170:0/709996268 shutdown_connections 2023-10-18T09:12:58.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.847+0000 7fc67fea5700 1 -- 172.21.15.170:0/709996268 wait complete. 2023-10-18T09:12:58.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 Processor -- start 2023-10-18T09:12:58.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 -- start start 2023-10-18T09:12:58.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 0x7fc678196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:58.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc678100910 0x7fc678196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:58.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:58.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67819e460 con 0x7fc678100910 2023-10-18T09:12:58.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67819e5a0 con 0x7fc6780ff710 2023-10-18T09:12:58.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67fea5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67819e730 con 0x7fc6781012b0 2023-10-18T09:12:58.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67e442700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:58.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67e442700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54314/0 (socket says 172.21.15.170:54314) 2023-10-18T09:12:58.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.848+0000 7fc67e442700 1 -- 172.21.15.170:0/179971679 learned_addr learned my addr 172.21.15.170:0/179971679 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-18T09:12:58.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67dc41700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 0x7fc678196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:58.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 -- 172.21.15.170:0/179971679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 msgr2=0x7fc678196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:58.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 0x7fc678196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 -- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc678100910 msgr2=0x7fc678196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:12:58.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc678100910 0x7fc678196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:12:58.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 -- 172.21.15.170:0/179971679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc670009d00 con 0x7fc6781012b0 2023-10-18T09:12:58.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67dc41700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 0x7fc678196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:12:58.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67e442700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc670004d30 tx=0x7fc670004e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:58.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67000e040 con 0x7fc6781012b0 2023-10-18T09:12:58.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc670009440 con 0x7fc6781012b0 2023-10-18T09:12:58.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67819e9b0 con 0x7fc6781012b0 2023-10-18T09:12:58.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67001b9b0 con 0x7fc6781012b0 2023-10-18T09:12:58.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.849+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67819eef0 con 0x7fc6781012b0 2023-10-18T09:12:58.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.850+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67804e4c0 con 0x7fc6781012b0 2023-10-18T09:12:58.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.852+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc67001dc50 con 0x7fc6781012b0 2023-10-18T09:12:58.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.852+0000 7fc66effd700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 0x7fc6640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:12:58.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.853+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fc6700ce240 con 0x7fc6781012b0 2023-10-18T09:12:58.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.853+0000 7fc67dc41700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 0x7fc6640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:12:58.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.853+0000 7fc67dc41700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 0x7fc6640a39c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc678100770 tx=0x7fc668008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:12:58.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:58.856+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc670098900 con 0x7fc6781012b0 2023-10-18T09:12:59.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:12:59.053+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc67810b030 con 0x7fc6640a1500 2023-10-18T09:12:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:12:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:12:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] boot 2023-10-18T09:12:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T09:12:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:59.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:59.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:12:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:12:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] boot 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:59.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:59.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:12:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:12:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:12:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:12:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: osd.4 [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] boot 2023-10-18T09:12:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: osdmap e34: 5 total, 5 up, 5 in 2023-10-18T09:12:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-18T09:12:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-18T09:12:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:12:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:12:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:12:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:00.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:00 smithi170 ceph-mon[135150]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:00.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:00 smithi170 ceph-mon[135150]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T09:13:00.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:00 smithi110 ceph-mon[134818]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:00.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:00 smithi110 ceph-mon[134818]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T09:13:00.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:00 smithi061 ceph-mon[98573]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:00.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:00 smithi061 ceph-mon[98573]: osdmap e35: 5 total, 5 up, 5 in 2023-10-18T09:13:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:01 smithi110 ceph-mon[134818]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:01 smithi061 ceph-mon[98573]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:01 smithi170 ceph-mon[135150]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:02.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/3834478505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]': finished 2023-10-18T09:13:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T09:13:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:02.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:02 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/703272721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:02.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/3834478505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]': finished 2023-10-18T09:13:02.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T09:13:02.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:02.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:02 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/703272721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:02.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/3834478505' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]: dispatch 2023-10-18T09:13:02.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2069c32b-e740-4ea4-b647-35be7d41f3df"}]': finished 2023-10-18T09:13:02.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: osdmap e36: 6 total, 5 up, 6 in 2023-10-18T09:13:02.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:02.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:02 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/703272721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:03.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:03 smithi110 ceph-mon[134818]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:03 smithi061 ceph-mon[98573]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:03 smithi170 ceph-mon[135150]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:05 smithi170 ceph-mon[135150]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:05 smithi110 ceph-mon[134818]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:05 smithi061 ceph-mon[98573]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:07.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:07 smithi110 ceph-mon[134818]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:07.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:07 smithi061 ceph-mon[98573]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:07.820 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:07 smithi170 ceph-mon[135150]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:08 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T09:13:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:08 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:08.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:08 smithi110 ceph-mon[134818]: Deploying daemon osd.5 on smithi170 2023-10-18T09:13:08.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:08 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T09:13:08.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:08 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:08.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:08 smithi061 ceph-mon[98573]: Deploying daemon osd.5 on smithi170 2023-10-18T09:13:08.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:08 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-18T09:13:08.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:08 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:08.781 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:08 smithi170 ceph-mon[135150]: Deploying daemon osd.5 on smithi170 2023-10-18T09:13:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:09 smithi110 ceph-mon[134818]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:09 smithi061 ceph-mon[98573]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:09.809 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:09 smithi170 ceph-mon[135150]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:12.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:11 smithi170 ceph-mon[135150]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:11 smithi110 ceph-mon[134818]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:11 smithi061 ceph-mon[98573]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:13.551 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.551 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.551 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:13.551 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:13 smithi170 ceph-mon[135150]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:13.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:13 smithi110 ceph-mon[134818]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:13.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:13 smithi061 ceph-mon[98573]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:15.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.018+0000 7fc66effd700 1 -- 172.21.15.170:0/179971679 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc67810b030 con 0x7fc6640a1500 2023-10-18T09:13:15.021 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2023-10-18T09:13:15.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 msgr2=0x7fc6640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:15.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 0x7fc6640a39c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc678100770 tx=0x7fc668008040 comp rx=0 tx=0).stop 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 msgr2=0x7fc67819de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc670004d30 tx=0x7fc670004e10 comp rx=0 tx=0).stop 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 shutdown_connections 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc6640a1500 0x7fc6640a39c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6781012b0 0x7fc67819de90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:15.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc6780ff710 0x7fc678196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:15.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 --2- 172.21.15.170:0/179971679 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc678100910 0x7fc678196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:15.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 >> 172.21.15.170:0/179971679 conn(0x7fc6780facb0 msgr2=0x7fc6780fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:13:15.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 shutdown_connections 2023-10-18T09:13:15.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:13:15.020+0000 7fc67fea5700 1 -- 172.21.15.170:0/179971679 wait complete. 2023-10-18T09:13:15.426 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.5.service 2023-10-18T09:13:15.428 INFO:tasks.cephadm:Deploying osd.6 on smithi061 with /dev/vg_nvme/lv_2... 2023-10-18T09:13:15.429 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T09:13:15.457 INFO:journalctl@ceph.osd.5.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:13:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:15 smithi170 ceph-mon[135150]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.186 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:13:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:15 smithi110 ceph-mon[134818]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:15 smithi061 ceph-mon[98573]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:16.517 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:13:16 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:13:16.396+0000 7faa5290b640 -1 osd.5 0 log_to_monitors true 2023-10-18T09:13:17.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:16 smithi170 ceph-mon[135150]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:17.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:16 smithi170 ceph-mon[135150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:16 smithi110 ceph-mon[134818]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:16 smithi110 ceph-mon[134818]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:16 smithi061 ceph-mon[98573]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:16 smithi061 ceph-mon[98573]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-18T09:13:18.142 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:13:17.863+0000 7faa47f82700 -1 osd.5 0 waiting for initial osdmap 2023-10-18T09:13:18.142 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:13:17.872+0000 7faa42574700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:13:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T09:13:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T09:13:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:17 smithi170 ceph-mon[135150]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T09:13:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T09:13:18.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:18.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:17 smithi110 ceph-mon[134818]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-18T09:13:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: osdmap e37: 6 total, 5 up, 6 in 2023-10-18T09:13:18.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:18.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: from='osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:17 smithi061 ceph-mon[98573]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:13:18.255 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T09:13:18.255 INFO:teuthology.orchestra.run.smithi061.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T09:13:18.255 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 10+0 records in 2023-10-18T09:13:18.255 INFO:teuthology.orchestra.run.smithi061.stderr:10+0 records out 2023-10-18T09:13:18.255 INFO:teuthology.orchestra.run.smithi061.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0505136 s, 208 MB/s 2023-10-18T09:13:18.256 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping successful for: 2023-10-18T09:13:18.784 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi061:vg_nvme/lv_2 2023-10-18T09:13:19.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:13:19.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T09:13:19.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:19.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:18 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:13:19.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T09:13:19.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:19.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.147 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:18 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:13:19.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: osdmap e38: 6 total, 5 up, 6 in 2023-10-18T09:13:19.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:19.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:19.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:19.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:19.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:19.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:18 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:13:20.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:13:20.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:13:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi170 2023-10-18T09:13:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi170 to 8584M 2023-10-18T09:13:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] boot 2023-10-18T09:13:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T09:13:20.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:20.168 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:13:20.168 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:13:20.168 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi170 2023-10-18T09:13:20.168 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi170 to 8584M 2023-10-18T09:13:20.169 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:20.169 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] boot 2023-10-18T09:13:20.169 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T09:13:20.169 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:13:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:13:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi170 2023-10-18T09:13:20.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi170 to 8584M 2023-10-18T09:13:20.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-18T09:13:20.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: osd.5 [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] boot 2023-10-18T09:13:20.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: osdmap e39: 6 total, 6 up, 6 in 2023-10-18T09:13:20.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-18T09:13:21.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:20 smithi170 ceph-mon[135150]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T09:13:21.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 -- 172.21.15.61:0/234683518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 msgr2=0x7f524c101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:21.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 --2- 172.21.15.61:0/234683518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c101ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52380099c0 tx=0x7f5238009cd0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 -- 172.21.15.61:0/234683518 shutdown_connections 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 --2- 172.21.15.61:0/234683518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c1023e0 0x7f524c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 --2- 172.21.15.61:0/234683518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c100840 0x7f524c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 --2- 172.21.15.61:0/234683518 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c101ea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 -- 172.21.15.61:0/234683518 >> 172.21.15.61:0/234683518 conn(0x7f524c0fbe00 msgr2=0x7f524c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 -- 172.21.15.61:0/234683518 shutdown_connections 2023-10-18T09:13:21.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.192+0000 7f5252fa9700 1 -- 172.21.15.61:0/234683518 wait complete. 2023-10-18T09:13:21.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.193+0000 7f5252fa9700 1 Processor -- start 2023-10-18T09:13:21.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 -- start start 2023-10-18T09:13:21.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c100840 0x7f524c074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:21.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:21.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c1023e0 0x7f524c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:21.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0738c0 con 0x7f524c101a40 2023-10-18T09:13:21.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c073a30 con 0x7f524c100840 2023-10-18T09:13:21.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5252fa9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c1a07a0 con 0x7f524c1023e0 2023-10-18T09:13:21.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f524bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:21.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f524bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:39740/0 (socket says 172.21.15.61:39740) 2023-10-18T09:13:21.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f524bfff700 1 -- 172.21.15.61:0/1403980511 learned_addr learned my addr 172.21.15.61:0/1403980511 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:13:21.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5250d45700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c100840 0x7f524c074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:21.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.194+0000 7f5251546700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c1023e0 0x7f524c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:21.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 -- 172.21.15.61:0/1403980511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c1023e0 msgr2=0x7f524c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:21.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c1023e0 0x7f524c073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 -- 172.21.15.61:0/1403980511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c100840 msgr2=0x7f524c074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:21.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c100840 0x7f524c074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:21.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 -- 172.21.15.61:0/1403980511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380095f0 con 0x7f524c101a40 2023-10-18T09:13:21.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f524bfff700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5238003730 tx=0x7f523800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:13:21.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523801d070 con 0x7f524c101a40 2023-10-18T09:13:21.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c1a09d0 con 0x7f524c101a40 2023-10-18T09:13:21.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52380059a0 con 0x7f524c101a40 2023-10-18T09:13:21.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238004a00 con 0x7f524c101a40 2023-10-18T09:13:21.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.195+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c1a0e70 con 0x7f524c101a40 2023-10-18T09:13:21.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.196+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c107d70 con 0x7f524c101a40 2023-10-18T09:13:21.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.198+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5238005210 con 0x7f524c101a40 2023-10-18T09:13:21.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.200+0000 7f5249ffb700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 0x7f523c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:21.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.200+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f52380cca70 con 0x7f524c101a40 2023-10-18T09:13:21.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.200+0000 7f5250d45700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 0x7f523c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:21.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.200+0000 7f5250d45700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 0x7f523c0a3940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c1018a0 tx=0x7f5240009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:13:21.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.204+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5238096f60 con 0x7f524c101a40 2023-10-18T09:13:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:20 smithi110 ceph-mon[134818]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T09:13:21.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:20 smithi061 ceph-mon[98573]: osdmap e40: 6 total, 6 up, 6 in 2023-10-18T09:13:21.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:21.425+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f524c02cf80 con 0x7f523c0a1480 2023-10-18T09:13:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T09:13:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:21 smithi170 ceph-mon[135150]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T09:13:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T09:13:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:21 smithi110 ceph-mon[134818]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T09:13:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: pgmap v150: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: osdmap e41: 6 total, 6 up, 6 in 2023-10-18T09:13:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:22.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:22.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:22.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:21 smithi061 ceph-mon[98573]: osdmap e42: 6 total, 6 up, 6 in 2023-10-18T09:13:23.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:22 smithi170 ceph-mon[135150]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:23.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:22 smithi170 ceph-mon[135150]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T09:13:23.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:22 smithi061 ceph-mon[98573]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:23.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:22 smithi061 ceph-mon[98573]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T09:13:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:22 smithi110 ceph-mon[134818]: from='client.14445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:22 smithi110 ceph-mon[134818]: osdmap e43: 6 total, 6 up, 6 in 2023-10-18T09:13:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:23 smithi170 ceph-mon[135150]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:23 smithi110 ceph-mon[134818]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:23 smithi061 ceph-mon[98573]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]: dispatch 2023-10-18T09:13:25.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]': finished 2023-10-18T09:13:25.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:24 smithi170 ceph-mon[135150]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T09:13:25.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]: dispatch 2023-10-18T09:13:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]': finished 2023-10-18T09:13:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:24 smithi110 ceph-mon[134818]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T09:13:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]: dispatch 2023-10-18T09:13:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1853917420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a152124f-a0de-47d0-97b3-b65467c83915"}]': finished 2023-10-18T09:13:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:24 smithi061 ceph-mon[98573]: osdmap e44: 7 total, 6 up, 7 in 2023-10-18T09:13:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:25 smithi170 ceph-mon[135150]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:13:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2704217491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:25 smithi110 ceph-mon[134818]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:13:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2704217491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:25 smithi061 ceph-mon[98573]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:13:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2704217491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:27.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:27 smithi110 ceph-mon[134818]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-10-18T09:13:27.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:27 smithi061 ceph-mon[98573]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-10-18T09:13:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:27 smithi170 ceph-mon[135150]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2023-10-18T09:13:30.107 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:29 smithi061 ceph-mon[98573]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-18T09:13:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:29 smithi170 ceph-mon[135150]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-18T09:13:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:29 smithi110 ceph-mon[134818]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-10-18T09:13:32.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:31 smithi061 ceph-mon[98573]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:13:32.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:31 smithi170 ceph-mon[135150]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:13:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:31 smithi110 ceph-mon[134818]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:13:33.116 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T09:13:33.116 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:32 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:33.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T09:13:33.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:32 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:33.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-18T09:13:33.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:32 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:34.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:33 smithi061 ceph-mon[98573]: Deploying daemon osd.6 on smithi061 2023-10-18T09:13:34.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:33 smithi061 ceph-mon[98573]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:13:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:33 smithi170 ceph-mon[135150]: Deploying daemon osd.6 on smithi061 2023-10-18T09:13:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:33 smithi170 ceph-mon[135150]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:13:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:33 smithi110 ceph-mon[134818]: Deploying daemon osd.6 on smithi061 2023-10-18T09:13:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:33 smithi110 ceph-mon[134818]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:13:36.053 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:35 smithi061 ceph-mon[98573]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:36.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:35 smithi170 ceph-mon[135150]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:35 smithi110 ceph-mon[134818]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:37 smithi110 ceph-mon[134818]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:37 smithi061 ceph-mon[98573]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:37 smithi170 ceph-mon[135150]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:40.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:39 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:40.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:39 smithi061 ceph-mon[98573]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:39 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:39 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:39 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:40.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:39 smithi170 ceph-mon[135150]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:39 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:39 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:39 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:39 smithi110 ceph-mon[134818]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:42.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:42.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.199 INFO:teuthology.orchestra.run.smithi061.stdout:Created osd(s) 6 on host 'smithi061' 2023-10-18T09:13:42.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.195+0000 7f5249ffb700 1 -- 172.21.15.61:0/1403980511 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f524c02cf80 con 0x7f523c0a1480 2023-10-18T09:13:42.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 msgr2=0x7f523c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:42.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 0x7f523c0a3940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c1018a0 tx=0x7f5240009260 comp rx=0 tx=0).stop 2023-10-18T09:13:42.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 msgr2=0x7f524c072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:42.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5238003730 tx=0x7f523800f670 comp rx=0 tx=0).stop 2023-10-18T09:13:42.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 shutdown_connections 2023-10-18T09:13:42.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f523c0a1480 0x7f523c0a3940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:42.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f524c1023e0 0x7f524c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:42.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f524c100840 0x7f524c074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:42.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 --2- 172.21.15.61:0/1403980511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f524c101a40 0x7f524c072be0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:42.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 >> 172.21.15.61:0/1403980511 conn(0x7f524c0fbe00 msgr2=0x7f524c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:13:42.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 shutdown_connections 2023-10-18T09:13:42.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:13:42.198+0000 7f5252fa9700 1 -- 172.21.15.61:0/1403980511 wait complete. 2023-10-18T09:13:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:42.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.223 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:42.223 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.224 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.224 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:42.224 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:42.224 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:42.666 DEBUG:teuthology.orchestra.run.smithi061:osd.6> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.6.service 2023-10-18T09:13:42.671 INFO:tasks.cephadm:Deploying osd.7 on smithi110 with /dev/vg_nvme/lv_2... 2023-10-18T09:13:42.672 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T09:13:42.756 INFO:journalctl@ceph.osd.6.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:13:43.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:13:42 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:13:42.753+0000 7f06597d6640 -1 osd.6 0 log_to_monitors true 2023-10-18T09:13:43.242 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:13:43.452 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.453 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.453 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:43 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:43.453 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:43 smithi110 ceph-mon[134818]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T09:13:43.453 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:43 smithi110 ceph-mon[134818]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:43.498 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.498 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.499 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:43 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:43.499 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:43 smithi061 ceph-mon[98573]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T09:13:43.499 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:43 smithi061 ceph-mon[98573]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:43.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:43 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:13:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:43 smithi170 ceph-mon[135150]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-18T09:13:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:43 smithi170 ceph-mon[135150]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:44.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:44 smithi110 ceph-mon[134818]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T09:13:44.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:44 smithi110 ceph-mon[134818]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T09:13:44.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:44 smithi110 ceph-mon[134818]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:13:44.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:44 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:44.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-mon[98573]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T09:13:44.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-mon[98573]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T09:13:44.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-mon[98573]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:13:44.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:44.504 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:13:44.217+0000 7f064ee4d700 -1 osd.6 0 waiting for initial osdmap 2023-10-18T09:13:44.505 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:13:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:13:44.229+0000 7f064843d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:13:44.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:44 smithi170 ceph-mon[135150]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-18T09:13:44.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:44 smithi170 ceph-mon[135150]: osdmap e45: 7 total, 6 up, 7 in 2023-10-18T09:13:44.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:44 smithi170 ceph-mon[135150]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:13:44.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:44 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:44.705 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T09:13:44.705 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T09:13:44.705 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2023-10-18T09:13:44.706 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2023-10-18T09:13:44.706 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123205 s, 851 MB/s 2023-10-18T09:13:44.706 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2023-10-18T09:13:45.084 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2023-10-18T09:13:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:45 smithi110 ceph-mon[134818]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:13:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:45 smithi110 ceph-mon[134818]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T09:13:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:45 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:45 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:45 smithi110 ceph-mon[134818]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:45.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:45 smithi061 ceph-mon[98573]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:13:45.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:45 smithi061 ceph-mon[98573]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T09:13:45.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:45 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:45 smithi061 ceph-mon[98573]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:45 smithi170 ceph-mon[135150]: from='osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:13:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:45 smithi170 ceph-mon[135150]: osdmap e46: 7 total, 6 up, 7 in 2023-10-18T09:13:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:45 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:45 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:45 smithi170 ceph-mon[135150]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-18T09:13:45.646 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:13:46.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:13:46.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:13:46.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] boot 2023-10-18T09:13:46.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T09:13:46.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:46.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:46.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:13:46.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:13:46.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] boot 2023-10-18T09:13:46.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T09:13:46.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:46.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:46.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:13:46.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:13:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: osd.6 [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] boot 2023-10-18T09:13:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: osdmap e47: 7 total, 7 up, 7 in 2023-10-18T09:13:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-18T09:13:46.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:13:46.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:46.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:46.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:13:46.837 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 -- 172.21.15.110:0/1245070267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4500ff730 msgr2=0x7fe4500ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 --2- 172.21.15.110:0/1245070267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4500ff730 0x7fe4500ffb50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4400099c0 tx=0x7fe440009cd0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 -- 172.21.15.110:0/1245070267 shutdown_connections 2023-10-18T09:13:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 --2- 172.21.15.110:0/1245070267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4501012d0 0x7fe450107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 --2- 172.21.15.110:0/1245070267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4500ff730 0x7fe4500ffb50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.838 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 --2- 172.21.15.110:0/1245070267 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe450100930 0x7fe450100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 -- 172.21.15.110:0/1245070267 >> 172.21.15.110:0/1245070267 conn(0x7fe4500facf0 msgr2=0x7fe4500fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:13:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 -- 172.21.15.110:0/1245070267 shutdown_connections 2023-10-18T09:13:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.833+0000 7fe4575b6700 1 -- 172.21.15.110:0/1245070267 wait complete. 2023-10-18T09:13:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 Processor -- start 2023-10-18T09:13:46.839 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 -- start start 2023-10-18T09:13:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4500ff730 0x7fe4501955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe450100930 0x7fe450195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45019d120 con 0x7fe450100930 2023-10-18T09:13:46.840 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45019d260 con 0x7fe4501012d0 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe4575b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45019d3f0 con 0x7fe4500ff730 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60328/0 (socket says 172.21.15.110:60328) 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 -- 172.21.15.110:0/3310926850 learned_addr learned my addr 172.21.15.110:0/3310926850 (peer_addr_for_me v2:172.21.15.110:0/0) 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455352700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4500ff730 0x7fe4501955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:46.841 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 -- 172.21.15.110:0/3310926850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4500ff730 msgr2=0x7fe4501955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4500ff730 0x7fe4501955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 -- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe450100930 msgr2=0x7fe450195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:13:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe450100930 0x7fe450195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:13:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 -- 172.21.15.110:0/3310926850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400095f0 con 0x7fe4501012d0 2023-10-18T09:13:46.842 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.834+0000 7fe455b53700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe448007d70 tx=0x7fe44800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:13:46.843 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44800ad40 con 0x7fe4501012d0 2023-10-18T09:13:46.843 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45019d6d0 con 0x7fe4501012d0 2023-10-18T09:13:46.843 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4480128e0 con 0x7fe4501012d0 2023-10-18T09:13:46.843 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480116b0 con 0x7fe4501012d0 2023-10-18T09:13:46.843 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45019db80 con 0x7fe4501012d0 2023-10-18T09:13:46.844 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.835+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450106b70 con 0x7fe4501012d0 2023-10-18T09:13:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.837+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe448012a50 con 0x7fe4501012d0 2023-10-18T09:13:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.837+0000 7fe4467fc700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 0x7fe43c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:13:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.837+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fe4480cc640 con 0x7fe4501012d0 2023-10-18T09:13:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.838+0000 7fe455352700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 0x7fe43c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:13:46.845 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.838+0000 7fe455352700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 0x7fe43c0a3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe440006010 tx=0x7fe440000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:13:46.846 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:46.841+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe4480969b0 con 0x7fe4501012d0 2023-10-18T09:13:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:13:47.036+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe45002cf80 con 0x7fe43c0a1480 2023-10-18T09:13:47.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi061 2023-10-18T09:13:47.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 5757M 2023-10-18T09:13:47.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T09:13:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:47.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:47.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi061 2023-10-18T09:13:47.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 5757M 2023-10-18T09:13:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T09:13:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:47.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi061 2023-10-18T09:13:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 5757M 2023-10-18T09:13:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:13:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: osdmap e48: 7 total, 7 up, 7 in 2023-10-18T09:13:47.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:47.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:13:47.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:13:47.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:48.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:48 smithi110 ceph-mon[134818]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:48.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:48 smithi061 ceph-mon[98573]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:48.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:48 smithi170 ceph-mon[135150]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:13:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:49.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/1148453553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]': finished 2023-10-18T09:13:49.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T09:13:49.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:13:49.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:49.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/1148453553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]': finished 2023-10-18T09:13:49.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T09:13:49.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:13:49.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:49.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/1148453553' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]: dispatch 2023-10-18T09:13:49.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e523e57-7424-443a-99ba-a6becb55b91a"}]': finished 2023-10-18T09:13:49.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: osdmap e49: 8 total, 7 up, 8 in 2023-10-18T09:13:49.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:13:50.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:50 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/2345499494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:50.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:50 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/2345499494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:50.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:50 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/2345499494' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:13:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:51 smithi061 ceph-mon[98573]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:51.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:51 smithi170 ceph-mon[135150]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:51.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:51 smithi110 ceph-mon[134818]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:54.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:53 smithi170 ceph-mon[135150]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:54.209 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:53 smithi110 ceph-mon[134818]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:53 smithi061 ceph-mon[98573]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:55 smithi110 ceph-mon[134818]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T09:13:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:55 smithi170 ceph-mon[135150]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T09:13:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:55 smithi061 ceph-mon[98573]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-18T09:13:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:13:56.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:56 smithi110 ceph-mon[134818]: Deploying daemon osd.7 on smithi110 2023-10-18T09:13:57.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:56 smithi170 ceph-mon[135150]: Deploying daemon osd.7 on smithi110 2023-10-18T09:13:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:56 smithi061 ceph-mon[98573]: Deploying daemon osd.7 on smithi110 2023-10-18T09:13:58.123 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:57 smithi110 ceph-mon[134818]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:57 smithi170 ceph-mon[135150]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:13:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:57 smithi061 ceph-mon[98573]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:00.109 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:13:59 smithi110 ceph-mon[134818]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:00.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:13:59 smithi170 ceph-mon[135150]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:00.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:13:59 smithi061 ceph-mon[98573]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:01.285 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.285 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.286 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:01.286 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:01 smithi110 ceph-mon[134818]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:01 smithi061 ceph-mon[98573]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:01.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:01.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:01 smithi170 ceph-mon[135150]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:02.856 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 7 on host 'smithi110' 2023-10-18T09:14:02.857 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.853+0000 7fe4467fc700 1 -- 172.21.15.110:0/3310926850 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe45002cf80 con 0x7fe43c0a1480 2023-10-18T09:14:02.857 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 msgr2=0x7fe43c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:02.858 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 0x7fe43c0a3940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe440006010 tx=0x7fe440000bc0 comp rx=0 tx=0).stop 2023-10-18T09:14:02.858 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 msgr2=0x7fe45019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:02.858 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe448007d70 tx=0x7fe44800a340 comp rx=0 tx=0).stop 2023-10-18T09:14:02.858 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 shutdown_connections 2023-10-18T09:14:02.858 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe43c0a1480 0x7fe43c0a3940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4501012d0 0x7fe45019cb50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4500ff730 0x7fe4501955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 --2- 172.21.15.110:0/3310926850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe450100930 0x7fe450195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 >> 172.21.15.110:0/3310926850 conn(0x7fe4500facf0 msgr2=0x7fe450103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 shutdown_connections 2023-10-18T09:14:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:02.855+0000 7fe4575b6700 1 -- 172.21.15.110:0/3310926850 wait complete. 2023-10-18T09:14:03.236 DEBUG:teuthology.orchestra.run.smithi110:osd.7> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.7.service 2023-10-18T09:14:03.239 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2023-10-18T09:14:03.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-18T09:14:03.268 INFO:journalctl@ceph.osd.7.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:14:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:03.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:03.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:03.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:03.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.675 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.675 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.675 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:03.676 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:03.676 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.676 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:03.676 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.677 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:03.677 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:03.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:03.808 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:14:04.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:14:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:14:04.133+0000 7fd58ed15640 -1 osd.7 0 log_to_monitors true 2023-10-18T09:14:05.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:04 smithi170 ceph-mon[135150]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:04 smithi170 ceph-mon[135150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:04 smithi110 ceph-mon[134818]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:04 smithi110 ceph-mon[134818]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:04 smithi061 ceph-mon[98573]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:04 smithi061 ceph-mon[98573]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-18T09:14:05.293 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-18T09:14:05.293 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-18T09:14:05.294 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-10-18T09:14:05.294 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-10-18T09:14:05.294 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00836253 s, 1.3 GB/s 2023-10-18T09:14:05.294 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-10-18T09:14:05.762 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2023-10-18T09:14:05.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:14:05.817+0000 7fd58438c700 -1 osd.7 0 waiting for initial osdmap 2023-10-18T09:14:05.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:14:05.827+0000 7fd57e17d700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:14:05.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:05.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T09:14:05.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:05.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T09:14:05.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:05.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:05.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:05.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:05.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:05.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:05.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:05.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:05.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:05.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:05.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T09:14:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T09:14:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:06.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:06.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:06.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-18T09:14:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: osdmap e50: 8 total, 7 up, 8 in 2023-10-18T09:14:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:14:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:06.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:06.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:06.324 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:14:07.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi110 2023-10-18T09:14:07.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi110 to 5722M 2023-10-18T09:14:07.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:14:07.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T09:14:07.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] boot 2023-10-18T09:14:07.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T09:14:07.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:06 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi110 2023-10-18T09:14:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi110 to 5722M 2023-10-18T09:14:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:14:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T09:14:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] boot 2023-10-18T09:14:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T09:14:07.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:06 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi110 2023-10-18T09:14:07.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi110 to 5722M 2023-10-18T09:14:07.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:14:07.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: osdmap e51: 8 total, 7 up, 8 in 2023-10-18T09:14:07.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: osd.7 [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] boot 2023-10-18T09:14:07.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: osdmap e52: 8 total, 8 up, 8 in 2023-10-18T09:14:07.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:06 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-18T09:14:07.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 -- 172.21.15.170:0/2932342980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17140ff730 msgr2=0x7f17140ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:07.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 --2- 172.21.15.170:0/2932342980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17140ff730 0x7f17140ffb50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1708009990 tx=0x7f1708009ca0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 -- 172.21.15.170:0/2932342980 shutdown_connections 2023-10-18T09:14:07.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 --2- 172.21.15.170:0/2932342980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 0x7f1714107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 --2- 172.21.15.170:0/2932342980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17140ff730 0x7f17140ffb50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 --2- 172.21.15.170:0/2932342980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1714100930 0x7f1714100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 -- 172.21.15.170:0/2932342980 >> 172.21.15.170:0/2932342980 conn(0x7f17140facf0 msgr2=0x7f17140fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:07.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.604+0000 7f1718af9700 1 -- 172.21.15.170:0/2932342980 shutdown_connections 2023-10-18T09:14:07.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 -- 172.21.15.170:0/2932342980 wait complete. 2023-10-18T09:14:07.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 Processor -- start 2023-10-18T09:14:07.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 -- start start 2023-10-18T09:14:07.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17140ff730 0x7f17141956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:07.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:07.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 0x7f171419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:07.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419d210 con 0x7f17141012d0 2023-10-18T09:14:07.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419d350 con 0x7f17140ff730 2023-10-18T09:14:07.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.605+0000 7f1718af9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171419d4e0 con 0x7f1714100930 2023-10-18T09:14:07.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:07.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54426/0 (socket says 172.21.15.170:54426) 2023-10-18T09:14:07.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 -- 172.21.15.170:0/3459100883 learned_addr learned my addr 172.21.15.170:0/3459100883 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-18T09:14:07.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1712d9d700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 0x7f171419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:07.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f171259c700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17140ff730 0x7f17141956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:07.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 -- 172.21.15.170:0/3459100883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17140ff730 msgr2=0x7f17141956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:07.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17140ff730 0x7f17141956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 -- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 msgr2=0x7f171419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:07.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 0x7f171419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:07.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 -- 172.21.15.170:0/3459100883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080095f0 con 0x7f1714100930 2023-10-18T09:14:07.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1711d9b700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f170400d980 tx=0x7f170400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:07.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704009b40 con 0x7f1714100930 2023-10-18T09:14:07.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171419d7c0 con 0x7f1714100930 2023-10-18T09:14:07.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f170400b6e0 con 0x7f1714100930 2023-10-18T09:14:07.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170400f620 con 0x7f1714100930 2023-10-18T09:14:07.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.606+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171419dd10 con 0x7f1714100930 2023-10-18T09:14:07.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.607+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4005320 con 0x7f1714100930 2023-10-18T09:14:07.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.609+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1704009ca0 con 0x7f1714100930 2023-10-18T09:14:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.609+0000 7f17037fe700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.609+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f17040cc3c0 con 0x7f1714100930 2023-10-18T09:14:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.610+0000 7f171259c700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.610+0000 7f171259c700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17080038c0 tx=0x7f1708005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:07.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.613+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1704096560 con 0x7f1714100930 2023-10-18T09:14:07.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:07.808+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f4000bf0 con 0x7f16fc0a1480 2023-10-18T09:14:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:14:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:14:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T09:14:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:08.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:08.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:14:08.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:14:08.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:08.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T09:14:08.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:08.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:08.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:14:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:14:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2023-10-18T09:14:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: osdmap e53: 8 total, 8 up, 8 in 2023-10-18T09:14:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:08.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:08 smithi170 ceph-mon[135150]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:08.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:08 smithi170 ceph-mon[135150]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T09:14:09.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:08 smithi110 ceph-mon[134818]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:09.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:08 smithi110 ceph-mon[134818]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T09:14:09.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:08 smithi061 ceph-mon[98573]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:09.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:08 smithi061 ceph-mon[98573]: osdmap e54: 8 total, 8 up, 8 in 2023-10-18T09:14:10.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:09 smithi170 ceph-mon[135150]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:10.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:09 smithi170 ceph-mon[135150]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T09:14:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:09 smithi110 ceph-mon[134818]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:09 smithi110 ceph-mon[134818]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T09:14:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:09 smithi061 ceph-mon[98573]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:09 smithi061 ceph-mon[98573]: osdmap e55: 8 total, 8 up, 8 in 2023-10-18T09:14:11.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:10 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/1009842195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:10 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:10 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]': finished 2023-10-18T09:14:11.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:10 smithi170 ceph-mon[135150]: osdmap e56: 9 total, 8 up, 9 in 2023-10-18T09:14:11.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:10 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/1009842195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:10 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:10 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]': finished 2023-10-18T09:14:11.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:10 smithi110 ceph-mon[134818]: osdmap e56: 9 total, 8 up, 9 in 2023-10-18T09:14:11.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:11.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:10 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/1009842195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:10 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]: dispatch 2023-10-18T09:14:11.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:10 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff8f0e0e-869f-4e76-a3b5-7f6d43633dff"}]': finished 2023-10-18T09:14:11.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:10 smithi061 ceph-mon[98573]: osdmap e56: 9 total, 8 up, 9 in 2023-10-18T09:14:11.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:11 smithi170 ceph-mon[135150]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:11 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/602944323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:11 smithi110 ceph-mon[134818]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:11 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/602944323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:11 smithi061 ceph-mon[98573]: pgmap v191: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:11 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/602944323' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:14.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:13 smithi170 ceph-mon[135150]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:13 smithi110 ceph-mon[134818]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:14.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:13 smithi061 ceph-mon[98573]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:16.147 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:15 smithi170 ceph-mon[135150]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:15 smithi110 ceph-mon[134818]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:15 smithi061 ceph-mon[98573]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:17.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:17 smithi170 ceph-mon[135150]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:17.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T09:14:17.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:17 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:17.732 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:17 smithi170 ceph-mon[135150]: Deploying daemon osd.8 on smithi170 2023-10-18T09:14:17.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:17 smithi061 ceph-mon[98573]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:17.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T09:14:17.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:17 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:17.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:17 smithi061 ceph-mon[98573]: Deploying daemon osd.8 on smithi170 2023-10-18T09:14:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:17 smithi110 ceph-mon[134818]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:17 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-18T09:14:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:17 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:17 smithi110 ceph-mon[134818]: Deploying daemon osd.8 on smithi170 2023-10-18T09:14:20.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:19 smithi170 ceph-mon[135150]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:19 smithi110 ceph-mon[134818]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:20.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:19 smithi061 ceph-mon[98573]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:21 smithi170 ceph-mon[135150]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:22.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:21 smithi110 ceph-mon[134818]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:21 smithi061 ceph-mon[98573]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:24.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.021+0000 7f17037fe700 1 -- 172.21.15.170:0/3459100883 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f16f4000bf0 con 0x7f16fc0a1480 2023-10-18T09:14:24.024 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2023-10-18T09:14:24.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 msgr2=0x7f16fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:24.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 0x7f16fc0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f17080038c0 tx=0x7f1708005fb0 comp rx=0 tx=0).stop 2023-10-18T09:14:24.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 msgr2=0x7f1714195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:24.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f170400d980 tx=0x7f170400dd40 comp rx=0 tx=0).stop 2023-10-18T09:14:24.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 shutdown_connections 2023-10-18T09:14:24.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f16fc0a1480 0x7f16fc0a3940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:24.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17141012d0 0x7f171419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:24.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17140ff730 0x7f17141956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:24.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 --2- 172.21.15.170:0/3459100883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1714100930 0x7f1714195c10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:24.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 >> 172.21.15.170:0/3459100883 conn(0x7f17140facf0 msgr2=0x7f1714103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:24.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 shutdown_connections 2023-10-18T09:14:24.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:14:24.024+0000 7f1718af9700 1 -- 172.21.15.170:0/3459100883 wait complete. 2023-10-18T09:14:24.052 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:24.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:24.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:24.053 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:24.534 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.8.service 2023-10-18T09:14:24.537 INFO:tasks.cephadm:Deploying osd.9 on smithi061 with /dev/vg_nvme/lv_1... 2023-10-18T09:14:24.538 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T09:14:24.570 INFO:journalctl@ceph.osd.8.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:14:25.250 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.251 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.251 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:25.251 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:25 smithi170 ceph-mon[135150]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:25.295 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:14:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:25 smithi110 ceph-mon[134818]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:25 smithi061 ceph-mon[98573]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:25.520 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:14:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:14:25.247+0000 7f0c5093f640 -1 osd.8 0 log_to_monitors true 2023-10-18T09:14:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:26 smithi170 ceph-mon[135150]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:26 smithi170 ceph-mon[135150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:26.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:26 smithi061 ceph-mon[98573]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:26.194 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:26 smithi061 ceph-mon[98573]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:26.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:26 smithi110 ceph-mon[134818]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:26.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:26 smithi110 ceph-mon[134818]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-18T09:14:27.354 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T09:14:27.355 INFO:teuthology.orchestra.run.smithi061.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T09:14:27.355 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 10+0 records in 2023-10-18T09:14:27.355 INFO:teuthology.orchestra.run.smithi061.stderr:10+0 records out 2023-10-18T09:14:27.355 INFO:teuthology.orchestra.run.smithi061.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0360999 s, 290 MB/s 2023-10-18T09:14:27.355 INFO:teuthology.orchestra.run.smithi061.stderr:--> Zapping successful for: 2023-10-18T09:14:27.392 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:14:27.024+0000 7f0c45fb6700 -1 osd.8 0 waiting for initial osdmap 2023-10-18T09:14:27.392 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:14:27.038+0000 7f0c405a8700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:14:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T09:14:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T09:14:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:27 smithi170 ceph-mon[135150]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:27.394 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T09:14:27.394 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T09:14:27.395 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:27.395 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.395 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.395 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:27 smithi061 ceph-mon[98573]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:27.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-18T09:14:27.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: osdmap e57: 9 total, 8 up, 9 in 2023-10-18T09:14:27.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: from='osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:14:27.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:27 smithi110 ceph-mon[134818]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:27.999 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi061:vg_nvme/lv_1 2023-10-18T09:14:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:14:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T09:14:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi170 2023-10-18T09:14:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi170 to 5722M 2023-10-18T09:14:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:28.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:28.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:14:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T09:14:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi170 2023-10-18T09:14:28.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi170 to 5722M 2023-10-18T09:14:28.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:28.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:28.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:14:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: osdmap e58: 9 total, 8 up, 9 in 2023-10-18T09:14:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi170 2023-10-18T09:14:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi170 to 5722M 2023-10-18T09:14:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:28.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:28.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:28.747 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:14:29.385 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:14:29.386 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:14:29.386 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.386 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] boot 2023-10-18T09:14:29.386 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: osdmap e59: 9 total, 9 up, 9 in 2023-10-18T09:14:29.387 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.387 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:29 smithi061 ceph-mon[98573]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:14:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:14:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] boot 2023-10-18T09:14:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: osdmap e59: 9 total, 9 up, 9 in 2023-10-18T09:14:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:29 smithi170 ceph-mon[135150]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:14:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:14:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: osd.8 [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] boot 2023-10-18T09:14:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: osdmap e59: 9 total, 9 up, 9 in 2023-10-18T09:14:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-18T09:14:29.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:29 smithi110 ceph-mon[134818]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-18T09:14:30.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 -- 172.21.15.61:0/3917550107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0101a40 msgr2=0x7f7af0101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:30.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 --2- 172.21.15.61:0/3917550107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0101a40 0x7f7af0101ea0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40099c0 tx=0x7f7ae4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 -- 172.21.15.61:0/3917550107 shutdown_connections 2023-10-18T09:14:30.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 --2- 172.21.15.61:0/3917550107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 0x7f7af0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 --2- 172.21.15.61:0/3917550107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0100840 0x7f7af0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 --2- 172.21.15.61:0/3917550107 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0101a40 0x7f7af0101ea0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 -- 172.21.15.61:0/3917550107 >> 172.21.15.61:0/3917550107 conn(0x7f7af00fbe00 msgr2=0x7f7af00fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:30.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 -- 172.21.15.61:0/3917550107 shutdown_connections 2023-10-18T09:14:30.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.310+0000 7f7af5565700 1 -- 172.21.15.61:0/3917550107 wait complete. 2023-10-18T09:14:30.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.311+0000 7f7af5565700 1 Processor -- start 2023-10-18T09:14:30.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.311+0000 7f7af5565700 1 -- start start 2023-10-18T09:14:30.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:30.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 0x7f7af0198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:30.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 0x7f7af019fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:30.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a04c0 con 0x7f7af0100840 2023-10-18T09:14:30.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a0600 con 0x7f7af0101a40 2023-10-18T09:14:30.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7af5565700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af01a0790 con 0x7f7af01023e0 2023-10-18T09:14:30.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:30.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 0x7f7af0198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:30.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:54798/0 (socket says 172.21.15.61:54798) 2023-10-18T09:14:30.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 -- 172.21.15.61:0/1605583391 learned_addr learned my addr 172.21.15.61:0/1605583391 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:14:30.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aef7fe700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 0x7f7af019fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:30.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 -- 172.21.15.61:0/1605583391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 msgr2=0x7f7af019fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:30.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 0x7f7af019fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 -- 172.21.15.61:0/1605583391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 msgr2=0x7f7af0198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:30.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 0x7f7af0198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:30.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aeeffd700 1 -- 172.21.15.61:0/1605583391 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40095f0 con 0x7f7af0100840 2023-10-18T09:14:30.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.312+0000 7f7aee7fc700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 0x7f7af0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:14:30.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7aeeffd700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:30.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0009790 con 0x7f7af0100840 2023-10-18T09:14:30.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af01a0a70 con 0x7f7af0100840 2023-10-18T09:14:30.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ae000c810 con 0x7f7af0100840 2023-10-18T09:14:30.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003c70 con 0x7f7af0100840 2023-10-18T09:14:30.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.313+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af01a0f70 con 0x7f7af0100840 2023-10-18T09:14:30.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.314+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af004e4c0 con 0x7f7af0100840 2023-10-18T09:14:30.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.316+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7ae000cc80 con 0x7f7af0100840 2023-10-18T09:14:30.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.317+0000 7f7ad7fff700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 0x7f7ad80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:30.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.317+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f7ae0014070 con 0x7f7af0100840 2023-10-18T09:14:30.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.318+0000 7f7aee7fc700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 0x7f7ad80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:30.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.318+0000 7f7aee7fc700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 0x7f7ad80a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006010 tx=0x7f7ae400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:30.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.321+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7ae00964c0 con 0x7f7af0100840 2023-10-18T09:14:30.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:30 smithi170 ceph-mon[135150]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T09:14:30.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:30 smithi110 ceph-mon[134818]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T09:14:30.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:30 smithi061 ceph-mon[98573]: osdmap e60: 9 total, 9 up, 9 in 2023-10-18T09:14:30.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:30.544+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7af0108900 con 0x7f7ad80a1480 2023-10-18T09:14:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T09:14:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:31 smithi170 ceph-mon[135150]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T09:14:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:31 smithi110 ceph-mon[134818]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: osdmap e61: 9 total, 9 up, 9 in 2023-10-18T09:14:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: from='client.14496 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi061:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:31.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:31 smithi061 ceph-mon[98573]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:32.225 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:32 smithi061 ceph-mon[98573]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T09:14:32.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:32 smithi170 ceph-mon[135150]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T09:14:32.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:32 smithi110 ceph-mon[134818]: osdmap e62: 9 total, 9 up, 9 in 2023-10-18T09:14:33.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:33 smithi170 ceph-mon[135150]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T09:14:33.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:33 smithi170 ceph-mon[135150]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:14:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:33 smithi110 ceph-mon[134818]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T09:14:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:33 smithi110 ceph-mon[134818]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:14:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:33 smithi061 ceph-mon[98573]: osdmap e63: 9 total, 9 up, 9 in 2023-10-18T09:14:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:33 smithi061 ceph-mon[98573]: pgmap v209: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-18T09:14:34.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:34 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]: dispatch 2023-10-18T09:14:34.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:34 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]': finished 2023-10-18T09:14:34.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:34 smithi170 ceph-mon[135150]: osdmap e64: 10 total, 9 up, 10 in 2023-10-18T09:14:34.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:34 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:34.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:34 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]: dispatch 2023-10-18T09:14:34.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:34 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]': finished 2023-10-18T09:14:34.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:34 smithi110 ceph-mon[134818]: osdmap e64: 10 total, 9 up, 10 in 2023-10-18T09:14:34.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:34 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]: dispatch 2023-10-18T09:14:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1981777583' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f66eb5b2-0e99-4f40-ad44-4d81f5a6c910"}]': finished 2023-10-18T09:14:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:34 smithi061 ceph-mon[98573]: osdmap e64: 10 total, 9 up, 10 in 2023-10-18T09:14:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:34 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:35 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1303065804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:35 smithi170 ceph-mon[135150]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-18T09:14:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:35 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1303065804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:35 smithi110 ceph-mon[134818]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-18T09:14:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:35 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1303065804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:35 smithi061 ceph-mon[98573]: pgmap v211: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2023-10-18T09:14:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:37 smithi061 ceph-mon[98573]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:14:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:37 smithi170 ceph-mon[135150]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:14:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:37 smithi110 ceph-mon[134818]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:14:39.906 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:39 smithi061 ceph-mon[98573]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-18T09:14:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:39 smithi170 ceph-mon[135150]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-18T09:14:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:39 smithi110 ceph-mon[134818]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2023-10-18T09:14:42.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:41 smithi170 ceph-mon[135150]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T09:14:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:41 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:41 smithi110 ceph-mon[134818]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T09:14:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:41 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:41 smithi061 ceph-mon[98573]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-18T09:14:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:41 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:43.130 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:42 smithi061 ceph-mon[98573]: Deploying daemon osd.9 on smithi061 2023-10-18T09:14:43.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:42 smithi170 ceph-mon[135150]: Deploying daemon osd.9 on smithi061 2023-10-18T09:14:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:42 smithi110 ceph-mon[134818]: Deploying daemon osd.9 on smithi061 2023-10-18T09:14:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:43 smithi170 ceph-mon[135150]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:43 smithi110 ceph-mon[134818]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:44.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:43 smithi061 ceph-mon[98573]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:45 smithi170 ceph-mon[135150]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:46.147 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:45 smithi061 ceph-mon[98573]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:45 smithi110 ceph-mon[134818]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:47 smithi061 ceph-mon[98573]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:47 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:47 smithi170 ceph-mon[135150]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:47 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:47 smithi110 ceph-mon[134818]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:47.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:47 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:50.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:49 smithi061 ceph-mon[98573]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:50.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:49 smithi170 ceph-mon[135150]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:49 smithi110 ceph-mon[134818]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:50.964 INFO:teuthology.orchestra.run.smithi061.stdout:Created osd(s) 9 on host 'smithi061' 2023-10-18T09:14:50.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.960+0000 7f7ad7fff700 1 -- 172.21.15.61:0/1605583391 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7af0108900 con 0x7f7ad80a1480 2023-10-18T09:14:50.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 msgr2=0x7f7ad80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:50.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 0x7f7ad80a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006010 tx=0x7f7ae400b540 comp rx=0 tx=0).stop 2023-10-18T09:14:50.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 msgr2=0x7f7af0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:50.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:14:50.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 shutdown_connections 2023-10-18T09:14:50.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7ad80a1480 0x7f7ad80a3940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:50.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af01023e0 0x7f7af019fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:50.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7af0100840 0x7f7af0198980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:50.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 --2- 172.21.15.61:0/1605583391 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af0101a40 0x7f7af0198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:50.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 >> 172.21.15.61:0/1605583391 conn(0x7f7af00fbe00 msgr2=0x7f7af01071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:50.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 shutdown_connections 2023-10-18T09:14:50.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:14:50.963+0000 7f7af5565700 1 -- 172.21.15.61:0/1605583391 wait complete. 2023-10-18T09:14:51.482 DEBUG:teuthology.orchestra.run.smithi061:osd.9> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.9.service 2023-10-18T09:14:51.487 INFO:tasks.cephadm:Deploying osd.10 on smithi110 with /dev/vg_nvme/lv_1... 2023-10-18T09:14:51.487 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T09:14:51.567 INFO:journalctl@ceph.osd.9.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:14:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:51.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:51.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:51 smithi061 ceph-mon[98573]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:51.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:51.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:51.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:51.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:51.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:51.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:51 smithi170 ceph-mon[135150]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:51.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:51.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:51.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:51.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:51.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:14:51.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:51.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:51 smithi110 ceph-mon[134818]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-18T09:14:52.057 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:14:53.125 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:14:52.959+0000 7f6d7ed89700 -1 osd.9 0 waiting for initial osdmap 2023-10-18T09:14:53.125 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:14:52.975+0000 7f6d7ab7e700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:14:53.126 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T09:14:53.126 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T09:14:53.126 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:53.126 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.126 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.127 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:52 smithi061 ceph-mon[98573]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:53.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T09:14:53.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T09:14:53.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:53.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:52 smithi110 ceph-mon[134818]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-18T09:14:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: osdmap e65: 10 total, 9 up, 10 in 2023-10-18T09:14:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: from='osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]: dispatch 2023-10-18T09:14:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:52 smithi170 ceph-mon[135150]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:53.740 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T09:14:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T09:14:53.741 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2023-10-18T09:14:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2023-10-18T09:14:53.741 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00820824 s, 1.3 GB/s 2023-10-18T09:14:53.741 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2023-10-18T09:14:54.097 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:53 smithi110 ceph-mon[134818]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:14:54.097 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:53 smithi110 ceph-mon[134818]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T09:14:54.097 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.097 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.098 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2023-10-18T09:14:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:53 smithi061 ceph-mon[98573]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:14:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:53 smithi061 ceph-mon[98573]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T09:14:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:53 smithi170 ceph-mon[135150]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi061", "root=default"]}]': finished 2023-10-18T09:14:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:53 smithi170 ceph-mon[135150]: osdmap e66: 10 total, 9 up, 10 in 2023-10-18T09:14:54.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:54.656 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:14:55.171 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:14:55.172 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:14:55.172 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] boot 2023-10-18T09:14:55.172 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: osdmap e67: 10 total, 10 up, 10 in 2023-10-18T09:14:55.172 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:55.173 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:54 smithi110 ceph-mon[134818]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:14:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:14:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] boot 2023-10-18T09:14:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: osdmap e67: 10 total, 10 up, 10 in 2023-10-18T09:14:55.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:55.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:54 smithi061 ceph-mon[98573]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:14:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:14:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: osd.9 [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] boot 2023-10-18T09:14:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: osdmap e67: 10 total, 10 up, 10 in 2023-10-18T09:14:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-18T09:14:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:54 smithi170 ceph-mon[135150]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-18T09:14:55.862 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3760090472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 msgr2=0x7f5f18101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:55.862 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3760090472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18101ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f080099c0 tx=0x7f5f08009cd0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.862 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3760090472 shutdown_connections 2023-10-18T09:14:55.863 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3760090472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f181023e0 0x7f5f18108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.863 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3760090472 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f18100840 0x7f5f18100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.863 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3760090472 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18101ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.863 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3760090472 >> 172.21.15.110:0/3760090472 conn(0x7f5f180fbe00 msgr2=0x7f5f180fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:14:55.863 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3760090472 shutdown_connections 2023-10-18T09:14:55.864 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.859+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3760090472 wait complete. 2023-10-18T09:14:55.864 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 Processor -- start 2023-10-18T09:14:55.864 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 -- start start 2023-10-18T09:14:55.866 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f18100840 0x7f5f18074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:55.866 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:55.867 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f181023e0 0x7f5f18073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:55.867 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180738c0 con 0x7f5f181023e0 2023-10-18T09:14:55.867 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f18073a30 con 0x7f5f18101a40 2023-10-18T09:14:55.867 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f1e0c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f181a0770 con 0x7f5f18100840 2023-10-18T09:14:55.867 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:55.868 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60460/0 (socket says 172.21.15.110:60460) 2023-10-18T09:14:55.868 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 -- 172.21.15.110:0/3989323850 learned_addr learned my addr 172.21.15.110:0/3989323850 (peer_addr_for_me v2:172.21.15.110:0/0) 2023-10-18T09:14:55.868 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f177fe700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f18100840 0x7f5f18074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:55.868 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f17fff700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f181023e0 0x7f5f18073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:55.868 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 -- 172.21.15.110:0/3989323850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f18100840 msgr2=0x7f5f18074540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:55.869 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f18100840 0x7f5f18074540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.869 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 -- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f181023e0 msgr2=0x7f5f18073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:14:55.869 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f181023e0 0x7f5f18073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:14:55.869 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.860+0000 7f5f16ffd700 1 -- 172.21.15.110:0/3989323850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f080095f0 con 0x7f5f18101a40 2023-10-18T09:14:55.869 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f16ffd700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f080036f0 tx=0x7f5f080047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:55.870 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0801d070 con 0x7f5f18101a40 2023-10-18T09:14:55.870 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f181a09a0 con 0x7f5f18101a40 2023-10-18T09:14:55.870 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f08004090 con 0x7f5f18101a40 2023-10-18T09:14:55.870 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08022d40 con 0x7f5f18101a40 2023-10-18T09:14:55.870 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.861+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f181a0e40 con 0x7f5f18101a40 2023-10-18T09:14:55.871 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.862+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f18107cd0 con 0x7f5f18101a40 2023-10-18T09:14:55.871 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.863+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5f08022470 con 0x7f5f18101a40 2023-10-18T09:14:55.871 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.863+0000 7f5f14ff9700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:14:55.871 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.863+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f5f080cd670 con 0x7f5f18101a40 2023-10-18T09:14:55.871 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.864+0000 7f5f177fe700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:14:55.872 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.864+0000 7f5f177fe700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 0x7f5f040a3940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f181018a0 tx=0x7f5f0000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:14:55.872 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:55.868+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5f0805d100 con 0x7f5f18101a40 2023-10-18T09:14:56.068 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:14:56.066+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f18060c30 con 0x7f5f040a1480 2023-10-18T09:14:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T09:14:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi061 2023-10-18T09:14:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 4317M 2023-10-18T09:14:56.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:56.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:56.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T09:14:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi061 2023-10-18T09:14:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 4317M 2023-10-18T09:14:56.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:56.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:56.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: osdmap e68: 10 total, 10 up, 10 in 2023-10-18T09:14:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi061 2023-10-18T09:14:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:14:56.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 4317M 2023-10-18T09:14:56.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:56.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:56.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:14:56.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:14:57.195 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:56 smithi110 ceph-mon[134818]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:57.195 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:57.195 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:57.195 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:57.196 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:56 smithi110 ceph-mon[134818]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:56 smithi061 ceph-mon[98573]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:56 smithi061 ceph-mon[98573]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:57.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:56 smithi170 ceph-mon[135150]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:14:57.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:14:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:14:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:14:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:56 smithi170 ceph-mon[135150]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:58 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/476607908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:58 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:58 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]': finished 2023-10-18T09:14:58.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:58 smithi061 ceph-mon[98573]: osdmap e69: 11 total, 10 up, 11 in 2023-10-18T09:14:58.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:14:58.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:58 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/476607908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:58 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:58 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]': finished 2023-10-18T09:14:58.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:58 smithi170 ceph-mon[135150]: osdmap e69: 11 total, 10 up, 11 in 2023-10-18T09:14:58.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:14:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:58 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/476607908' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:58 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]: dispatch 2023-10-18T09:14:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:58 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cbca8f8-088f-43d3-bd0e-dcb212e8756e"}]': finished 2023-10-18T09:14:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:58 smithi110 ceph-mon[134818]: osdmap e69: 11 total, 10 up, 11 in 2023-10-18T09:14:58.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:14:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:59 smithi061 ceph-mon[98573]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:14:59 smithi061 ceph-mon[98573]: from='client.? 172.21.15.110:0/1959422199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:59.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:59 smithi170 ceph-mon[135150]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:59.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:14:59 smithi170 ceph-mon[135150]: from='client.? 172.21.15.110:0/1959422199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:14:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:59 smithi110 ceph-mon[134818]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:14:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:14:59 smithi110 ceph-mon[134818]: from='client.? 172.21.15.110:0/1959422199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:15:02.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:01 smithi170 ceph-mon[135150]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:02.156 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:01 smithi110 ceph-mon[134818]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:02.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:01 smithi061 ceph-mon[98573]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:03 smithi110 ceph-mon[134818]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:04.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:03 smithi170 ceph-mon[135150]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:04.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:03 smithi061 ceph-mon[98573]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:05 smithi170 ceph-mon[135150]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T09:15:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:06.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:05 smithi170 ceph-mon[135150]: Deploying daemon osd.10 on smithi110 2023-10-18T09:15:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:05 smithi110 ceph-mon[134818]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T09:15:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:05 smithi110 ceph-mon[134818]: Deploying daemon osd.10 on smithi110 2023-10-18T09:15:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:05 smithi061 ceph-mon[98573]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-18T09:15:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:05 smithi061 ceph-mon[98573]: Deploying daemon osd.10 on smithi110 2023-10-18T09:15:07.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:07 smithi110 ceph-mon[134818]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:07.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:07 smithi061 ceph-mon[98573]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:07.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:07 smithi170 ceph-mon[135150]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:09.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:09 smithi110 ceph-mon[134818]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:09.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:09.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:09.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:10.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:09 smithi170 ceph-mon[135150]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:10.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:10.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:10.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:09 smithi061 ceph-mon[98573]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:11.784 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 10 on host 'smithi110' 2023-10-18T09:15:11.784 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.780+0000 7f5f14ff9700 1 -- 172.21.15.110:0/3989323850 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f5f18060c30 con 0x7f5f040a1480 2023-10-18T09:15:11.786 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 msgr2=0x7f5f040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:11.786 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 0x7f5f040a3940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5f181018a0 tx=0x7f5f0000b410 comp rx=0 tx=0).stop 2023-10-18T09:15:11.786 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 msgr2=0x7f5f18072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:11.786 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f080036f0 tx=0x7f5f080047b0 comp rx=0 tx=0).stop 2023-10-18T09:15:11.787 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 shutdown_connections 2023-10-18T09:15:11.787 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5f040a1480 0x7f5f040a3940 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:11.787 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5f181023e0 0x7f5f18073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:11.787 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f18100840 0x7f5f18074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:11.787 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 --2- 172.21.15.110:0/3989323850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f18101a40 0x7f5f18072be0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:11.788 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.782+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 >> 172.21.15.110:0/3989323850 conn(0x7f5f180fbe00 msgr2=0x7f5f18104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:11.788 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.783+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 shutdown_connections 2023-10-18T09:15:11.788 INFO:teuthology.orchestra.run.smithi110.stderr:2023-10-18T09:15:11.783+0000 7f5f1e0c2700 1 -- 172.21.15.110:0/3989323850 wait complete. 2023-10-18T09:15:12.121 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:11 smithi110 ceph-mon[134818]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:12.121 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.122 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.122 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.122 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:11 smithi170 ceph-mon[135150]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.147 DEBUG:teuthology.orchestra.run.smithi110:osd.10> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.10.service 2023-10-18T09:15:12.149 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2023-10-18T09:15:12.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-18T09:15:12.177 INFO:journalctl@ceph.osd.10.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:15:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:11 smithi061 ceph-mon[98573]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:12.717 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:15:13.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:15:13 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:15:13.178+0000 7f0f63afe640 -1 osd.10 0 log_to_monitors true 2023-10-18T09:15:14.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:13 smithi170 ceph-mon[135150]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:14.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:13 smithi170 ceph-mon[135150]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:13 smithi170 ceph-mon[135150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:13 smithi110 ceph-mon[134818]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:13 smithi110 ceph-mon[134818]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:13 smithi110 ceph-mon[134818]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.215 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-18T09:15:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-18T09:15:14.216 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2023-10-18T09:15:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2023-10-18T09:15:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00823213 s, 1.3 GB/s 2023-10-18T09:15:14.216 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2023-10-18T09:15:14.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:13 smithi061 ceph-mon[98573]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:14.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:13 smithi061 ceph-mon[98573]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:13 smithi061 ceph-mon[98573]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-18T09:15:14.659 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2023-10-18T09:15:15.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:15:14.888+0000 7f0f59175700 -1 osd.10 0 waiting for initial osdmap 2023-10-18T09:15:15.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:15:14.901+0000 7f0f52f66700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:15:15.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T09:15:15.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T09:15:15.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:15.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:15.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:15.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.220 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:15:15.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T09:15:15.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T09:15:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:15.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:15.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-18T09:15:15.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: osdmap e70: 11 total, 10 up, 11 in 2023-10-18T09:15:15.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:15.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2023-10-18T09:15:15.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:15.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:15.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:15.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi110 2023-10-18T09:15:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi110 to 4292M 2023-10-18T09:15:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:15:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T09:15:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] boot 2023-10-18T09:15:16.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: osdmap e72: 11 total, 11 up, 11 in 2023-10-18T09:15:16.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi110 2023-10-18T09:15:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi110 to 4292M 2023-10-18T09:15:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:15:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T09:15:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] boot 2023-10-18T09:15:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: osdmap e72: 11 total, 11 up, 11 in 2023-10-18T09:15:16.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi110 2023-10-18T09:15:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi110 to 4292M 2023-10-18T09:15:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-18T09:15:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2023-10-18T09:15:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: osdmap e71: 11 total, 10 up, 11 in 2023-10-18T09:15:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: osd.10 [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] boot 2023-10-18T09:15:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: osdmap e72: 11 total, 11 up, 11 in 2023-10-18T09:15:16.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-18T09:15:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 -- 172.21.15.170:0/4235519599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 msgr2=0x7fd9e4108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 --2- 172.21.15.170:0/4235519599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e4108c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b0a0 tx=0x7fd9dc00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 -- 172.21.15.170:0/4235519599 shutdown_connections 2023-10-18T09:15:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 --2- 172.21.15.170:0/4235519599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e4108c90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 --2- 172.21.15.170:0/4235519599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9e4100840 0x7fd9e4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 --2- 172.21.15.170:0/4235519599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4101a40 0x7fd9e4101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 -- 172.21.15.170:0/4235519599 >> 172.21.15.170:0/4235519599 conn(0x7fd9e40fbde0 msgr2=0x7fd9e40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 -- 172.21.15.170:0/4235519599 shutdown_connections 2023-10-18T09:15:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.490+0000 7fd9e9210700 1 -- 172.21.15.170:0/4235519599 wait complete. 2023-10-18T09:15:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.491+0000 7fd9e9210700 1 Processor -- start 2023-10-18T09:15:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.491+0000 7fd9e9210700 1 -- start start 2023-10-18T09:15:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4100840 0x7fd9e4196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9e4101a40 0x7fd9e4196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419e440 con 0x7fd9e4101a40 2023-10-18T09:15:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419e580 con 0x7fd9e4100840 2023-10-18T09:15:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e419e710 con 0x7fd9e41023e0 2023-10-18T09:15:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54568/0 (socket says 172.21.15.170:54568) 2023-10-18T09:15:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 -- 172.21.15.170:0/2944461917 learned_addr learned my addr 172.21.15.170:0/2944461917 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-18T09:15:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e2d9d700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4100840 0x7fd9e4196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:16.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 -- 172.21.15.170:0/2944461917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4100840 msgr2=0x7fd9e4196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4100840 0x7fd9e4196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 -- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9e4101a40 msgr2=0x7fd9e4196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9e4101a40 0x7fd9e4196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 -- 172.21.15.170:0/2944461917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc009d00 con 0x7fd9e41023e0 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e359e700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b070 tx=0x7fd9dc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:16.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00e040 con 0x7fd9e41023e0 2023-10-18T09:15:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e419e990 con 0x7fd9e41023e0 2023-10-18T09:15:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9dc00ceb0 con 0x7fd9e41023e0 2023-10-18T09:15:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc01dab0 con 0x7fd9e41023e0 2023-10-18T09:15:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.492+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e419eda0 con 0x7fd9e41023e0 2023-10-18T09:15:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.494+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e404e530 con 0x7fd9e41023e0 2023-10-18T09:15:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.494+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd9dc0058a0 con 0x7fd9e41023e0 2023-10-18T09:15:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.495+0000 7fd9cbfff700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 0x7fd9cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.495+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fd9dc0ce0e0 con 0x7fd9e41023e0 2023-10-18T09:15:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.496+0000 7fd9e2d9d700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 0x7fd9cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.496+0000 7fd9e2d9d700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 0x7fd9cc0a3940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9e41018a0 tx=0x7fd9d4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.500+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd9e404e530 con 0x7fd9e41023e0 2023-10-18T09:15:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:16.696+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9e404e530 con 0x7fd9cc0a1480 2023-10-18T09:15:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:15:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:15:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T09:15:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:15:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:15:17.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:15:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:15:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T09:15:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:15:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:15:17.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:17.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:15:17.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:15:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: osdmap e73: 11 total, 11 up, 11 in 2023-10-18T09:15:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-18T09:15:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-18T09:15:17.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:18.194 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:17 smithi170 ceph-mon[135150]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:18.195 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:17 smithi170 ceph-mon[135150]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:18.195 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:17 smithi170 ceph-mon[135150]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T09:15:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:17 smithi110 ceph-mon[134818]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:17 smithi110 ceph-mon[134818]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:17 smithi110 ceph-mon[134818]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T09:15:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:17 smithi061 ceph-mon[98573]: from='client.24514 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:17 smithi061 ceph-mon[98573]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:17 smithi061 ceph-mon[98573]: osdmap e74: 11 total, 11 up, 11 in 2023-10-18T09:15:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T09:15:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:19.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/3070887610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]': finished 2023-10-18T09:15:19.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: osdmap e76: 12 total, 11 up, 12 in 2023-10-18T09:15:19.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:19.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T09:15:19.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:19.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/3070887610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]': finished 2023-10-18T09:15:19.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: osdmap e76: 12 total, 11 up, 12 in 2023-10-18T09:15:19.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:19.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: osdmap e75: 11 total, 11 up, 11 in 2023-10-18T09:15:19.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/3070887610' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]: dispatch 2023-10-18T09:15:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f2b592b-6b55-4c04-b153-da70a63b88f8"}]': finished 2023-10-18T09:15:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: osdmap e76: 12 total, 11 up, 12 in 2023-10-18T09:15:19.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:20.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:20 smithi061 ceph-mon[98573]: from='client.? 172.21.15.170:0/2204030647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:15:20.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:20 smithi170 ceph-mon[135150]: from='client.? 172.21.15.170:0/2204030647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:15:20.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:20 smithi110 ceph-mon[134818]: from='client.? 172.21.15.170:0/2204030647' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-18T09:15:21.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:21 smithi170 ceph-mon[135150]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:21.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:21 smithi110 ceph-mon[134818]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:22.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:21 smithi061 ceph-mon[98573]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:23 smithi170 ceph-mon[135150]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:15:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:23 smithi110 ceph-mon[134818]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:15:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:23 smithi061 ceph-mon[98573]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-18T09:15:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:25 smithi170 ceph-mon[135150]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-10-18T09:15:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:25 smithi110 ceph-mon[134818]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-10-18T09:15:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:25 smithi061 ceph-mon[98573]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-10-18T09:15:27.068 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T09:15:27.140 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T09:15:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-18T09:15:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:28.049 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:27 smithi170 ceph-mon[135150]: Deploying daemon osd.11 on smithi170 2023-10-18T09:15:28.049 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:27 smithi170 ceph-mon[135150]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:15:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:27 smithi110 ceph-mon[134818]: Deploying daemon osd.11 on smithi170 2023-10-18T09:15:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:27 smithi110 ceph-mon[134818]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:15:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:27 smithi061 ceph-mon[98573]: Deploying daemon osd.11 on smithi170 2023-10-18T09:15:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:27 smithi061 ceph-mon[98573]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-10-18T09:15:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:29 smithi170 ceph-mon[135150]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:15:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:29 smithi110 ceph-mon[134818]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:15:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:29 smithi061 ceph-mon[98573]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-18T09:15:31.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:31.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:31.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:32.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:31 smithi170 ceph-mon[135150]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-18T09:15:32.220 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:31 smithi110 ceph-mon[134818]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-18T09:15:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:31 smithi061 ceph-mon[98573]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-10-18T09:15:33.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.260+0000 7fd9cbfff700 1 -- 172.21.15.170:0/2944461917 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd9e404e530 con 0x7fd9cc0a1480 2023-10-18T09:15:33.264 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2023-10-18T09:15:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 msgr2=0x7fd9cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 0x7fd9cc0a3940 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd9e41018a0 tx=0x7fd9d4009190 comp rx=0 tx=0).stop 2023-10-18T09:15:33.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 msgr2=0x7fd9e419de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:33.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc00b070 tx=0x7fd9dc004c80 comp rx=0 tx=0).stop 2023-10-18T09:15:33.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 shutdown_connections 2023-10-18T09:15:33.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd9cc0a1480 0x7fd9cc0a3940 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:33.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9e41023e0 0x7fd9e419de70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:33.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9e4100840 0x7fd9e4196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:33.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 --2- 172.21.15.170:0/2944461917 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd9e4101a40 0x7fd9e4196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:33.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 >> 172.21.15.170:0/2944461917 conn(0x7fd9e40fbde0 msgr2=0x7fd9e4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:33.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.262+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 shutdown_connections 2023-10-18T09:15:33.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:33.263+0000 7fd9e9210700 1 -- 172.21.15.170:0/2944461917 wait complete. 2023-10-18T09:15:33.716 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.11.service 2023-10-18T09:15:33.719 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-18T09:15:33.719 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-18T09:15:33.751 INFO:journalctl@ceph.osd.11.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:15:34.064 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-18T09:15:34.065 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.065 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.065 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:34.065 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:34.066 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.066 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.066 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.066 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-18T09:15:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-18T09:15:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:34.392 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:15:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:15:34.060+0000 7fa639260640 -1 osd.11 0 log_to_monitors true 2023-10-18T09:15:34.481 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:15:35.118 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:34 smithi170 ceph-mon[135150]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T09:15:35.142 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:34 smithi061 ceph-mon[98573]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T09:15:35.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:34 smithi110 ceph-mon[134818]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-18T09:15:36.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.124+0000 7fca5d78e700 1 -- 172.21.15.61:0/2520071728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 msgr2=0x7fca58101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:36.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.124+0000 7fca5d78e700 1 --2- 172.21.15.61:0/2520071728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58101ea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0099c0 tx=0x7fca4c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 -- 172.21.15.61:0/2520071728 shutdown_connections 2023-10-18T09:15:36.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 --2- 172.21.15.61:0/2520071728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 0x7fca58108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 --2- 172.21.15.61:0/2520071728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 --2- 172.21.15.61:0/2520071728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58101ea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 -- 172.21.15.61:0/2520071728 >> 172.21.15.61:0/2520071728 conn(0x7fca580fbe00 msgr2=0x7fca580fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:36.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 -- 172.21.15.61:0/2520071728 shutdown_connections 2023-10-18T09:15:36.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.125+0000 7fca5d78e700 1 -- 172.21.15.61:0/2520071728 wait complete. 2023-10-18T09:15:36.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 Processor -- start 2023-10-18T09:15:36.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 -- start start 2023-10-18T09:15:36.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:36.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:36.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 0x7fca5819fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:36.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581a0470 con 0x7fca58101a40 2023-10-18T09:15:36.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581a05b0 con 0x7fca58100840 2023-10-18T09:15:36.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.126+0000 7fca5d78e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca581a0740 con 0x7fca581023e0 2023-10-18T09:15:36.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:36.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca567fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:36.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:57482/0 (socket says 172.21.15.61:57482) 2023-10-18T09:15:36.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 -- 172.21.15.61:0/3777952217 learned_addr learned my addr 172.21.15.61:0/3777952217 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:36.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca577fe700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 0x7fca5819fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:36.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 -- 172.21.15.61:0/3777952217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 msgr2=0x7fca5819fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:36.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 0x7fca5819fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 -- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 msgr2=0x7fca58198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:36.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 -- 172.21.15.61:0/3777952217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0095f0 con 0x7fca58100840 2023-10-18T09:15:36.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca567fc700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:36.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca56ffd700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e970 tx=0x7fca4800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:36.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.127+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48009790 con 0x7fca58100840 2023-10-18T09:15:36.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.128+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca4800c7e0 con 0x7fca58100840 2023-10-18T09:15:36.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.128+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca581a0a20 con 0x7fca58100840 2023-10-18T09:15:36.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.128+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca48003c70 con 0x7fca58100840 2023-10-18T09:15:36.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.128+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca581a0f20 con 0x7fca58100840 2023-10-18T09:15:36.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.129+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5804e4c0 con 0x7fca58100840 2023-10-18T09:15:36.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.131+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fca4800c950 con 0x7fca58100840 2023-10-18T09:15:36.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.132+0000 7fca3ffff700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 0x7fca400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:36.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.132+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fca48014070 con 0x7fca58100840 2023-10-18T09:15:36.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.133+0000 7fca567fc700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 0x7fca400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:36.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.133+0000 7fca567fc700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 0x7fca400a3940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009990 tx=0x7fca4c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:36.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.134+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fca48096690 con 0x7fca58100840 2023-10-18T09:15:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:35 smithi110 ceph-mon[134818]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:35 smithi110 ceph-mon[134818]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T09:15:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:35 smithi110 ceph-mon[134818]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T09:15:36.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:35 smithi110 ceph-mon[134818]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:15:36.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:36.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:35 smithi061 ceph-mon[98573]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:36.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:35 smithi061 ceph-mon[98573]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T09:15:36.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:35 smithi061 ceph-mon[98573]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T09:15:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:35 smithi061 ceph-mon[98573]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:15:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:36.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.350+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fca5810c040 con 0x7fca58100840 2023-10-18T09:15:36.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.351+0000 7fca3ffff700 1 -- 172.21.15.61:0/3777952217 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fca48003dd0 con 0x7fca58100840 2023-10-18T09:15:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:36.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 msgr2=0x7fca400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:36.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 0x7fca400a3940 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009990 tx=0x7fca4c01a040 comp rx=0 tx=0).stop 2023-10-18T09:15:36.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 msgr2=0x7fca58198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:36.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e970 tx=0x7fca4800ec80 comp rx=0 tx=0).stop 2023-10-18T09:15:36.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 shutdown_connections 2023-10-18T09:15:36.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca400a1480 0x7fca400a3940 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca581023e0 0x7fca5819fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca58100840 0x7fca58198930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 --2- 172.21.15.61:0/3777952217 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca58101a40 0x7fca58198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:36.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 >> 172.21.15.61:0/3777952217 conn(0x7fca580fbe00 msgr2=0x7fca58072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:36.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 shutdown_connections 2023-10-18T09:15:36.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:36.354+0000 7fca5d78e700 1 -- 172.21.15.61:0/3777952217 wait complete. 2023-10-18T09:15:36.392 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:15:35.902+0000 7fa62e8d7700 -1 osd.11 0 waiting for initial osdmap 2023-10-18T09:15:36.392 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:15:35.918+0000 7fa6266c4700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-18T09:15:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-mon[135150]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-mon[135150]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-18T09:15:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-mon[135150]: osdmap e77: 12 total, 11 up, 12 in 2023-10-18T09:15:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-mon[135150]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2023-10-18T09:15:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:36.880 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1697620515,"num_in_osds":12,"osd_in_since":1697620519,"num_remapped_pgs":0} 2023-10-18T09:15:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:15:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T09:15:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3777952217' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:37.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:37.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:36 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:15:37.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T09:15:37.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3777952217' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:37.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:37.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:37.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:36 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2023-10-18T09:15:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: osdmap e78: 12 total, 11 up, 12 in 2023-10-18T09:15:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3777952217' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:15:37.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:37.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:37.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:36 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:37.882 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-18T09:15:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: purged_snaps scrub starts 2023-10-18T09:15:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: purged_snaps scrub ok 2023-10-18T09:15:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: Detected new or changed devices on smithi170 2023-10-18T09:15:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi170 to 4292M 2023-10-18T09:15:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] boot 2023-10-18T09:15:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: osdmap e79: 12 total, 12 up, 12 in 2023-10-18T09:15:38.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: purged_snaps scrub starts 2023-10-18T09:15:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: purged_snaps scrub ok 2023-10-18T09:15:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: Detected new or changed devices on smithi170 2023-10-18T09:15:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi170 to 4292M 2023-10-18T09:15:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] boot 2023-10-18T09:15:38.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: osdmap e79: 12 total, 12 up, 12 in 2023-10-18T09:15:38.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: purged_snaps scrub starts 2023-10-18T09:15:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: purged_snaps scrub ok 2023-10-18T09:15:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: Detected new or changed devices on smithi170 2023-10-18T09:15:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi170 to 4292M 2023-10-18T09:15:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-18T09:15:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: osd.11 [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] boot 2023-10-18T09:15:38.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: osdmap e79: 12 total, 12 up, 12 in 2023-10-18T09:15:38.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-18T09:15:38.646 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:15:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:38 smithi110 ceph-mon[134818]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T09:15:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:38 smithi110 ceph-mon[134818]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:39.251 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:38 smithi061 ceph-mon[98573]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T09:15:39.251 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:38 smithi061 ceph-mon[98573]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:38 smithi170 ceph-mon[135150]: osdmap e80: 12 total, 12 up, 12 in 2023-10-18T09:15:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:38 smithi170 ceph-mon[135150]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:40.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 -- 172.21.15.61:0/89736550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc1023e0 msgr2=0x7f8bbc108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:40.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/89736550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc108c90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b0d0 tx=0x7f8bb400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 -- 172.21.15.61:0/89736550 shutdown_connections 2023-10-18T09:15:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/89736550 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc108c90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/89736550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc100840 0x7f8bbc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/89736550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 0x7f8bbc101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 -- 172.21.15.61:0/89736550 >> 172.21.15.61:0/89736550 conn(0x7f8bbc0fbe00 msgr2=0x7f8bbc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:40.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.272+0000 7f8bc2f68700 1 -- 172.21.15.61:0/89736550 shutdown_connections 2023-10-18T09:15:40.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.273+0000 7f8bc2f68700 1 -- 172.21.15.61:0/89736550 wait complete. 2023-10-18T09:15:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.273+0000 7f8bc2f68700 1 Processor -- start 2023-10-18T09:15:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 -- start start 2023-10-18T09:15:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 0x7f8bbc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1a0600 con 0x7f8bbc100840 2023-10-18T09:15:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1a0740 con 0x7f8bbc101a40 2023-10-18T09:15:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc2f68700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1a08d0 con 0x7f8bbc1023e0 2023-10-18T09:15:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.274+0000 7f8bc0d04700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56142/0 (socket says 172.21.15.61:56142) 2023-10-18T09:15:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 -- 172.21.15.61:0/2714870454 learned_addr learned my addr 172.21.15.61:0/2714870454 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bbbfff700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 0x7f8bbc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc1505700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:40.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 -- 172.21.15.61:0/2714870454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 msgr2=0x7f8bbc1a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:40.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc1a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 -- 172.21.15.61:0/2714870454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 msgr2=0x7f8bbc199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:40.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 0x7f8bbc199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 -- 172.21.15.61:0/2714870454 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4009d00 con 0x7f8bbc100840 2023-10-18T09:15:40.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc1505700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:15:40.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bc0d04700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e980 tx=0x7f8bb000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:40.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.275+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0009770 con 0x7f8bbc100840 2023-10-18T09:15:40.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.276+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc1a0bb0 con 0x7f8bbc100840 2023-10-18T09:15:40.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.276+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bb000eed0 con 0x7f8bbc100840 2023-10-18T09:15:40.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.276+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003cd0 con 0x7f8bbc100840 2023-10-18T09:15:40.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.276+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc1a1060 con 0x7f8bbc100840 2023-10-18T09:15:40.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.277+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc04e4c0 con 0x7f8bbc100840 2023-10-18T09:15:40.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.278+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8bb000cbd0 con 0x7f8bbc100840 2023-10-18T09:15:40.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.280+0000 7f8bb9ffb700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 0x7f8bac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:40.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.280+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8bb0014070 con 0x7f8bbc100840 2023-10-18T09:15:40.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.280+0000 7f8bbbfff700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 0x7f8bac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:40.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.281+0000 7f8bbbfff700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 0x7f8bac0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009c60 tx=0x7f8ba8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:40.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.284+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8bb00965d0 con 0x7f8bbc100840 2023-10-18T09:15:40.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.497+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8bbc10c0c0 con 0x7f8bbc100840 2023-10-18T09:15:40.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.497+0000 7f8bb9ffb700 1 -- 172.21.15.61:0/2714870454 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8bb0099bf0 con 0x7f8bbc100840 2023-10-18T09:15:40.500 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:40.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.499+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 msgr2=0x7f8bac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:40.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.499+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 0x7f8bac0a3940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009c60 tx=0x7f8ba8009260 comp rx=0 tx=0).stop 2023-10-18T09:15:40.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.499+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 msgr2=0x7f8bbc198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:40.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.499+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000e980 tx=0x7f8bb000ec90 comp rx=0 tx=0).stop 2023-10-18T09:15:40.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 shutdown_connections 2023-10-18T09:15:40.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bac0a1480 0x7f8bac0a3940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bbc1023e0 0x7f8bbc1a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bbc100840 0x7f8bbc198ac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 --2- 172.21.15.61:0/2714870454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bbc101a40 0x7f8bbc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:40.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 >> 172.21.15.61:0/2714870454 conn(0x7f8bbc0fbe00 msgr2=0x7f8bbc072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:40.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 shutdown_connections 2023-10-18T09:15:40.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:40.500+0000 7f8bc2f68700 1 -- 172.21.15.61:0/2714870454 wait complete. 2023-10-18T09:15:40.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:40 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2714870454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:40.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:40 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2714870454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:40.972 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:40 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2714870454' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-18T09:15:41.044 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1697620536,"num_in_osds":12,"osd_in_since":1697620519,"num_remapped_pgs":0} 2023-10-18T09:15:41.045 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T09:15:41.791 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:15:41.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:41 smithi170 ceph-mon[135150]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:41.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:41 smithi110 ceph-mon[134818]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:42.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:41 smithi061 ceph-mon[98573]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:43.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 -- 172.21.15.61:0/914759234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f602810d250 msgr2=0x7f602810f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:43.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 --2- 172.21.15.61:0/914759234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f602810d250 0x7f602810f640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f602000b0a0 tx=0x7f602000b3b0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 -- 172.21.15.61:0/914759234 shutdown_connections 2023-10-18T09:15:43.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 --2- 172.21.15.61:0/914759234 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f602810d250 0x7f602810f640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 --2- 172.21.15.61:0/914759234 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6028072a20 0x7f6028072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 --2- 172.21.15.61:0/914759234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 0x7f602810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 -- 172.21.15.61:0/914759234 >> 172.21.15.61:0/914759234 conn(0x7f60280779e0 msgr2=0x7f6028077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:43.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 -- 172.21.15.61:0/914759234 shutdown_connections 2023-10-18T09:15:43.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.407+0000 7f602e5a2700 1 -- 172.21.15.61:0/914759234 wait complete. 2023-10-18T09:15:43.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.408+0000 7f602e5a2700 1 Processor -- start 2023-10-18T09:15:43.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.408+0000 7f602e5a2700 1 -- start start 2023-10-18T09:15:43.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:43.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 0x7f60281995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:43.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f602810d250 0x7f602819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:43.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602819df90 con 0x7f6028072a20 2023-10-18T09:15:43.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602819e100 con 0x7f602810d250 2023-10-18T09:15:43.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602e5a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602819e2c0 con 0x7f6028073340 2023-10-18T09:15:43.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:43.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56166/0 (socket says 172.21.15.61:56166) 2023-10-18T09:15:43.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 -- 172.21.15.61:0/571405457 learned_addr learned my addr 172.21.15.61:0/571405457 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:43.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f60277fe700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 0x7f60281995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:43.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f602cb3f700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f602810d250 0x7f602819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:43.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 -- 172.21.15.61:0/571405457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 msgr2=0x7f60281995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:43.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 0x7f60281995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.409+0000 7f6027fff700 1 -- 172.21.15.61:0/571405457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f602810d250 msgr2=0x7f602819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f6027fff700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f602810d250 0x7f602819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f6027fff700 1 -- 172.21.15.61:0/571405457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009d00 con 0x7f6028072a20 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f6027fff700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f601800e9a0 tx=0x7f601800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018009790 con 0x7f6028072a20 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602819e5a0 con 0x7f6028072a20 2023-10-18T09:15:43.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6018010430 con 0x7f6028072a20 2023-10-18T09:15:43.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6018018680 con 0x7f6028072a20 2023-10-18T09:15:43.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.410+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60281a5e00 con 0x7f6028072a20 2023-10-18T09:15:43.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.411+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6028108460 con 0x7f6028072a20 2023-10-18T09:15:43.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.413+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f60180105a0 con 0x7f6028072a20 2023-10-18T09:15:43.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.415+0000 7f60257fa700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 0x7f60100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:43.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.415+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6018014070 con 0x7f6028072a20 2023-10-18T09:15:43.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.415+0000 7f60277fe700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 0x7f60100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:43.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.415+0000 7f60277fe700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 0x7f60100a3940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f602819a5f0 tx=0x7f601c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:43.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.419+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6018052100 con 0x7f6028072a20 2023-10-18T09:15:43.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.633+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60280740b0 con 0x7f6028072a20 2023-10-18T09:15:43.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.634+0000 7f60257fa700 1 -- 172.21.15.61:0/571405457 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f6018096510 con 0x7f6028072a20 2023-10-18T09:15:43.637 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:43.637 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":80,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:15:37.907737+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:15:43.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 msgr2=0x7f60100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:43.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 0x7f60100a3940 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f602819a5f0 tx=0x7f601c009260 comp rx=0 tx=0).stop 2023-10-18T09:15:43.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 msgr2=0x7f6028199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:43.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f601800e9a0 tx=0x7f601800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 shutdown_connections 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f60100a1480 0x7f60100a3940 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f602810d250 0x7f602819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6028072a20 0x7f6028199090 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 --2- 172.21.15.61:0/571405457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6028073340 0x7f60281995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:43.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 >> 172.21.15.61:0/571405457 conn(0x7f60280779e0 msgr2=0x7f60280feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:43.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 shutdown_connections 2023-10-18T09:15:43.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:43.636+0000 7f602e5a2700 1 -- 172.21.15.61:0/571405457 wait complete. 2023-10-18T09:15:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:43 smithi061 ceph-mon[98573]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/571405457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:43 smithi170 ceph-mon[135150]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/571405457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:44.184 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:15:44.184 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-18T09:15:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:43 smithi110 ceph-mon[134818]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/571405457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:44.928 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:15:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:45 smithi170 ceph-mon[135150]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:46.185 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:45 smithi061 ceph-mon[98573]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:45 smithi110 ceph-mon[134818]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:46.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 -- 172.21.15.61:0/129674949 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 msgr2=0x7f67ac100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:46.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 --2- 172.21.15.61:0/129674949 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac100d20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009990 tx=0x7f67a0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 -- 172.21.15.61:0/129674949 shutdown_connections 2023-10-18T09:15:46.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 --2- 172.21.15.61:0/129674949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac101260 0x7f67ac10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 --2- 172.21.15.61:0/129674949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac1057c0 0x7f67ac105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 --2- 172.21.15.61:0/129674949 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac100d20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 -- 172.21.15.61:0/129674949 >> 172.21.15.61:0/129674949 conn(0x7f67ac0779e0 msgr2=0x7f67ac077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:46.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 -- 172.21.15.61:0/129674949 shutdown_connections 2023-10-18T09:15:46.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.439+0000 7f67b24b4700 1 -- 172.21.15.61:0/129674949 wait complete. 2023-10-18T09:15:46.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.440+0000 7f67b24b4700 1 Processor -- start 2023-10-18T09:15:46.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 -- start start 2023-10-18T09:15:46.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:46.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac101260 0x7f67ac073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:46.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac1057c0 0x7f67ac073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:46.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1a4f90 con 0x7f67ac1008a0 2023-10-18T09:15:46.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1a5100 con 0x7f67ac101260 2023-10-18T09:15:46.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.441+0000 7f67b24b4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1a52c0 con 0x7f67ac1057c0 2023-10-18T09:15:46.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:46.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56178/0 (socket says 172.21.15.61:56178) 2023-10-18T09:15:46.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 -- 172.21.15.61:0/4003262103 learned_addr learned my addr 172.21.15.61:0/4003262103 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:46.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67ab7fe700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac101260 0x7f67ac073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:46.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67b0a51700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac1057c0 0x7f67ac073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:46.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 -- 172.21.15.61:0/4003262103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac1057c0 msgr2=0x7f67ac073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:46.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac1057c0 0x7f67ac073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 -- 172.21.15.61:0/4003262103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac101260 msgr2=0x7f67ac073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:46.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac101260 0x7f67ac073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 -- 172.21.15.61:0/4003262103 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00095f0 con 0x7f67ac1008a0 2023-10-18T09:15:46.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.442+0000 7f67abfff700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f679c00d980 tx=0x7f679c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:46.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.443+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00b950 con 0x7f67ac1008a0 2023-10-18T09:15:46.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.443+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f679c007620 con 0x7f67ac1008a0 2023-10-18T09:15:46.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.443+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c00f3c0 con 0x7f67ac1008a0 2023-10-18T09:15:46.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.443+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac1a55a0 con 0x7f67ac1008a0 2023-10-18T09:15:46.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.443+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac1a5a50 con 0x7f67ac1008a0 2023-10-18T09:15:46.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.444+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac108da0 con 0x7f67ac1008a0 2023-10-18T09:15:46.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.446+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f679c010640 con 0x7f67ac1008a0 2023-10-18T09:15:46.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.447+0000 7f67a97fa700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 0x7f67940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:46.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.447+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f679c0cc620 con 0x7f67ac1008a0 2023-10-18T09:15:46.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.447+0000 7f67ab7fe700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 0x7f67940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:46.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.448+0000 7f67ab7fe700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 0x7f67940a39c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0741d0 tx=0x7f67a0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:46.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.451+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f679c096080 con 0x7f67ac1008a0 2023-10-18T09:15:46.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.661+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67ac074020 con 0x7f67ac1008a0 2023-10-18T09:15:46.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.662+0000 7f67a97fa700 1 -- 172.21.15.61:0/4003262103 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f679c0996a0 con 0x7f67ac1008a0 2023-10-18T09:15:46.666 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:15:46.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 msgr2=0x7f67940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:46.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 0x7f67940a39c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0741d0 tx=0x7f67a0005fd0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 msgr2=0x7f67ac072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:46.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f679c00d980 tx=0x7f679c00dc90 comp rx=0 tx=0).stop 2023-10-18T09:15:46.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 shutdown_connections 2023-10-18T09:15:46.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67940a1500 0x7f67940a39c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67ac1057c0 0x7f67ac073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67ac1008a0 0x7f67ac072af0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 --2- 172.21.15.61:0/4003262103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67ac101260 0x7f67ac073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:46.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.664+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 >> 172.21.15.61:0/4003262103 conn(0x7f67ac0779e0 msgr2=0x7f67ac0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:46.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.665+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 shutdown_connections 2023-10-18T09:15:46.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:46.665+0000 7f67b24b4700 1 -- 172.21.15.61:0/4003262103 wait complete. 2023-10-18T09:15:47.216 INFO:tasks.cephadm:Adding mds.a on smithi061 2023-10-18T09:15:47.216 INFO:tasks.cephadm:Adding mds.d on smithi061 2023-10-18T09:15:47.217 INFO:tasks.cephadm:Adding mds.g on smithi061 2023-10-18T09:15:47.217 INFO:tasks.cephadm:Adding mds.j on smithi061 2023-10-18T09:15:47.217 INFO:tasks.cephadm:Adding mds.b on smithi110 2023-10-18T09:15:47.217 INFO:tasks.cephadm:Adding mds.e on smithi110 2023-10-18T09:15:47.218 INFO:tasks.cephadm:Adding mds.h on smithi110 2023-10-18T09:15:47.218 INFO:tasks.cephadm:Adding mds.k on smithi110 2023-10-18T09:15:47.218 INFO:tasks.cephadm:Adding mds.c on smithi170 2023-10-18T09:15:47.218 INFO:tasks.cephadm:Adding mds.f on smithi170 2023-10-18T09:15:47.219 INFO:tasks.cephadm:Adding mds.i on smithi170 2023-10-18T09:15:47.219 INFO:tasks.cephadm:Adding mds.l on smithi170 2023-10-18T09:15:47.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph orch apply mds all '12;smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2023-10-18T09:15:47.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:47 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4003262103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:47.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:47 smithi170 ceph-mon[135150]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:47 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4003262103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:47.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:47 smithi061 ceph-mon[98573]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:47.792 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:15:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:47 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4003262103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:47 smithi110 ceph-mon[134818]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:49.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 -- 172.21.15.170:0/1583403708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb758100930 msgr2=0x7fb758100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 --2- 172.21.15.170:0/1583403708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb758100930 0x7fb758100d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb754009990 tx=0x7fb754009ca0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 -- 172.21.15.170:0/1583403708 shutdown_connections 2023-10-18T09:15:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 --2- 172.21.15.170:0/1583403708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb7581012d0 0x7fb758107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 --2- 172.21.15.170:0/1583403708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 0x7fb7580ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 --2- 172.21.15.170:0/1583403708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb758100930 0x7fb758100d90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 -- 172.21.15.170:0/1583403708 >> 172.21.15.170:0/1583403708 conn(0x7fb7580facf0 msgr2=0x7fb7580fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 -- 172.21.15.170:0/1583403708 shutdown_connections 2023-10-18T09:15:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 -- 172.21.15.170:0/1583403708 wait complete. 2023-10-18T09:15:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.015+0000 7fb760406700 1 Processor -- start 2023-10-18T09:15:49.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 -- start start 2023-10-18T09:15:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 0x7fb758068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb758100930 0x7fb75819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75819e240 con 0x7fb758100930 2023-10-18T09:15:49.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75819e3b0 con 0x7fb7580ff730 2023-10-18T09:15:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb760406700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75819e570 con 0x7fb7581012d0 2023-10-18T09:15:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54700/0 (socket says 172.21.15.170:54700) 2023-10-18T09:15:49.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 -- 172.21.15.170:0/1415005015 learned_addr learned my addr 172.21.15.170:0/1415005015 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e1a2700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 0x7fb758068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75d9a1700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb758100930 0x7fb75819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 -- 172.21.15.170:0/1415005015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 msgr2=0x7fb758068c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 0x7fb758068c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 -- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb758100930 msgr2=0x7fb75819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:49.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb758100930 0x7fb75819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 -- 172.21.15.170:0/1415005015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540095f0 con 0x7fb7581012d0 2023-10-18T09:15:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb75e9a3700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75000ada0 tx=0x7fb75000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.016+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75000f070 con 0x7fb7581012d0 2023-10-18T09:15:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.017+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb750007b30 con 0x7fb7581012d0 2023-10-18T09:15:49.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.017+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75819e850 con 0x7fb7581012d0 2023-10-18T09:15:49.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.017+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7500053e0 con 0x7fb7581012d0 2023-10-18T09:15:49.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.017+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75819ed00 con 0x7fb7581012d0 2023-10-18T09:15:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.017+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb758106c30 con 0x7fb7581012d0 2023-10-18T09:15:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.019+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb75000c980 con 0x7fb7581012d0 2023-10-18T09:15:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.019+0000 7fb74f7fe700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.019+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7500cc4a0 con 0x7fb7581012d0 2023-10-18T09:15:49.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.020+0000 7fb75e1a2700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:49.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.020+0000 7fb75e1a2700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb758100790 tx=0x7fb74800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:49.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.023+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb750096030 con 0x7fb7581012d0 2023-10-18T09:15:49.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.221+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fb75802d080 con 0x7fb7440a1480 2023-10-18T09:15:49.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.230+0000 7fb74f7fe700 1 -- 172.21.15.170:0/1415005015 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fb75802d080 con 0x7fb7440a1480 2023-10-18T09:15:49.232 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2023-10-18T09:15:49.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 msgr2=0x7fb7440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:49.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb758100790 tx=0x7fb74800b410 comp rx=0 tx=0).stop 2023-10-18T09:15:49.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 msgr2=0x7fb75819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:49.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb75000ada0 tx=0x7fb75000c330 comp rx=0 tx=0).stop 2023-10-18T09:15:49.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 shutdown_connections 2023-10-18T09:15:49.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7581012d0 0x7fb75819daa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7580ff730 0x7fb758068c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 --2- 172.21.15.170:0/1415005015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb758100930 0x7fb75819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:49.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 >> 172.21.15.170:0/1415005015 conn(0x7fb7580facf0 msgr2=0x7fb758103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:49.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 shutdown_connections 2023-10-18T09:15:49.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-18T09:15:49.232+0000 7fb760406700 1 -- 172.21.15.170:0/1415005015 wait complete. 2023-10-18T09:15:49.646 DEBUG:teuthology.orchestra.run.smithi061:mds.a> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.a.service 2023-10-18T09:15:49.652 DEBUG:teuthology.orchestra.run.smithi061:mds.d> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.d.service 2023-10-18T09:15:49.698 DEBUG:teuthology.orchestra.run.smithi061:mds.g> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.g.service 2023-10-18T09:15:49.735 INFO:journalctl@ceph.mds.a.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:15:49.738 DEBUG:teuthology.orchestra.run.smithi061:mds.j> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.j.service 2023-10-18T09:15:49.773 INFO:journalctl@ceph.mds.d.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:15:49.777 DEBUG:teuthology.orchestra.run.smithi110:mds.b> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.b.service 2023-10-18T09:15:49.779 DEBUG:teuthology.orchestra.run.smithi110:mds.e> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.e.service 2023-10-18T09:15:49.808 INFO:journalctl@ceph.mds.b.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:15:49.809 DEBUG:teuthology.orchestra.run.smithi110:mds.h> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.h.service 2023-10-18T09:15:49.812 INFO:journalctl@ceph.mds.g.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:15:49.835 INFO:journalctl@ceph.mds.e.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:15:49.836 DEBUG:teuthology.orchestra.run.smithi110:mds.k> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.k.service 2023-10-18T09:15:49.848 INFO:journalctl@ceph.mds.j.smithi061.stdout:-- Logs begin at Wed 2023-10-18 08:51:09 UTC. -- 2023-10-18T09:15:49.861 INFO:journalctl@ceph.mds.h.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:15:49.862 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.c.service 2023-10-18T09:15:49.865 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.f.service 2023-10-18T09:15:49.888 INFO:journalctl@ceph.mds.k.smithi110.stdout:-- Logs begin at Wed 2023-10-18 08:47:05 UTC. -- 2023-10-18T09:15:49.896 INFO:journalctl@ceph.mds.c.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:15:49.897 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.i.service 2023-10-18T09:15:49.923 INFO:journalctl@ceph.mds.f.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:15:49.924 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.l.service 2023-10-18T09:15:49.950 INFO:journalctl@ceph.mds.i.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:15:49.951 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-18T09:15:49.951 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:15:49.976 INFO:journalctl@ceph.mds.l.smithi170.stdout:-- Logs begin at Wed 2023-10-18 08:47:07 UTC. -- 2023-10-18T09:15:50.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:50.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:50.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:50.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:50.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:50.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 -- 172.21.15.61:0/2641923637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83801022d0 msgr2=0x7f8380108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 --2- 172.21.15.61:0/2641923637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83801022d0 0x7f8380108b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f837800b0d0 tx=0x7f837800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 -- 172.21.15.61:0/2641923637 shutdown_connections 2023-10-18T09:15:50.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 --2- 172.21.15.61:0/2641923637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f83801022d0 0x7f8380108b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 --2- 172.21.15.61:0/2641923637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8380100740 0x7f8380100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 --2- 172.21.15.61:0/2641923637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380101930 0x7f8380101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 -- 172.21.15.61:0/2641923637 >> 172.21.15.61:0/2641923637 conn(0x7f83800fbc90 msgr2=0x7f83800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:50.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 -- 172.21.15.61:0/2641923637 shutdown_connections 2023-10-18T09:15:50.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.162+0000 7f83849ff700 1 -- 172.21.15.61:0/2641923637 wait complete. 2023-10-18T09:15:50.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 Processor -- start 2023-10-18T09:15:50.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 -- start start 2023-10-18T09:15:50.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 0x7f838019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83801a04f0 con 0x7f8380101930 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83801a0630 con 0x7f83801022d0 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.163+0000 7f83849ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83801a07c0 con 0x7f8380100740 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837dd9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 0x7f838019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:38876/0 (socket says 172.21.15.61:38876) 2023-10-18T09:15:50.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 -- 172.21.15.61:0/846120990 learned_addr learned my addr 172.21.15.61:0/846120990 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:50.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837dd9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36324/0 (socket says 172.21.15.61:36324) 2023-10-18T09:15:50.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 -- 172.21.15.61:0/846120990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 msgr2=0x7f838019ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 0x7f838019ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 -- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 msgr2=0x7f8380198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 -- 172.21.15.61:0/846120990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8378009d00 con 0x7f8380100740 2023-10-18T09:15:50.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837ed9d700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 0x7f838019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:50.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837dd9b700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:50.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.164+0000 7f837e59c700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f837000e970 tx=0x7f837000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:50.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.165+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370009790 con 0x7f8380100740 2023-10-18T09:15:50.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.165+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83801a0aa0 con 0x7f8380100740 2023-10-18T09:15:50.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.165+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f837000c7e0 con 0x7f8380100740 2023-10-18T09:15:50.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.165+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370003c70 con 0x7f8380100740 2023-10-18T09:15:50.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.165+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83801a1000 con 0x7f8380100740 2023-10-18T09:15:50.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.166+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8380067530 con 0x7f8380100740 2023-10-18T09:15:50.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.168+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f837000cc50 con 0x7f8380100740 2023-10-18T09:15:50.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.169+0000 7f836f7fe700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 0x7f83680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.169+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8370014070 con 0x7f8380100740 2023-10-18T09:15:50.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.169+0000 7f837dd9b700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 0x7f83680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.170+0000 7f837dd9b700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 0x7f83680a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8374011490 tx=0x7f8374009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:50.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.171+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f83700968f0 con 0x7f8380100740 2023-10-18T09:15:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:15:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:15:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:50.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:50.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:50.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.389+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f838010bfb0 con 0x7f8380100740 2023-10-18T09:15:50.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.391+0000 7f836f7fe700 1 -- 172.21.15.61:0/846120990 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f8370003dd0 con 0x7f8380100740 2023-10-18T09:15:50.395 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:50.395 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":80,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:15:37.907737+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:15:50.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 msgr2=0x7f83680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 0x7f83680a3940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8374011490 tx=0x7f8374009260 comp rx=0 tx=0).stop 2023-10-18T09:15:50.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 msgr2=0x7f83801989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f837000e970 tx=0x7f837000ec80 comp rx=0 tx=0).stop 2023-10-18T09:15:50.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 shutdown_connections 2023-10-18T09:15:50.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f83680a1480 0x7f83680a3940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83801022d0 0x7f838019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8380100740 0x7f83801989c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 --2- 172.21.15.61:0/846120990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8380101930 0x7f8380198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 >> 172.21.15.61:0/846120990 conn(0x7f83800fbc90 msgr2=0x7f8380072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:50.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 shutdown_connections 2023-10-18T09:15:50.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.393+0000 7f83849ff700 1 -- 172.21.15.61:0/846120990 wait complete. 2023-10-18T09:15:50.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:15:50.418 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:15:50.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 -- 172.21.15.61:0/2162277840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb134101990 msgr2=0x7fb134101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/2162277840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb134101990 0x7fb134101df0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb124009990 tx=0x7fb124009ca0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 -- 172.21.15.61:0/2162277840 shutdown_connections 2023-10-18T09:15:50.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/2162277840 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/2162277840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1341007a0 0x7fb134100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/2162277840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb134101990 0x7fb134101df0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.622+0000 7fb13b9ff700 1 -- 172.21.15.61:0/2162277840 >> 172.21.15.61:0/2162277840 conn(0x7fb1340fbcf0 msgr2=0x7fb1340fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:50.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.623+0000 7fb13b9ff700 1 -- 172.21.15.61:0/2162277840 shutdown_connections 2023-10-18T09:15:50.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.623+0000 7fb13b9ff700 1 -- 172.21.15.61:0/2162277840 wait complete. 2023-10-18T09:15:50.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.623+0000 7fb13b9ff700 1 Processor -- start 2023-10-18T09:15:50.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 -- start start 2023-10-18T09:15:50.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb134101990 0x7fb134072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340738c0 con 0x7fb134102330 2023-10-18T09:15:50.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134073a30 con 0x7fb134101990 2023-10-18T09:15:50.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13b9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1341a0790 con 0x7fb1341007a0 2023-10-18T09:15:50.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb139f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:38898/0 (socket says 172.21.15.61:38898) 2023-10-18T09:15:50.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb13979b700 1 -- 172.21.15.61:0/295813797 learned_addr learned my addr 172.21.15.61:0/295813797 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:50.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.624+0000 7fb138f9a700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb134101990 0x7fb134072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 -- 172.21.15.61:0/295813797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb134101990 msgr2=0x7fb134072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb134101990 0x7fb134072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 -- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 msgr2=0x7fb134073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 -- 172.21.15.61:0/295813797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240095f0 con 0x7fb1341007a0 2023-10-18T09:15:50.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb139f9c700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:50.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13979b700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb13000e970 tx=0x7fb13000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:50.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130009790 con 0x7fb1341007a0 2023-10-18T09:15:50.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1341a09c0 con 0x7fb1341007a0 2023-10-18T09:15:50.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb13000c7e0 con 0x7fb1341007a0 2023-10-18T09:15:50.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130003c70 con 0x7fb1341007a0 2023-10-18T09:15:50.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.625+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1341a0f20 con 0x7fb1341007a0 2023-10-18T09:15:50.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.626+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb134107c70 con 0x7fb1341007a0 2023-10-18T09:15:50.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.628+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb13000cc50 con 0x7fb1341007a0 2023-10-18T09:15:50.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.630+0000 7fb12a7fc700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:50.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.630+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb130014070 con 0x7fb1341007a0 2023-10-18T09:15:50.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.630+0000 7fb138f9a700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:50.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.630+0000 7fb138f9a700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 0x7fb1200a39c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb12400b2e0 tx=0x7fb124005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:50.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.634+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb130096510 con 0x7fb1341007a0 2023-10-18T09:15:50.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.846+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb134107650 con 0x7fb1341007a0 2023-10-18T09:15:50.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.846+0000 7fb12a7fc700 1 -- 172.21.15.61:0/295813797 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb130099b30 con 0x7fb1341007a0 2023-10-18T09:15:50.850 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:15:50.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 msgr2=0x7fb1200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 0x7fb1200a39c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb12400b2e0 tx=0x7fb124005fb0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 msgr2=0x7fb134074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:50.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb13000e970 tx=0x7fb13000ec80 comp rx=0 tx=0).stop 2023-10-18T09:15:50.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 shutdown_connections 2023-10-18T09:15:50.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1200a1500 0x7fb1200a39c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb134102330 0x7fb134073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1341007a0 0x7fb134074550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 --2- 172.21.15.61:0/295813797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb134101990 0x7fb134072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:50.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 >> 172.21.15.61:0/295813797 conn(0x7fb1340fbcf0 msgr2=0x7fb134104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:50.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 shutdown_connections 2023-10-18T09:15:50.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:50.849+0000 7fb13b9ff700 1 -- 172.21.15.61:0/295813797 wait complete. 2023-10-18T09:15:50.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:15:51.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 -- 172.21.15.61:0/2173320328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c1022d0 msgr2=0x7f8b0c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:51.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 --2- 172.21.15.61:0/2173320328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c108b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b0d0 tx=0x7f8b0400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 -- 172.21.15.61:0/2173320328 shutdown_connections 2023-10-18T09:15:51.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 --2- 172.21.15.61:0/2173320328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c108b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 --2- 172.21.15.61:0/2173320328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c100740 0x7f8b0c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 --2- 172.21.15.61:0/2173320328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 0x7f8b0c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 -- 172.21.15.61:0/2173320328 >> 172.21.15.61:0/2173320328 conn(0x7f8b0c0fbc90 msgr2=0x7f8b0c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:51.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 -- 172.21.15.61:0/2173320328 shutdown_connections 2023-10-18T09:15:51.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.073+0000 7f8b13dff700 1 -- 172.21.15.61:0/2173320328 wait complete. 2023-10-18T09:15:51.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.074+0000 7f8b13dff700 1 Processor -- start 2023-10-18T09:15:51.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 -- start start 2023-10-18T09:15:51.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:51.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 0x7f8b0c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:51.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c19ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:51.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a0510 con 0x7f8b0c100740 2023-10-18T09:15:51.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a0650 con 0x7f8b0c101930 2023-10-18T09:15:51.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b13dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a07e0 con 0x7f8b0c1022d0 2023-10-18T09:15:51.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b11b9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:51.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b11b9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36344/0 (socket says 172.21.15.61:36344) 2023-10-18T09:15:51.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b11b9b700 1 -- 172.21.15.61:0/820806987 learned_addr learned my addr 172.21.15.61:0/820806987 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:51.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b1139a700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 0x7f8b0c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:51.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.075+0000 7f8b1239c700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c19ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:51.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 -- 172.21.15.61:0/820806987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1022d0 msgr2=0x7f8b0c19ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:51.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c19ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 -- 172.21.15.61:0/820806987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 msgr2=0x7f8b0c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:51.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 0x7f8b0c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 -- 172.21.15.61:0/820806987 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009d00 con 0x7f8b0c100740 2023-10-18T09:15:51.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b11b9b700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800e970 tx=0x7f8b0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:51.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08009790 con 0x7f8b0c100740 2023-10-18T09:15:51.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b0800c7e0 con 0x7f8b0c100740 2023-10-18T09:15:51.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c1a0ac0 con 0x7f8b0c100740 2023-10-18T09:15:51.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08003c70 con 0x7f8b0c100740 2023-10-18T09:15:51.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.076+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c1a0fd0 con 0x7f8b0c100740 2023-10-18T09:15:51.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.077+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c067530 con 0x7f8b0c100740 2023-10-18T09:15:51.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.079+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8b0800cc50 con 0x7f8b0c100740 2023-10-18T09:15:51.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.081+0000 7f8b02ffd700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 0x7f8af80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:51.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.081+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b08014070 con 0x7f8b0c100740 2023-10-18T09:15:51.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.081+0000 7f8b1139a700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 0x7f8af80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:51.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.081+0000 7f8b1139a700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 0x7f8af80a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009c60 tx=0x7f8afc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:51.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.085+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8b08096550 con 0x7f8b0c100740 2023-10-18T09:15:51.110 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:51.110 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: Saving service mds.all spec with placement smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-10-18T09:15:51.110 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:51.111 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: Deploying daemon mds.l on smithi170 2023-10-18T09:15:51.111 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/846120990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:51.111 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:50 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/295813797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: Saving service mds.all spec with placement smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-10-18T09:15:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:51.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: Deploying daemon mds.l on smithi170 2023-10-18T09:15:51.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/846120990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:51.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:50 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/295813797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: from='client.24565 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:15:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: Saving service mds.all spec with placement smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-10-18T09:15:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:51.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: Deploying daemon mds.l on smithi170 2023-10-18T09:15:51.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/846120990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:15:51.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:50 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/295813797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:15:51.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.346+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b0c10bfb0 con 0x7f8b0c100740 2023-10-18T09:15:51.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.346+0000 7f8b02ffd700 1 -- 172.21.15.61:0/820806987 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f8b08003dd0 con 0x7f8b0c100740 2023-10-18T09:15:51.350 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:51.351 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-18T09:15:51.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 msgr2=0x7f8af80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:51.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 0x7f8af80a3940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009c60 tx=0x7f8afc009260 comp rx=0 tx=0).stop 2023-10-18T09:15:51.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 msgr2=0x7f8b0c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:51.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800e970 tx=0x7f8b0800ec80 comp rx=0 tx=0).stop 2023-10-18T09:15:51.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 shutdown_connections 2023-10-18T09:15:51.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8af80a1480 0x7f8af80a3940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c1022d0 0x7f8b0c19ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8b0c100740 0x7f8b0c1989e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 --2- 172.21.15.61:0/820806987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b0c101930 0x7f8b0c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:51.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 >> 172.21.15.61:0/820806987 conn(0x7f8b0c0fbc90 msgr2=0x7f8b0c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:51.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 shutdown_connections 2023-10-18T09:15:51.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:51.349+0000 7f8b13dff700 1 -- 172.21.15.61:0/820806987 wait complete. 2023-10-18T09:15:51.355 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 1 2023-10-18T09:15:51.392 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 systemd[1]: Starting Ceph mds.l for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:15:51.886 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 podman[172582]: 2023-10-18T09:15:52.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:51 smithi170 ceph-mon[135150]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:52.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:51 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/820806987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:52.143 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 ceph-mds[172622]: -- 172.21.15.170:0/649141376 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563c6bd711e0 con 0x563c6bd46800 2023-10-18T09:15:52.143 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: starting mds.l at 2023-10-18T09:15:52.143 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 bash[172582]: e0e11fe610bb095d2f9826070ec370604e4010e2acc005f43d0f1751f3b720eb 2023-10-18T09:15:52.143 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:15:51 smithi170 systemd[1]: Started Ceph mds.l for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:15:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:51 smithi110 ceph-mon[134818]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:51 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/820806987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:51 smithi061 ceph-mon[98573]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:52.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:51 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/820806987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:53.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6832/2686391817,v1:172.21.15.170:6833/2686391817] up:boot 2023-10-18T09:15:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: fsmap 1 up:standby 2023-10-18T09:15:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:15:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:53.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:53.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:53.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: Deploying daemon mds.k on smithi110 2023-10-18T09:15:53.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:52 smithi110 ceph-mon[134818]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:53.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6832/2686391817,v1:172.21.15.170:6833/2686391817] up:boot 2023-10-18T09:15:53.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: fsmap 1 up:standby 2023-10-18T09:15:53.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:15:53.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:53.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:53.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:53.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: Deploying daemon mds.k on smithi110 2023-10-18T09:15:53.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:52 smithi061 ceph-mon[98573]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:53.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:15:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6832/2686391817,v1:172.21.15.170:6833/2686391817] up:boot 2023-10-18T09:15:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: fsmap 1 up:standby 2023-10-18T09:15:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:15:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:53.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:53.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:53.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: Deploying daemon mds.k on smithi110 2023-10-18T09:15:53.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:52 smithi170 ceph-mon[135150]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:53.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 -- 172.21.15.61:0/3780658403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 msgr2=0x7fc080072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:53.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 --2- 172.21.15.61:0/3780658403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080072c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc070009990 tx=0x7fc070009ca0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 -- 172.21.15.61:0/3780658403 shutdown_connections 2023-10-18T09:15:53.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 --2- 172.21.15.61:0/3780658403 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08010a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 --2- 172.21.15.61:0/3780658403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080072c80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 --2- 172.21.15.61:0/3780658403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 0x7fc0800736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 -- 172.21.15.61:0/3780658403 >> 172.21.15.61:0/3780658403 conn(0x7fc0800fba70 msgr2=0x7fc0800fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:53.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.584+0000 7fc08559d700 1 -- 172.21.15.61:0/3780658403 shutdown_connections 2023-10-18T09:15:53.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.585+0000 7fc08559d700 1 -- 172.21.15.61:0/3780658403 wait complete. 2023-10-18T09:15:53.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.585+0000 7fc08559d700 1 Processor -- start 2023-10-18T09:15:53.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 -- start start 2023-10-18T09:15:53.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 0x7fc0801986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:53.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:53.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:53.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801a01e0 con 0x7fc0801083e0 2023-10-18T09:15:53.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801a0320 con 0x7fc080074830 2023-10-18T09:15:53.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc08559d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801a04b0 con 0x7fc080073250 2023-10-18T09:15:53.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:53.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39208/0 (socket says 172.21.15.61:39208) 2023-10-18T09:15:53.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc084d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:53.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 -- 172.21.15.61:0/803362777 learned_addr learned my addr 172.21.15.61:0/803362777 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:53.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07ffff700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 0x7fc0801986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:53.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 -- 172.21.15.61:0/803362777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 msgr2=0x7fc0801986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:53.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 0x7fc0801986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.586+0000 7fc07f7fe700 1 -- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 msgr2=0x7fc08019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:53.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.587+0000 7fc07f7fe700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.587+0000 7fc07f7fe700 1 -- 172.21.15.61:0/803362777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009c40 con 0x7fc080074830 2023-10-18T09:15:53.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.587+0000 7fc084d9c700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:53.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.588+0000 7fc07f7fe700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d980 tx=0x7fc07400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:53.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.588+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400de60 con 0x7fc080074830 2023-10-18T09:15:53.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.588+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0740044e0 con 0x7fc080074830 2023-10-18T09:15:53.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.588+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0700095f0 con 0x7fc080074830 2023-10-18T09:15:53.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.588+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074003a50 con 0x7fc080074830 2023-10-18T09:15:53.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.589+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0801a0b60 con 0x7fc080074830 2023-10-18T09:15:53.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.589+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0801071a0 con 0x7fc080074830 2023-10-18T09:15:53.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.592+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc074009ad0 con 0x7fc080074830 2023-10-18T09:15:53.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.593+0000 7fc07d7fa700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 0x7fc0680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:53.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.593+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0740cc900 con 0x7fc080074830 2023-10-18T09:15:53.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.593+0000 7fc07ffff700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 0x7fc0680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:53.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.594+0000 7fc07ffff700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 0x7fc0680a39c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b7b0 tx=0x7fc070005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:53.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.595+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0740963e0 con 0x7fc080074830 2023-10-18T09:15:53.712 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:53 smithi110 systemd[1]: Starting Ceph mds.k for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:15:53.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.854+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0800673e0 con 0x7fc080074830 2023-10-18T09:15:53.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.855+0000 7fc07d7fa700 1 -- 172.21.15.61:0/803362777 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fc074099a00 con 0x7fc080074830 2023-10-18T09:15:53.859 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:53.859 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-18T09:15:53.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 msgr2=0x7fc0680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:53.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 0x7fc0680a39c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b7b0 tx=0x7fc070005fd0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 msgr2=0x7fc080198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:53.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d980 tx=0x7fc07400dd40 comp rx=0 tx=0).stop 2023-10-18T09:15:53.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 shutdown_connections 2023-10-18T09:15:53.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc0680a1500 0x7fc0680a39c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc0801083e0 0x7fc08019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc080073250 0x7fc0801986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 --2- 172.21.15.61:0/803362777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc080074830 0x7fc080198bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:53.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 >> 172.21.15.61:0/803362777 conn(0x7fc0800fba70 msgr2=0x7fc0801042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:53.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 shutdown_connections 2023-10-18T09:15:53.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:53.858+0000 7fc08559d700 1 -- 172.21.15.61:0/803362777 wait complete. 2023-10-18T09:15:53.864 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 2 2023-10-18T09:15:54.019 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:53 smithi110 podman[170756]: 2023-10-18T09:15:54.019 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:54 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/803362777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:54 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/803362777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:54 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/803362777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:54.462 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:54 smithi110 ceph-mds[170796]: -- 172.21.15.110:0/1204834437 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d5b5ff1e0 con 0x556d5b5d4c00 2023-10-18T09:15:54.462 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:54 smithi110 bash[170756]: 7089d5bedc0109bf19a59f448e0300a8aa7019a602888436ac6f7dd661d0010e 2023-10-18T09:15:54.462 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: starting mds.k at 2023-10-18T09:15:54.463 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:15:54 smithi110 systemd[1]: Started Ceph mds.k for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:15:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: Deploying daemon mds.j on smithi061 2023-10-18T09:15:55.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:55.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6832/3590204423,v1:172.21.15.110:6833/3590204423] up:boot 2023-10-18T09:15:55.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: fsmap 2 up:standby 2023-10-18T09:15:55.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:15:55.415 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.415 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.415 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.415 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:55.415 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: Deploying daemon mds.j on smithi061 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6832/3590204423,v1:172.21.15.110:6833/3590204423] up:boot 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: fsmap 2 up:standby 2023-10-18T09:15:55.416 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:15:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: Deploying daemon mds.j on smithi061 2023-10-18T09:15:55.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:55.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6832/3590204423,v1:172.21.15.110:6833/3590204423] up:boot 2023-10-18T09:15:55.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: fsmap 2 up:standby 2023-10-18T09:15:55.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:15:55.881 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:15:56.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.307+0000 7fd6a4fff700 1 -- 172.21.15.61:0/2250811938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 msgr2=0x7fd6a010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:56.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.307+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/2250811938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a010aa80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd698009f90 tx=0x7fd69800b330 comp rx=0 tx=0).stop 2023-10-18T09:15:56.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 -- 172.21.15.61:0/2250811938 shutdown_connections 2023-10-18T09:15:56.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/2250811938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a010aa80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/2250811938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00747d0 0x7fd6a0072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/2250811938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00731f0 0x7fd6a0073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 -- 172.21.15.61:0/2250811938 >> 172.21.15.61:0/2250811938 conn(0x7fd6a00fbc90 msgr2=0x7fd6a00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:56.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 -- 172.21.15.61:0/2250811938 shutdown_connections 2023-10-18T09:15:56.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.308+0000 7fd6a4fff700 1 -- 172.21.15.61:0/2250811938 wait complete. 2023-10-18T09:15:56.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 Processor -- start 2023-10-18T09:15:56.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 -- start start 2023-10-18T09:15:56.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:56.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00747d0 0x7fd6a0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:56.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a019ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:56.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01a04e0 con 0x7fd6a00731f0 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01a0620 con 0x7fd6a0108610 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.309+0000 7fd6a4fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01a07b0 con 0x7fd6a00747d0 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36388/0 (socket says 172.21.15.61:36388) 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 -- 172.21.15.61:0/218413072 learned_addr learned my addr 172.21.15.61:0/218413072 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69ed9d700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a019ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69dd9b700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00747d0 0x7fd6a0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 -- 172.21.15.61:0/218413072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00747d0 msgr2=0x7fd6a0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00747d0 0x7fd6a0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 -- 172.21.15.61:0/218413072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 msgr2=0x7fd6a019ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a019ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 -- 172.21.15.61:0/218413072 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900095f0 con 0x7fd6a00731f0 2023-10-18T09:15:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69ed9d700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.310+0000 7fd69e59c700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd69000ec10 tx=0x7fd69000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.311+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000cc90 con 0x7fd6a00731f0 2023-10-18T09:15:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.311+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6900044e0 con 0x7fd6a00731f0 2023-10-18T09:15:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.311+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd698009c40 con 0x7fd6a00731f0 2023-10-18T09:15:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.311+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690010430 con 0x7fd6a00731f0 2023-10-18T09:15:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.311+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a01a0e10 con 0x7fd6a00731f0 2023-10-18T09:15:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.312+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a0107420 con 0x7fd6a00731f0 2023-10-18T09:15:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.315+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd690003fb0 con 0x7fd6a00731f0 2023-10-18T09:15:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.317+0000 7fd68f7fe700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 0x7fd6880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.317+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd690014070 con 0x7fd6a00731f0 2023-10-18T09:15:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.317+0000 7fd69dd9b700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 0x7fd6880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.318+0000 7fd69dd9b700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 0x7fd6880a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd694009c60 tx=0x7fd694009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.319+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd690051100 con 0x7fd6a00731f0 2023-10-18T09:15:56.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.586+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6a0067530 con 0x7fd6a00731f0 2023-10-18T09:15:56.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.587+0000 7fd68f7fe700 1 -- 172.21.15.61:0/218413072 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fd690096a00 con 0x7fd6a00731f0 2023-10-18T09:15:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:56.592 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T09:15:56.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 msgr2=0x7fd6880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:56.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 0x7fd6880a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd694009c60 tx=0x7fd694009260 comp rx=0 tx=0).stop 2023-10-18T09:15:56.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 msgr2=0x7fd6a01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:56.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd69000ec10 tx=0x7fd69000ef20 comp rx=0 tx=0).stop 2023-10-18T09:15:56.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 shutdown_connections 2023-10-18T09:15:56.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6880a1480 0x7fd6880a3940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd6a0108610 0x7fd6a019ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd6a00731f0 0x7fd6a01989b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 --2- 172.21.15.61:0/218413072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6a00747d0 0x7fd6a0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:56.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.589+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 >> 172.21.15.61:0/218413072 conn(0x7fd6a00fbc90 msgr2=0x7fd6a0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:56.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.590+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 shutdown_connections 2023-10-18T09:15:56.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:56.590+0000 7fd6a4fff700 1 -- 172.21.15.61:0/218413072 wait complete. 2023-10-18T09:15:56.598 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 3 2023-10-18T09:15:56.753 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:56 smithi061 systemd[1]: Starting Ceph mds.j for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:15:57.253 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:56 smithi061 podman[149723]: 2023-10-18T09:15:57.753 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:57 smithi061 bash[149723]: 33ab81c00017a5df378bfd7a47feaa97b0a24a50e5c377ca5ec34e54a4fadf5c 2023-10-18T09:15:57.754 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mds[149763]: -- 172.21.15.61:0/2024618157 <== mon.0 v2:172.21.15.61:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558427dbb1e0 con 0x558427d93000 2023-10-18T09:15:57.754 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-j[149739]: starting mds.j at 2023-10-18T09:15:57.754 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:15:57 smithi061 systemd[1]: Started Ceph mds.j for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:15:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/218413072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:57.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:57.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/218413072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:57.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:57.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/218413072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:57.989 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:57.990 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.990 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.990 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:15:57.990 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:15:57.991 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:15:57.991 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:15:58.616 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:15:58.657 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:58 smithi061 ceph-mon[98573]: Deploying daemon mds.d on smithi061 2023-10-18T09:15:58.657 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:58 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6834/2809482931,v1:172.21.15.61:6835/2809482931] up:boot 2023-10-18T09:15:58.657 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:58 smithi061 ceph-mon[98573]: fsmap 3 up:standby 2023-10-18T09:15:58.658 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T09:15:58.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 -- 172.21.15.61:0/2823512955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0731f0 msgr2=0x7f41bc073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:58.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 --2- 172.21.15.61:0/2823512955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0731f0 0x7f41bc073670 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009990 tx=0x7f41ac009ca0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 -- 172.21.15.61:0/2823512955 shutdown_connections 2023-10-18T09:15:58.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 --2- 172.21.15.61:0/2823512955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 0x7f41bc10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 --2- 172.21.15.61:0/2823512955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0747d0 0x7f41bc072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 --2- 172.21.15.61:0/2823512955 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0731f0 0x7f41bc073670 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 -- 172.21.15.61:0/2823512955 >> 172.21.15.61:0/2823512955 conn(0x7f41bc0fbc90 msgr2=0x7f41bc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:58.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 -- 172.21.15.61:0/2823512955 shutdown_connections 2023-10-18T09:15:58.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.827+0000 7f41c41ff700 1 -- 172.21.15.61:0/2823512955 wait complete. 2023-10-18T09:15:58.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 Processor -- start 2023-10-18T09:15:58.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 -- start start 2023-10-18T09:15:58.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:58.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 0x7f41bc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:58.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 0x7f41bc19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:58.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1a0440 con 0x7f41bc0747d0 2023-10-18T09:15:58.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1a0580 con 0x7f41bc108610 2023-10-18T09:15:58.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.828+0000 7f41c41ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1a0710 con 0x7f41bc0731f0 2023-10-18T09:15:58.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:58.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:36076/0 (socket says 172.21.15.61:36076) 2023-10-18T09:15:58.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c279c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 0x7f41bc19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:58.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 -- 172.21.15.61:0/3234434320 learned_addr learned my addr 172.21.15.61:0/3234434320 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:15:58.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c179a700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 0x7f41bc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:58.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 -- 172.21.15.61:0/3234434320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 msgr2=0x7f41bc19fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:58.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 0x7f41bc19fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 -- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 msgr2=0x7f41bc198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:58.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 0x7f41bc198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:58.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 -- 172.21.15.61:0/3234434320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0095f0 con 0x7f41bc0731f0 2023-10-18T09:15:58.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c179a700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 0x7f41bc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:15:58.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.829+0000 7f41c1f9b700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41b800d950 tx=0x7f41b800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:58.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.830+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b800f890 con 0x7f41bc0731f0 2023-10-18T09:15:58.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.830+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41b8010460 con 0x7f41bc0731f0 2023-10-18T09:15:58.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.830+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc1a09f0 con 0x7f41bc0731f0 2023-10-18T09:15:58.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.830+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b800e610 con 0x7f41bc0731f0 2023-10-18T09:15:58.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.830+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1a0f00 con 0x7f41bc0731f0 2023-10-18T09:15:58.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.831+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc107420 con 0x7f41bc0731f0 2023-10-18T09:15:58.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.833+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f41b80108d0 con 0x7f41bc0731f0 2023-10-18T09:15:58.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.834+0000 7f41b2ffd700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 0x7f41a80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:15:58.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.834+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f41b80cc690 con 0x7f41bc0731f0 2023-10-18T09:15:58.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.834+0000 7f41c179a700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 0x7f41a80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:15:58.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.835+0000 7f41c179a700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 0x7f41a80a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00bc60 tx=0x7f41ac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:15:58.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:58.838+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41b80960f0 con 0x7f41bc0731f0 2023-10-18T09:15:58.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:58 smithi170 ceph-mon[135150]: Deploying daemon mds.d on smithi061 2023-10-18T09:15:58.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:58 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6834/2809482931,v1:172.21.15.61:6835/2809482931] up:boot 2023-10-18T09:15:58.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:58 smithi170 ceph-mon[135150]: fsmap 3 up:standby 2023-10-18T09:15:58.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T09:15:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:58 smithi110 ceph-mon[134818]: Deploying daemon mds.d on smithi061 2023-10-18T09:15:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:58 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6834/2809482931,v1:172.21.15.61:6835/2809482931] up:boot 2023-10-18T09:15:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:58 smithi110 ceph-mon[134818]: fsmap 3 up:standby 2023-10-18T09:15:58.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-18T09:15:59.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.113+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41bc067530 con 0x7f41bc0731f0 2023-10-18T09:15:59.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.114+0000 7f41b2ffd700 1 -- 172.21.15.61:0/3234434320 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f41b8099710 con 0x7f41bc0731f0 2023-10-18T09:15:59.117 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:15:59.117 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T09:15:59.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.118+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 msgr2=0x7f41a80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:59.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.118+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 0x7f41a80a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00bc60 tx=0x7f41ac005fd0 comp rx=0 tx=0).stop 2023-10-18T09:15:59.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.118+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 msgr2=0x7f41bc198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:15:59.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.118+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41b800d950 tx=0x7f41b800dc60 comp rx=0 tx=0).stop 2023-10-18T09:15:59.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 shutdown_connections 2023-10-18T09:15:59.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41a80a1480 0x7f41a80a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:59.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41bc108610 0x7f41bc19fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:59.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41bc0731f0 0x7f41bc198960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:59.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 --2- 172.21.15.61:0/3234434320 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41bc0747d0 0x7f41bc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:15:59.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 >> 172.21.15.61:0/3234434320 conn(0x7f41bc0fbc90 msgr2=0x7f41bc104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:15:59.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 shutdown_connections 2023-10-18T09:15:59.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:15:59.119+0000 7f41c41ff700 1 -- 172.21.15.61:0/3234434320 wait complete. 2023-10-18T09:15:59.124 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 4 2023-10-18T09:15:59.644 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:59 smithi061 ceph-mon[98573]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:59.644 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:15:59 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3234434320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:59.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:59 smithi170 ceph-mon[135150]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:15:59 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3234434320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:15:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:59 smithi110 ceph-mon[134818]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:15:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:15:59 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3234434320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:00.003 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:15:59 smithi061 systemd[1]: Starting Ceph mds.d for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:00.445 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:16:00 smithi061 podman[150110]: 2023-10-18T09:16:00.753 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:16:00 smithi061 bash[150110]: 6284a8d9360242019b72689c9e42cb8dc5c4f02109019ffffdc96692d7bcc7e5 2023-10-18T09:16:00.753 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:16:00 smithi061 ceph-mds[150150]: -- 172.21.15.61:0/1706764007 <== mon.0 v2:172.21.15.61:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e9dc5ff1e0 con 0x55e9dc5d6c00 2023-10-18T09:16:00.753 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:16:00 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-d[150126]: starting mds.d at 2023-10-18T09:16:00.754 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:16:00 smithi061 systemd[1]: Started Ceph mds.d for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:01.149 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:01.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1436755321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 msgr2=0x7fbc44107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:01.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1436755321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc44107a50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00b0d0 tx=0x7fbc3c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1436755321 shutdown_connections 2023-10-18T09:16:01.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1436755321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc44107a50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1436755321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc440ff610 0x7fbc440ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1436755321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc44100800 0x7fbc44100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1436755321 >> 172.21.15.61:0/1436755321 conn(0x7fbc440faba0 msgr2=0x7fbc440fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:01.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1436755321 shutdown_connections 2023-10-18T09:16:01.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.360+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1436755321 wait complete. 2023-10-18T09:16:01.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.361+0000 7fbc4a9ff700 1 Processor -- start 2023-10-18T09:16:01.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 -- start start 2023-10-18T09:16:01.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 0x7fbc441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:01.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc44100800 0x7fbc44198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:01.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:01.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc441a04e0 con 0x7fbc441011a0 2023-10-18T09:16:01.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc441a0620 con 0x7fbc440ff610 2023-10-18T09:16:01.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc4a9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc441a07b0 con 0x7fbc44100800 2023-10-18T09:16:01.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc48f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:01.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc48f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34248/0 (socket says 172.21.15.61:34248) 2023-10-18T09:16:01.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc48f9c700 1 -- 172.21.15.61:0/1467977231 learned_addr learned my addr 172.21.15.61:0/1467977231 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:01.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc437fe700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc44100800 0x7fbc44198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:01.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.362+0000 7fbc43fff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 0x7fbc441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:01.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 -- 172.21.15.61:0/1467977231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc44100800 msgr2=0x7fbc44198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:01.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc44100800 0x7fbc44198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 -- 172.21.15.61:0/1467977231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 msgr2=0x7fbc441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:01.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 0x7fbc441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 -- 172.21.15.61:0/1467977231 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38009520 con 0x7fbc441011a0 2023-10-18T09:16:01.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc43fff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 0x7fbc441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:01.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc48f9c700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007b60 tx=0x7fbc3c008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00e040 con 0x7fbc441011a0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc3c01d930 con 0x7fbc441011a0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c009d00 con 0x7fbc441011a0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c01b9f0 con 0x7fbc441011a0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.363+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc441a0d70 con 0x7fbc441011a0 2023-10-18T09:16:01.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.364+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc44067530 con 0x7fbc441011a0 2023-10-18T09:16:01.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.366+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbc3c01daa0 con 0x7fbc441011a0 2023-10-18T09:16:01.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.368+0000 7fbc417fa700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 0x7fbc2c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:01.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.368+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbc3c0cdc90 con 0x7fbc441011a0 2023-10-18T09:16:01.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.368+0000 7fbc43fff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 0x7fbc2c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:01.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.368+0000 7fbc43fff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 0x7fbc2c0a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc44100660 tx=0x7fbc34007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:01.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.372+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc3c097780 con 0x7fbc441011a0 2023-10-18T09:16:01.578 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6836/1807421967,v1:172.21.15.61:6837/1807421967] up:boot 2023-10-18T09:16:01.578 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: fsmap 4 up:standby 2023-10-18T09:16:01.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T09:16:01.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.579 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:01.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:01.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:01.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: Deploying daemon mds.e on smithi110 2023-10-18T09:16:01.580 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:01 smithi110 ceph-mon[134818]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:01.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.630+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc441a0970 con 0x7fbc441011a0 2023-10-18T09:16:01.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.631+0000 7fbc417fa700 1 -- 172.21.15.61:0/1467977231 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7fbc3c021e90 con 0x7fbc441011a0 2023-10-18T09:16:01.633 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-18T09:16:01.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 msgr2=0x7fbc2c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:01.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 0x7fbc2c0a39c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc44100660 tx=0x7fbc34007400 comp rx=0 tx=0).stop 2023-10-18T09:16:01.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 msgr2=0x7fbc4419ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:01.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c007b60 tx=0x7fbc3c008e90 comp rx=0 tx=0).stop 2023-10-18T09:16:01.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 shutdown_connections 2023-10-18T09:16:01.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbc2c0a1500 0x7fbc2c0a39c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbc441011a0 0x7fbc4419ff10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc440ff610 0x7fbc441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 --2- 172.21.15.61:0/1467977231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc44100800 0x7fbc44198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:01.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 >> 172.21.15.61:0/1467977231 conn(0x7fbc440faba0 msgr2=0x7fbc440fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:01.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 shutdown_connections 2023-10-18T09:16:01.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:01.633+0000 7fbc4a9ff700 1 -- 172.21.15.61:0/1467977231 wait complete. 2023-10-18T09:16:01.638 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 5 2023-10-18T09:16:01.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6836/1807421967,v1:172.21.15.61:6837/1807421967] up:boot 2023-10-18T09:16:01.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: fsmap 4 up:standby 2023-10-18T09:16:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T09:16:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:01.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:01.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:01.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: Deploying daemon mds.e on smithi110 2023-10-18T09:16:01.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:01 smithi170 ceph-mon[135150]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:02.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6836/1807421967,v1:172.21.15.61:6837/1807421967] up:boot 2023-10-18T09:16:02.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: fsmap 4 up:standby 2023-10-18T09:16:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-18T09:16:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:02.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: Deploying daemon mds.e on smithi110 2023-10-18T09:16:02.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:01 smithi061 ceph-mon[98573]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:02.212 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 systemd[1]: Starting Ceph mds.e for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:02.655 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 podman[171095]: 2023-10-18T09:16:02.656 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:02 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1467977231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:02.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:02 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1467977231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:02.962 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 ceph-mds[171135]: -- 172.21.15.110:0/1328962606 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5607521031e0 con 0x5607520d9000 2023-10-18T09:16:02.962 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: starting mds.e at 2023-10-18T09:16:02.962 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 bash[171095]: 713b4ffd248d8760fc4392b468741668dd2612d471708863582218a33c53f8fb 2023-10-18T09:16:02.962 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:16:02 smithi110 systemd[1]: Started Ceph mds.e for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:03.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:02 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1467977231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:03.656 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:03.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.874+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/3144255659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c80747d0 msgr2=0x7fb1c8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.874+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/3144255659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c80747d0 0x7fb1c8072c20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009990 tx=0x7fb1b8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/3144255659 shutdown_connections 2023-10-18T09:16:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/3144255659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8108610 0x7fb1c810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/3144255659 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c80747d0 0x7fb1c8072c20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/3144255659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80731f0 0x7fb1c8073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/3144255659 >> 172.21.15.61:0/3144255659 conn(0x7fb1c80fbc90 msgr2=0x7fb1c80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/3144255659 shutdown_connections 2023-10-18T09:16:03.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.875+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/3144255659 wait complete. 2023-10-18T09:16:03.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 Processor -- start 2023-10-18T09:16:03.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 -- start start 2023-10-18T09:16:03.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:03.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80747d0 0x7fb1c8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:03.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 0x7fb1c819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:03.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81a0440 con 0x7fb1c8108610 2023-10-18T09:16:03.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81a0580 con 0x7fb1c80747d0 2023-10-18T09:16:03.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ce5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c81a0710 con 0x7fb1c80731f0 2023-10-18T09:16:03.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:03.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1ccb9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 0x7fb1c819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:03.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.876+0000 7fb1c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:36128/0 (socket says 172.21.15.61:36128) 2023-10-18T09:16:03.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 -- 172.21.15.61:0/413533136 learned_addr learned my addr 172.21.15.61:0/413533136 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:03.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c77fe700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80747d0 0x7fb1c8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:03.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 -- 172.21.15.61:0/413533136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80747d0 msgr2=0x7fb1c8198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:03.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80747d0 0x7fb1c8198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 -- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 msgr2=0x7fb1c819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:03.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 0x7fb1c819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 -- 172.21.15.61:0/413533136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80095f0 con 0x7fb1c80731f0 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1ccb9c700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 0x7fb1c819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c7fff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80095c0 tx=0x7fb1b800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b801d070 con 0x7fb1c80731f0 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1b8005030 con 0x7fb1c80731f0 2023-10-18T09:16:03.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c81a0990 con 0x7fb1c80731f0 2023-10-18T09:16:03.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.877+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8004a50 con 0x7fb1c80731f0 2023-10-18T09:16:03.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.878+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c81a0db0 con 0x7fb1c80731f0 2023-10-18T09:16:03.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.879+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c8107420 con 0x7fb1c80731f0 2023-10-18T09:16:03.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.880+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb1b8005450 con 0x7fb1c80731f0 2023-10-18T09:16:03.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.881+0000 7fb1c57fa700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 0x7fb1b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:03.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.882+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb1b80cd740 con 0x7fb1c80731f0 2023-10-18T09:16:03.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.882+0000 7fb1c77fe700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 0x7fb1b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:03.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.882+0000 7fb1c77fe700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 0x7fb1b00a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007c60 tx=0x7fb1bc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:03.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:03.886+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1b8097230 con 0x7fb1c80731f0 2023-10-18T09:16:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:04.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: Deploying daemon mds.f on smithi170 2023-10-18T09:16:04.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:boot 2023-10-18T09:16:04.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: fsmap 5 up:standby 2023-10-18T09:16:04.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:03 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:16:04.049 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.050 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.050 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.050 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:04.050 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:04.050 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:04.051 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:04.051 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: Deploying daemon mds.f on smithi170 2023-10-18T09:16:04.051 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:boot 2023-10-18T09:16:04.051 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: fsmap 5 up:standby 2023-10-18T09:16:04.051 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:03 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:16:04.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.152+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c8067530 con 0x7fb1c80731f0 2023-10-18T09:16:04.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.153+0000 7fb1c57fa700 1 -- 172.21.15.61:0/413533136 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7fb1b809a850 con 0x7fb1c80731f0 2023-10-18T09:16:04.155 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:04.156 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-18T09:16:04.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 msgr2=0x7fb1b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:04.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 0x7fb1b00a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc007c60 tx=0x7fb1bc0073d0 comp rx=0 tx=0).stop 2023-10-18T09:16:04.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 msgr2=0x7fb1c8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:04.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80095c0 tx=0x7fb1b800f690 comp rx=0 tx=0).stop 2023-10-18T09:16:04.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 shutdown_connections 2023-10-18T09:16:04.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb1b00a1480 0x7fb1b00a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:04.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb1c8108610 0x7fb1c819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:04.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c80731f0 0x7fb1c8198910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:04.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 --2- 172.21.15.61:0/413533136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1c80747d0 0x7fb1c8198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:04.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 >> 172.21.15.61:0/413533136 conn(0x7fb1c80fbc90 msgr2=0x7fb1c8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:04.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 shutdown_connections 2023-10-18T09:16:04.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:04.156+0000 7fb1ce5ff700 1 -- 172.21.15.61:0/413533136 wait complete. 2023-10-18T09:16:04.161 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 6 2023-10-18T09:16:04.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:04.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:04.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:04.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:04.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:04.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: Deploying daemon mds.f on smithi170 2023-10-18T09:16:04.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:boot 2023-10-18T09:16:04.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: fsmap 5 up:standby 2023-10-18T09:16:04.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:03 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:16:04.300 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 systemd[1]: Starting Ceph mds.f for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:04.846 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 podman[172923]: 2023-10-18T09:16:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:04 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/413533136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:05.142 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 bash[172923]: 007a91b4f85a0af55fe49b5650360bc24280d2316343d490f1c81fde4fa8c0d1 2023-10-18T09:16:05.143 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 ceph-mds[172963]: -- 172.21.15.170:0/1640245702 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55844bdfd1e0 con 0x55844bdd3000 2023-10-18T09:16:05.143 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: starting mds.f at 2023-10-18T09:16:05.143 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:16:04 smithi170 systemd[1]: Started Ceph mds.f for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:05.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:04 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/413533136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:05.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:04 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/413533136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:05.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:05.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:05.835 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:05.835 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:06.177 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:06.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:06.245 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:06.245 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.246 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.246 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:06.246 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:06.246 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:06.246 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 -- 172.21.15.61:0/3070744535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 msgr2=0x7f8564072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 --2- 172.21.15.61:0/3070744535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564072c20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8560009990 tx=0x7f8560009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 -- 172.21.15.61:0/3070744535 shutdown_connections 2023-10-18T09:16:06.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 --2- 172.21.15.61:0/3070744535 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 0x7f856410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 --2- 172.21.15.61:0/3070744535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564072c20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 --2- 172.21.15.61:0/3070744535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 0x7f8564073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 -- 172.21.15.61:0/3070744535 >> 172.21.15.61:0/3070744535 conn(0x7f85640fbc90 msgr2=0x7f85640fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.390+0000 7f856c1ff700 1 -- 172.21.15.61:0/3070744535 shutdown_connections 2023-10-18T09:16:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.391+0000 7f856c1ff700 1 -- 172.21.15.61:0/3070744535 wait complete. 2023-10-18T09:16:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.391+0000 7f856c1ff700 1 Processor -- start 2023-10-18T09:16:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 -- start start 2023-10-18T09:16:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 0x7f85641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 0x7f856419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641a03d0 con 0x7f8564108610 2023-10-18T09:16:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641a0510 con 0x7f85640731f0 2023-10-18T09:16:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856c1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85641a06a0 con 0x7f85640747d0 2023-10-18T09:16:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856979a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856979a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:36134/0 (socket says 172.21.15.61:36134) 2023-10-18T09:16:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856979a700 1 -- 172.21.15.61:0/2969527084 learned_addr learned my addr 172.21.15.61:0/2969527084 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f856a79c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 0x7f856419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.392+0000 7f8569f9b700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 0x7f85641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 -- 172.21.15.61:0/2969527084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 msgr2=0x7f85641988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 0x7f85641988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 -- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 msgr2=0x7f856419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 0x7f856419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 -- 172.21.15.61:0/2969527084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600095f0 con 0x7f85640747d0 2023-10-18T09:16:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856979a700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f855400e9a0 tx=0x7f855400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554009790 con 0x7f85640747d0 2023-10-18T09:16:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8554010430 con 0x7f85640747d0 2023-10-18T09:16:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85641a0980 con 0x7f85640747d0 2023-10-18T09:16:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554018680 con 0x7f85640747d0 2023-10-18T09:16:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.393+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85641a0e90 con 0x7f85640747d0 2023-10-18T09:16:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.394+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8564107420 con 0x7f85640747d0 2023-10-18T09:16:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.396+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f85540105a0 con 0x7f85640747d0 2023-10-18T09:16:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.398+0000 7f855affd700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 0x7f85500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:06.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.398+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8554014070 con 0x7f85640747d0 2023-10-18T09:16:06.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.398+0000 7f8569f9b700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 0x7f85500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:06.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.398+0000 7f8569f9b700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 0x7f85500a3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f856000b2e0 tx=0x7f8560005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:06.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.401+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8554052100 con 0x7f85640747d0 2023-10-18T09:16:06.445 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:06 smithi170 systemd[1]: Starting Ceph mds.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:06.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.662+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8564067530 con 0x7f85640747d0 2023-10-18T09:16:06.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.663+0000 7f855affd700 1 -- 172.21.15.61:0/2969527084 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f8554096500 con 0x7f85640747d0 2023-10-18T09:16:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:06.667 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-18T09:16:06.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.665+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 msgr2=0x7f85500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:06.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.665+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 0x7f85500a3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f856000b2e0 tx=0x7f8560005fb0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.665+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 msgr2=0x7f8564198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:06.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.665+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f855400e9a0 tx=0x7f855400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 shutdown_connections 2023-10-18T09:16:06.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85500a1480 0x7f85500a3940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8564108610 0x7f856419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85640731f0 0x7f85641988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 --2- 172.21.15.61:0/2969527084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85640747d0 0x7f8564198e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:06.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 >> 172.21.15.61:0/2969527084 conn(0x7f85640fbc90 msgr2=0x7f8564104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:06.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 shutdown_connections 2023-10-18T09:16:06.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:06.666+0000 7f856c1ff700 1 -- 172.21.15.61:0/2969527084 wait complete. 2023-10-18T09:16:06.671 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 7 2023-10-18T09:16:07.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:06 smithi061 ceph-mon[98573]: Deploying daemon mds.c on smithi170 2023-10-18T09:16:07.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:06 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:boot 2023-10-18T09:16:07.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:06 smithi061 ceph-mon[98573]: fsmap 6 up:standby 2023-10-18T09:16:07.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:06 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:16:07.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:06 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2969527084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:07.033 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mon[135150]: Deploying daemon mds.c on smithi170 2023-10-18T09:16:07.033 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:boot 2023-10-18T09:16:07.033 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mon[135150]: fsmap 6 up:standby 2023-10-18T09:16:07.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:16:07.034 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2969527084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:07.034 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:06 smithi170 podman[173263]: 2023-10-18T09:16:07.034 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-mds[173303]: -- 172.21.15.170:0/2332369843 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd22d491e0 con 0x55cd22d1f000 2023-10-18T09:16:07.035 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:06 smithi170 bash[173263]: 4a7fb03c2f56d14b332eefeb94f7694eca754e1cf6b6bff41bf0e6b99bcd35f1 2023-10-18T09:16:07.035 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: starting mds.c at 2023-10-18T09:16:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:06 smithi110 ceph-mon[134818]: Deploying daemon mds.c on smithi170 2023-10-18T09:16:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:06 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:boot 2023-10-18T09:16:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:06 smithi110 ceph-mon[134818]: fsmap 6 up:standby 2023-10-18T09:16:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:06 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:16:07.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:06 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2969527084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:07.392 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:16:07 smithi170 systemd[1]: Started Ceph mds.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:08.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:08.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:08.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.161 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:08.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:08.162 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:08.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:08.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:08.845 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:08 smithi110 systemd[1]: Starting Ceph mds.h for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:08.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 -- 172.21.15.61:0/1887969851 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c101930 msgr2=0x7fb22c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:08.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 --2- 172.21.15.61:0/1887969851 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c101930 0x7fb22c101d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2200099c0 tx=0x7fb220009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 -- 172.21.15.61:0/1887969851 shutdown_connections 2023-10-18T09:16:08.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 --2- 172.21.15.61:0/1887969851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 0x7fb22c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 --2- 172.21.15.61:0/1887969851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c100740 0x7fb22c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 --2- 172.21.15.61:0/1887969851 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c101930 0x7fb22c101d90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 -- 172.21.15.61:0/1887969851 >> 172.21.15.61:0/1887969851 conn(0x7fb22c0fbc90 msgr2=0x7fb22c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:08.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 -- 172.21.15.61:0/1887969851 shutdown_connections 2023-10-18T09:16:08.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.903+0000 7fb2315ff700 1 -- 172.21.15.61:0/1887969851 wait complete. 2023-10-18T09:16:08.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.904+0000 7fb2315ff700 1 Processor -- start 2023-10-18T09:16:08.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.904+0000 7fb2315ff700 1 -- start start 2023-10-18T09:16:08.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 0x7fb22c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:08.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 0x7fb22c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:08.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 0x7fb22c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:08.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1a0400 con 0x7fb22c100740 2023-10-18T09:16:08.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1a0540 con 0x7fb22c1022d0 2023-10-18T09:16:08.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb2315ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1a06d0 con 0x7fb22c101930 2023-10-18T09:16:08.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 0x7fb22c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:08.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 0x7fb22c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:08.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 0x7fb22c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53514/0 (socket says 172.21.15.61:53514) 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22affd700 1 -- 172.21.15.61:0/4131075022 learned_addr learned my addr 172.21.15.61:0/4131075022 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22b7fe700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 0x7fb22c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 -- 172.21.15.61:0/4131075022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 msgr2=0x7fb22c19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 0x7fb22c19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 -- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 msgr2=0x7fb22c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:08.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 0x7fb22c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:08.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.905+0000 7fb22a7fc700 1 -- 172.21.15.61:0/4131075022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2200095f0 con 0x7fb22c101930 2023-10-18T09:16:08.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb22a7fc700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 0x7fb22c198e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098e0 tx=0x7fb22000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:08.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22001d070 con 0x7fb22c101930 2023-10-18T09:16:08.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb22000bda0 con 0x7fb22c101930 2023-10-18T09:16:08.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200173f0 con 0x7fb22c101930 2023-10-18T09:16:08.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c1a0950 con 0x7fb22c101930 2023-10-18T09:16:08.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.906+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c1a0d70 con 0x7fb22c101930 2023-10-18T09:16:08.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.907+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c067530 con 0x7fb22c101930 2023-10-18T09:16:08.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.909+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb220022470 con 0x7fb22c101930 2023-10-18T09:16:08.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.910+0000 7fb213fff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 0x7fb2140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:08.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.911+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb2200cdb60 con 0x7fb22c101930 2023-10-18T09:16:08.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.911+0000 7fb22affd700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 0x7fb2140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:08.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.911+0000 7fb22affd700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 0x7fb2140a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007cf0 tx=0x7fb21c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:08.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:08.912+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb220097650 con 0x7fb22c101930 2023-10-18T09:16:09.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:08 smithi170 ceph-mon[135150]: Deploying daemon mds.h on smithi110 2023-10-18T09:16:09.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:08 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6836/3025282224,v1:172.21.15.170:6837/3025282224] up:boot 2023-10-18T09:16:09.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:08 smithi170 ceph-mon[135150]: fsmap 7 up:standby 2023-10-18T09:16:09.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:08 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:16:09.154 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:09 smithi110 podman[171438]: 2023-10-18T09:16:09.155 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:08 smithi110 ceph-mon[134818]: Deploying daemon mds.h on smithi110 2023-10-18T09:16:09.155 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:08 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6836/3025282224,v1:172.21.15.170:6837/3025282224] up:boot 2023-10-18T09:16:09.155 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:08 smithi110 ceph-mon[134818]: fsmap 7 up:standby 2023-10-18T09:16:09.155 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:08 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:16:09.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.171+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb22c10bfb0 con 0x7fb22c101930 2023-10-18T09:16:09.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.172+0000 7fb213fff700 1 -- 172.21.15.61:0/4131075022 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fb22009ac70 con 0x7fb22c101930 2023-10-18T09:16:09.174 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:09.175 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-18T09:16:09.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 msgr2=0x7fb2140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:09.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 0x7fb2140a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21c007cf0 tx=0x7fb21c007480 comp rx=0 tx=0).stop 2023-10-18T09:16:09.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 msgr2=0x7fb22c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:09.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 0x7fb22c198e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb2200098e0 tx=0x7fb22000fa00 comp rx=0 tx=0).stop 2023-10-18T09:16:09.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 shutdown_connections 2023-10-18T09:16:09.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2140a1500 0x7fb2140a39c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:09.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb22c1022d0 0x7fb22c19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:09.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb22c100740 0x7fb22c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:09.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 --2- 172.21.15.61:0/4131075022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c101930 0x7fb22c198e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:09.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 >> 172.21.15.61:0/4131075022 conn(0x7fb22c0fbc90 msgr2=0x7fb22c107100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:09.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 shutdown_connections 2023-10-18T09:16:09.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:09.174+0000 7fb2315ff700 1 -- 172.21.15.61:0/4131075022 wait complete. 2023-10-18T09:16:09.179 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 8 2023-10-18T09:16:09.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:08 smithi061 ceph-mon[98573]: Deploying daemon mds.h on smithi110 2023-10-18T09:16:09.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:08 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6836/3025282224,v1:172.21.15.170:6837/3025282224] up:boot 2023-10-18T09:16:09.193 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:08 smithi061 ceph-mon[98573]: fsmap 7 up:standby 2023-10-18T09:16:09.194 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:08 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:16:09.712 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mds[171478]: -- 172.21.15.110:0/2235288650 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d118a811e0 con 0x55d118a56800 2023-10-18T09:16:09.712 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: starting mds.h at 2023-10-18T09:16:09.712 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:09 smithi110 bash[171438]: d3620978b1c472afa3af9470e43bf6a90829661f0aed5df2ddc5aac2a03869d5 2023-10-18T09:16:09.713 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:16:09 smithi110 systemd[1]: Started Ceph mds.h for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:10.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:10.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4131075022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:10.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:10.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:10.105 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:09 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4131075022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:10.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:09 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4131075022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:10.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:09 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:10.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:10 smithi170 ceph-mon[135150]: Deploying daemon mds.i on smithi170 2023-10-18T09:16:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:10 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:boot 2023-10-18T09:16:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:10 smithi170 ceph-mon[135150]: fsmap 8 up:standby 2023-10-18T09:16:10.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:16:11.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:10 smithi110 ceph-mon[134818]: Deploying daemon mds.i on smithi170 2023-10-18T09:16:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:10 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:boot 2023-10-18T09:16:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:10 smithi110 ceph-mon[134818]: fsmap 8 up:standby 2023-10-18T09:16:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:16:11.235 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:10 smithi170 systemd[1]: Starting Ceph mds.i for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:11.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:10 smithi061 ceph-mon[98573]: Deploying daemon mds.i on smithi170 2023-10-18T09:16:11.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:10 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:boot 2023-10-18T09:16:11.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:10 smithi061 ceph-mon[98573]: fsmap 8 up:standby 2023-10-18T09:16:11.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:16:11.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.409+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/1761525328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24101930 msgr2=0x7f8a24101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:11.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.409+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/1761525328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24101930 0x7f8a24101d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009990 tx=0x7f8a14009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/1761525328 shutdown_connections 2023-10-18T09:16:11.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/1761525328 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a241022d0 0x7f8a24108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/1761525328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a24100740 0x7f8a24100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/1761525328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24101930 0x7f8a24101d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/1761525328 >> 172.21.15.61:0/1761525328 conn(0x7f8a240fbc90 msgr2=0x7f8a240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:11.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/1761525328 shutdown_connections 2023-10-18T09:16:11.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.410+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/1761525328 wait complete. 2023-10-18T09:16:11.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 Processor -- start 2023-10-18T09:16:11.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 -- start start 2023-10-18T09:16:11.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:11.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a24101930 0x7f8a24196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:11.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 0x7f8a2419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:11.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419e230 con 0x7f8a24101930 2023-10-18T09:16:11.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419e370 con 0x7f8a241022d0 2023-10-18T09:16:11.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2c7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2419e500 con 0x7f8a24100740 2023-10-18T09:16:11.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2a59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:11.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2a59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44600/0 (socket says 172.21.15.61:44600) 2023-10-18T09:16:11.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a2a59b700 1 -- 172.21.15.61:0/992348688 learned_addr learned my addr 172.21.15.61:0/992348688 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:11.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.411+0000 7f8a29d9a700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a24101930 0x7f8a24196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:11.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2ad9c700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 0x7f8a2419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:11.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 -- 172.21.15.61:0/992348688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 msgr2=0x7f8a2419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:11.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 0x7f8a2419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 -- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a24101930 msgr2=0x7f8a24196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:11.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a24101930 0x7f8a24196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 -- 172.21.15.61:0/992348688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140095f0 con 0x7f8a24100740 2023-10-18T09:16:11.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2ad9c700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 0x7f8a2419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:16:11.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2a59b700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e970 tx=0x7f8a2000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:11.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20009790 con 0x7f8a24100740 2023-10-18T09:16:11.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a2000eec0 con 0x7f8a24100740 2023-10-18T09:16:11.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20003cf0 con 0x7f8a24100740 2023-10-18T09:16:11.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2419e7e0 con 0x7f8a24100740 2023-10-18T09:16:11.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.412+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2419eca0 con 0x7f8a24100740 2023-10-18T09:16:11.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.413+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24107c60 con 0x7f8a24100740 2023-10-18T09:16:11.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.415+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8a2000ccd0 con 0x7f8a24100740 2023-10-18T09:16:11.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.417+0000 7f8a1b7fe700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 0x7f8a100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:11.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.417+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a20014070 con 0x7f8a24100740 2023-10-18T09:16:11.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.417+0000 7f8a29d9a700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 0x7f8a100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:11.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.417+0000 7f8a29d9a700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 0x7f8a100a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400b2e0 tx=0x7f8a14005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:11.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.420+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8a200965d0 con 0x7f8a24100740 2023-10-18T09:16:11.504 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:11 smithi170 podman[173601]: 2023-10-18T09:16:11.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.685+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a2410c260 con 0x7f8a24100740 2023-10-18T09:16:11.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.686+0000 7f8a1b7fe700 1 -- 172.21.15.61:0/992348688 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f8a20099bf0 con 0x7f8a24100740 2023-10-18T09:16:11.689 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:11.689 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-18T09:16:11.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 msgr2=0x7f8a100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:11.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 0x7f8a100a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400b2e0 tx=0x7f8a14005fb0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 msgr2=0x7f8a24196750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:11.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000e970 tx=0x7f8a2000ec80 comp rx=0 tx=0).stop 2023-10-18T09:16:11.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 shutdown_connections 2023-10-18T09:16:11.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a100a1480 0x7f8a100a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a241022d0 0x7f8a2419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a24100740 0x7f8a24196750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 --2- 172.21.15.61:0/992348688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a24101930 0x7f8a24196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:11.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 >> 172.21.15.61:0/992348688 conn(0x7f8a240fbc90 msgr2=0x7f8a24104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:11.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 shutdown_connections 2023-10-18T09:16:11.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:11.689+0000 7f8a2c7ff700 1 -- 172.21.15.61:0/992348688 wait complete. 2023-10-18T09:16:11.694 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 9 2023-10-18T09:16:11.859 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mds[173641]: -- 172.21.15.170:0/1600864208 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564425eff1e0 con 0x564425ed4800 2023-10-18T09:16:11.860 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: starting mds.i at 2023-10-18T09:16:11.860 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:11 smithi170 bash[173601]: 83140e29df1bd77f894a084a6f8f8d097e924a3502af0a36d3c1aed141905446 2023-10-18T09:16:11.860 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:16:11 smithi170 systemd[1]: Started Ceph mds.i for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:12.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:12.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:12.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:11 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/992348688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:12.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:12.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:11 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/992348688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:12.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:12.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:12.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:12.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:11 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/992348688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:12.971 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:12 smithi061 ceph-mon[98573]: Deploying daemon mds.g on smithi061 2023-10-18T09:16:12.971 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:12 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:boot 2023-10-18T09:16:12.971 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:12 smithi061 ceph-mon[98573]: fsmap 9 up:standby 2023-10-18T09:16:12.972 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:12 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:16:13.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:12 smithi170 ceph-mon[135150]: Deploying daemon mds.g on smithi061 2023-10-18T09:16:13.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:12 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:boot 2023-10-18T09:16:13.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:12 smithi170 ceph-mon[135150]: fsmap 9 up:standby 2023-10-18T09:16:13.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:12 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:16:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:12 smithi110 ceph-mon[134818]: Deploying daemon mds.g on smithi061 2023-10-18T09:16:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:12 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:boot 2023-10-18T09:16:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:12 smithi110 ceph-mon[134818]: fsmap 9 up:standby 2023-10-18T09:16:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:12 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:16:13.714 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:14.017 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 systemd[1]: Starting Ceph mds.g for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:14.017 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:13 smithi061 ceph-mon[98573]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:14.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:13 smithi170 ceph-mon[135150]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:14.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 -- 172.21.15.61:0/3093246147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 msgr2=0x7f7184108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:14.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 --2- 172.21.15.61:0/3093246147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184108b70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f717c00b0d0 tx=0x7f717c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 -- 172.21.15.61:0/3093246147 shutdown_connections 2023-10-18T09:16:14.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 --2- 172.21.15.61:0/3093246147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184108b70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 --2- 172.21.15.61:0/3093246147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184100730 0x7f7184100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 --2- 172.21.15.61:0/3093246147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184101920 0x7f7184101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 -- 172.21.15.61:0/3093246147 >> 172.21.15.61:0/3093246147 conn(0x7f71840fbcc0 msgr2=0x7f71840fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:14.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 -- 172.21.15.61:0/3093246147 shutdown_connections 2023-10-18T09:16:14.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.143+0000 7f718bbff700 1 -- 172.21.15.61:0/3093246147 wait complete. 2023-10-18T09:16:14.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.144+0000 7f718bbff700 1 Processor -- start 2023-10-18T09:16:14.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.144+0000 7f718bbff700 1 -- start start 2023-10-18T09:16:14.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184100730 0x7f7184074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71840738c0 con 0x7f71841022c0 2023-10-18T09:16:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184073a30 con 0x7f7184101920 2023-10-18T09:16:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718bbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71841a07d0 con 0x7f7184100730 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718999b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184100730 0x7f7184074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:51126/0 (socket says 172.21.15.61:51126) 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 -- 172.21.15.61:0/950894940 learned_addr learned my addr 172.21.15.61:0/950894940 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718a19c700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:14.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 -- 172.21.15.61:0/950894940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184100730 msgr2=0x7f7184074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:14.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184100730 0x7f7184074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 -- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 msgr2=0x7f7184073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:14.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.145+0000 7f718919a700 1 -- 172.21.15.61:0/950894940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c009d00 con 0x7f7184101920 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f718919a700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f717400e9a0 tx=0x7f717400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174009790 con 0x7f7184101920 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71841a0a60 con 0x7f7184101920 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f717400ce70 con 0x7f7184101920 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174018420 con 0x7f7184101920 2023-10-18T09:16:14.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.146+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71841a0f70 con 0x7f7184101920 2023-10-18T09:16:14.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.148+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7184107ae0 con 0x7f7184101920 2023-10-18T09:16:14.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.149+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f71740106f0 con 0x7f7184101920 2023-10-18T09:16:14.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.151+0000 7f717affd700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 0x7f71700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:14.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.151+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7174014070 con 0x7f7184101920 2023-10-18T09:16:14.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.151+0000 7f718999b700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 0x7f71700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:14.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.152+0000 7f718999b700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 0x7f71700a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7184101780 tx=0x7f7180009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:14.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.155+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7174052100 con 0x7f7184101920 2023-10-18T09:16:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:13 smithi110 ceph-mon[134818]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:14.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.432+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7184067980 con 0x7f7184101920 2023-10-18T09:16:14.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.432+0000 7f717affd700 1 -- 172.21.15.61:0/950894940 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7673 (secure 0 0 0) 0x7f7174096510 con 0x7f7184101920 2023-10-18T09:16:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:14.435 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-18T09:16:14.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 msgr2=0x7f71700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:14.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 0x7f71700a3940 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7184101780 tx=0x7f7180009260 comp rx=0 tx=0).stop 2023-10-18T09:16:14.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 msgr2=0x7f7184072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:14.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f717400e9a0 tx=0x7f717400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 shutdown_connections 2023-10-18T09:16:14.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71700a1480 0x7f71700a3940 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71841022c0 0x7f7184073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7184100730 0x7f7184074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 --2- 172.21.15.61:0/950894940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7184101920 0x7f7184072be0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:14.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 >> 172.21.15.61:0/950894940 conn(0x7f71840fbcc0 msgr2=0x7f7184104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:14.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 shutdown_connections 2023-10-18T09:16:14.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:14.435+0000 7f718bbff700 1 -- 172.21.15.61:0/950894940 wait complete. 2023-10-18T09:16:14.440 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 10 2023-10-18T09:16:14.710 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 podman[150719]: 2023-10-18T09:16:15.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:14 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/950894940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:15.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:14 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/950894940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:15.253 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 bash[150719]: 2fd453c37166ed10dc511ff49ad453762d25baa2b46fc0183cbf4914eb4da4aa 2023-10-18T09:16:15.253 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 ceph-mds[150761]: -- 172.21.15.61:0/3800355495 <== mon.0 v2:172.21.15.61:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563f828191e0 con 0x563f827f0c00 2023-10-18T09:16:15.253 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: starting mds.g at 2023-10-18T09:16:15.254 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:16:14 smithi061 systemd[1]: Started Ceph mds.g for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:15.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:14 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/950894940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:16.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6838/2550944546,v1:172.21.15.61:6839/2550944546] up:boot 2023-10-18T09:16:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: fsmap 10 up:standby 2023-10-18T09:16:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:16:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:16.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:15 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6838/2550944546,v1:172.21.15.61:6839/2550944546] up:boot 2023-10-18T09:16:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: fsmap 10 up:standby 2023-10-18T09:16:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:16:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:16.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:15 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6838/2550944546,v1:172.21.15.61:6839/2550944546] up:boot 2023-10-18T09:16:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: fsmap 10 up:standby 2023-10-18T09:16:16.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:16:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:16.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:16.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:15 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:16.457 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:16.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 -- 172.21.15.61:0/2635756361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100740 msgr2=0x7fa55c100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:16.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 --2- 172.21.15.61:0/2635756361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100740 0x7fa55c100b60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa54c0099c0 tx=0x7fa54c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 -- 172.21.15.61:0/2635756361 shutdown_connections 2023-10-18T09:16:16.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 --2- 172.21.15.61:0/2635756361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c1022d0 0x7fa55c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 --2- 172.21.15.61:0/2635756361 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c100740 0x7fa55c100b60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 --2- 172.21.15.61:0/2635756361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c101930 0x7fa55c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.674+0000 7fa561dff700 1 -- 172.21.15.61:0/2635756361 >> 172.21.15.61:0/2635756361 conn(0x7fa55c0fbc90 msgr2=0x7fa55c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:16.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.675+0000 7fa561dff700 1 -- 172.21.15.61:0/2635756361 shutdown_connections 2023-10-18T09:16:16.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.675+0000 7fa561dff700 1 -- 172.21.15.61:0/2635756361 wait complete. 2023-10-18T09:16:16.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.676+0000 7fa561dff700 1 Processor -- start 2023-10-18T09:16:16.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.676+0000 7fa561dff700 1 -- start start 2023-10-18T09:16:16.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.676+0000 7fa561dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:16.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.676+0000 7fa561dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 0x7fa55c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:16.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa561dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1022d0 0x7fa55c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:16.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa561dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0738c0 con 0x7fa55c1022d0 2023-10-18T09:16:16.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa561dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c073a30 con 0x7fa55c100740 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa561dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c1a0740 con 0x7fa55c101930 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:51150/0 (socket says 172.21.15.61:51150) 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 -- 172.21.15.61:0/1723438951 learned_addr learned my addr 172.21.15.61:0/1723438951 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55affd700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 0x7fa55c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:16.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55bfff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1022d0 0x7fa55c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:16.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 -- 172.21.15.61:0/1723438951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 msgr2=0x7fa55c072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:16.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 0x7fa55c072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 -- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1022d0 msgr2=0x7fa55c073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:16.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1022d0 0x7fa55c073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55b7fe700 1 -- 172.21.15.61:0/1723438951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0095f0 con 0x7fa55c100740 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.677+0000 7fa55affd700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 0x7fa55c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa55b7fe700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006010 tx=0x7fa54c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c01d070 con 0x7fa55c100740 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa54c005020 con 0x7fa55c100740 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c1a0970 con 0x7fa55c100740 2023-10-18T09:16:16.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c004bb0 con 0x7fa55c100740 2023-10-18T09:16:16.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.678+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c1a0d30 con 0x7fa55c100740 2023-10-18T09:16:16.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.679+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c107c10 con 0x7fa55c100740 2023-10-18T09:16:16.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.681+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa54c005190 con 0x7fa55c100740 2023-10-18T09:16:16.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.682+0000 7fa558ff9700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 0x7fa5440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:16.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.682+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa54c0cd380 con 0x7fa55c100740 2023-10-18T09:16:16.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.682+0000 7fa55affd700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 0x7fa5440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:16.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.683+0000 7fa55affd700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 0x7fa5440a39c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa550009cb0 tx=0x7fa550009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:16.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.684+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa54c096e70 con 0x7fa55c100740 2023-10-18T09:16:16.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.962+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa55c067530 con 0x7fa55c100740 2023-10-18T09:16:16.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.963+0000 7fa558ff9700 1 -- 172.21.15.61:0/1723438951 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7fa54c09a490 con 0x7fa55c100740 2023-10-18T09:16:16.966 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:16.966 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24668,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-18T09:16:16.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.966+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 msgr2=0x7fa5440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:16.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.966+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 0x7fa5440a39c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa550009cb0 tx=0x7fa550009260 comp rx=0 tx=0).stop 2023-10-18T09:16:16.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.966+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 msgr2=0x7fa55c074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:16.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.966+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa54c006010 tx=0x7fa54c00f690 comp rx=0 tx=0).stop 2023-10-18T09:16:16.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 shutdown_connections 2023-10-18T09:16:16.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5440a1500 0x7fa5440a39c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa55c1022d0 0x7fa55c073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa55c100740 0x7fa55c074550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 --2- 172.21.15.61:0/1723438951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa55c101930 0x7fa55c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:16.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 >> 172.21.15.61:0/1723438951 conn(0x7fa55c0fbc90 msgr2=0x7fa55c104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:16.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 shutdown_connections 2023-10-18T09:16:16.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:16.967+0000 7fa561dff700 1 -- 172.21.15.61:0/1723438951 wait complete. 2023-10-18T09:16:16.973 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 11 2023-10-18T09:16:17.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:16 smithi170 ceph-mon[135150]: Deploying daemon mds.a on smithi061 2023-10-18T09:16:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:16 smithi110 ceph-mon[134818]: Deploying daemon mds.a on smithi061 2023-10-18T09:16:17.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:16 smithi061 ceph-mon[98573]: Deploying daemon mds.a on smithi061 2023-10-18T09:16:17.753 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:17 smithi061 systemd[1]: Starting Ceph mds.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:18.026 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:17 smithi061 podman[151109]: 2023-10-18T09:16:18.027 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:17 smithi061 ceph-mon[98573]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:18.027 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:17 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1723438951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:17 smithi170 ceph-mon[135150]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:17 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1723438951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:17 smithi110 ceph-mon[134818]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:17 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1723438951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:18.503 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:18 smithi061 bash[151109]: 00d0119d4836ea0ccc3a6038881d35e4e333703fc903b18abcbe9014ddd987ee 2023-10-18T09:16:18.503 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:18 smithi061 ceph-mds[151149]: -- 172.21.15.61:0/1495565148 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5568fd2191e0 con 0x5568fd1f0800 2023-10-18T09:16:18.504 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: starting mds.a at 2023-10-18T09:16:18.504 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:16:18 smithi061 systemd[1]: Started Ceph mds.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:18.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:19.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 -- 172.21.15.61:0/3495723618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80101930 msgr2=0x7f5d80101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:19.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 --2- 172.21.15.61:0/3495723618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80101930 0x7f5d80101d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5d740099c0 tx=0x7f5d74009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 -- 172.21.15.61:0/3495723618 shutdown_connections 2023-10-18T09:16:19.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 --2- 172.21.15.61:0/3495723618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d801022d0 0x7f5d80108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 --2- 172.21.15.61:0/3495723618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d80100740 0x7f5d80100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 --2- 172.21.15.61:0/3495723618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80101930 0x7f5d80101d90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 -- 172.21.15.61:0/3495723618 >> 172.21.15.61:0/3495723618 conn(0x7f5d800fbc90 msgr2=0x7f5d800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:19.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 -- 172.21.15.61:0/3495723618 shutdown_connections 2023-10-18T09:16:19.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.206+0000 7f5d85bff700 1 -- 172.21.15.61:0/3495723618 wait complete. 2023-10-18T09:16:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 Processor -- start 2023-10-18T09:16:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 -- start start 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d80101930 0x7f5d80198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 0x7f5d8019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a0410 con 0x7f5d80100740 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a0550 con 0x7f5d801022d0 2023-10-18T09:16:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.207+0000 7f5d85bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a06e0 con 0x7f5d80101930 2023-10-18T09:16:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49790/0 (socket says 172.21.15.61:49790) 2023-10-18T09:16:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 -- 172.21.15.61:0/2946387752 learned_addr learned my addr 172.21.15.61:0/2946387752 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7ffff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 0x7f5d8019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7effd700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d80101930 0x7f5d80198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 -- 172.21.15.61:0/2946387752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d80101930 msgr2=0x7f5d80198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d80101930 0x7f5d80198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 -- 172.21.15.61:0/2946387752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 msgr2=0x7f5d8019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 0x7f5d8019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 -- 172.21.15.61:0/2946387752 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740095f0 con 0x7f5d80100740 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7ffff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 0x7f5d8019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7f7fe700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e970 tx=0x7f5d7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.208+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70009790 con 0x7f5d80100740 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.209+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d801a09c0 con 0x7f5d80100740 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.209+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7000c7e0 con 0x7f5d80100740 2023-10-18T09:16:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.209+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70003c70 con 0x7f5d80100740 2023-10-18T09:16:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.209+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d801a0f20 con 0x7f5d80100740 2023-10-18T09:16:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.210+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d80067530 con 0x7f5d80100740 2023-10-18T09:16:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.212+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5d7000cc50 con 0x7f5d80100740 2023-10-18T09:16:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.213+0000 7f5d7cff9700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 0x7f5d680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.213+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5d70014070 con 0x7f5d80100740 2023-10-18T09:16:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.213+0000 7f5d7effd700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 0x7f5d680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.214+0000 7f5d7effd700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 0x7f5d680a39c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d74003730 tx=0x7f5d7400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:19.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.218+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d70096490 con 0x7f5d80100740 2023-10-18T09:16:19.330 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.330 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: Deploying daemon mds.b on smithi110 2023-10-18T09:16:19.331 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:19.332 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6840/2114349313,v1:172.21.15.61:6841/2114349313] up:boot 2023-10-18T09:16:19.332 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: fsmap 11 up:standby 2023-10-18T09:16:19.332 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:19 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:16:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:19.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:19.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: Deploying daemon mds.b on smithi110 2023-10-18T09:16:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6840/2114349313,v1:172.21.15.61:6841/2114349313] up:boot 2023-10-18T09:16:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: fsmap 11 up:standby 2023-10-18T09:16:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:19 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:16:19.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.473+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d8010bfb0 con 0x7f5d80100740 2023-10-18T09:16:19.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.474+0000 7f5d7cff9700 1 -- 172.21.15.61:0/2946387752 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7f5d70003dd0 con 0x7f5d80100740 2023-10-18T09:16:19.476 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:19.477 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24668,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-18T09:16:19.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 msgr2=0x7f5d680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:19.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 0x7f5d680a39c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d74003730 tx=0x7f5d7400b540 comp rx=0 tx=0).stop 2023-10-18T09:16:19.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 msgr2=0x7f5d80198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:19.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e970 tx=0x7f5d7000ec80 comp rx=0 tx=0).stop 2023-10-18T09:16:19.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 shutdown_connections 2023-10-18T09:16:19.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d680a1500 0x7f5d680a39c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d801022d0 0x7f5d8019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d80100740 0x7f5d80198930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 --2- 172.21.15.61:0/2946387752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d80101930 0x7f5d80198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:19.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.476+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 >> 172.21.15.61:0/2946387752 conn(0x7f5d800fbc90 msgr2=0x7f5d80072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:19.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.477+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 shutdown_connections 2023-10-18T09:16:19.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:19.477+0000 7f5d85bff700 1 -- 172.21.15.61:0/2946387752 wait complete. 2023-10-18T09:16:19.481 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 12 2023-10-18T09:16:19.496 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.496 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.496 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:19.496 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-18T09:16:19.496 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-18T09:16:19.497 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:19.497 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: Deploying daemon mds.b on smithi110 2023-10-18T09:16:19.497 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:19.497 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6840/2114349313,v1:172.21.15.61:6841/2114349313] up:boot 2023-10-18T09:16:19.497 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: fsmap 11 up:standby 2023-10-18T09:16:19.498 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:19 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:16:19.918 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:19 smithi110 systemd[1]: Starting Ceph mds.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:16:20.199 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:19 smithi110 podman[171788]: 2023-10-18T09:16:20.199 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:20 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2946387752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:20.462 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:20 smithi110 bash[171788]: d8c60f9a4a84c1dff527e17fd67250ffb1ad4fa2b3361a3977c7fd2029ce5a35 2023-10-18T09:16:20.462 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:20 smithi110 ceph-mds[171828]: -- 172.21.15.110:0/548036521 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed5055f1e0 con 0x55ed50534c00 2023-10-18T09:16:20.463 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:20 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: starting mds.b at 2023-10-18T09:16:20.463 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:20 smithi110 systemd[1]: Started Ceph mds.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7. 2023-10-18T09:16:20.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:20 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2946387752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:20.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:20 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2946387752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:21.500 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:21.565 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.566 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.566 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.566 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.566 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.566 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:21.567 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.567 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:21.567 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:boot 2023-10-18T09:16:21.567 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: fsmap 12 up:standby 2023-10-18T09:16:21.567 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:16:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:21.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:boot 2023-10-18T09:16:21.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: fsmap 12 up:standby 2023-10-18T09:16:21.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:16:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:boot 2023-10-18T09:16:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: fsmap 12 up:standby 2023-10-18T09:16:21.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:16:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 -- 172.21.15.61:0/2017219190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40747b0 msgr2=0x7fc1f4072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/2017219190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40747b0 0x7fc1f4072c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009990 tx=0x7fc1e4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 -- 172.21.15.61:0/2017219190 shutdown_connections 2023-10-18T09:16:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/2017219190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f4108630 0x7fc1f410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/2017219190 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40747b0 0x7fc1f4072c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/2017219190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f40731d0 0x7fc1f4073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 -- 172.21.15.61:0/2017219190 >> 172.21.15.61:0/2017219190 conn(0x7fc1f40fbc90 msgr2=0x7fc1f40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 -- 172.21.15.61:0/2017219190 shutdown_connections 2023-10-18T09:16:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.711+0000 7fc1f8dff700 1 -- 172.21.15.61:0/2017219190 wait complete. 2023-10-18T09:16:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.712+0000 7fc1f8dff700 1 Processor -- start 2023-10-18T09:16:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 -- start start 2023-10-18T09:16:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f40747b0 0x7fc1f4198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4108630 0x7fc1f419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41a0390 con 0x7fc1f40731d0 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41a04d0 con 0x7fc1f40747b0 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f8dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f41a0660 con 0x7fc1f4108630 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49814/0 (socket says 172.21.15.61:49814) 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f259c700 1 -- 172.21.15.61:0/1101255345 learned_addr learned my addr 172.21.15.61:0/1101255345 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.713+0000 7fc1f2d9d700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4108630 0x7fc1f419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1ebfff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f40747b0 0x7fc1f4198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 -- 172.21.15.61:0/1101255345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4108630 msgr2=0x7fc1f419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4108630 0x7fc1f419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 -- 172.21.15.61:0/1101255345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f40747b0 msgr2=0x7fc1f4198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f40747b0 0x7fc1f4198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 -- 172.21.15.61:0/1101255345 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40095f0 con 0x7fc1f40731d0 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f259c700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4000c00 tx=0x7fc1e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e401d070 con 0x7fc1f40731d0 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e4021410 con 0x7fc1f40731d0 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f41a08e0 con 0x7fc1f40731d0 2023-10-18T09:16:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4017650 con 0x7fc1f40731d0 2023-10-18T09:16:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.714+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f41a0cb0 con 0x7fc1f40731d0 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.716+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f4107440 con 0x7fc1f40731d0 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.717+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc1e4021810 con 0x7fc1f40731d0 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.719+0000 7fc1eb7fe700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 0x7fc1e00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.719+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1e40cde60 con 0x7fc1f40731d0 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.720+0000 7fc1ebfff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 0x7fc1e00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.720+0000 7fc1ebfff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 0x7fc1e00a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009bf0 tx=0x7fc1dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:21.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:21.723+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc1e4097950 con 0x7fc1f40731d0 2023-10-18T09:16:22.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.015+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1f402cf10 con 0x7fc1f40731d0 2023-10-18T09:16:22.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.016+0000 7fc1eb7fe700 1 -- 172.21.15.61:0/1101255345 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7fc1e409af70 con 0x7fc1f40731d0 2023-10-18T09:16:22.019 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:22.019 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24668,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24692,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-18T09:16:22.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 msgr2=0x7fc1e00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 0x7fc1e00a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009bf0 tx=0x7fc1dc009260 comp rx=0 tx=0).stop 2023-10-18T09:16:22.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 msgr2=0x7fc1f4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4000c00 tx=0x7fc1e4005740 comp rx=0 tx=0).stop 2023-10-18T09:16:22.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 shutdown_connections 2023-10-18T09:16:22.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1e00a1480 0x7fc1e00a3940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f4108630 0x7fc1f419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f40731d0 0x7fc1f4198860 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 --2- 172.21.15.61:0/1101255345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f40747b0 0x7fc1f4198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 >> 172.21.15.61:0/1101255345 conn(0x7fc1f40fbc90 msgr2=0x7fc1f4104580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:22.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 shutdown_connections 2023-10-18T09:16:22.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.019+0000 7fc1f8dff700 1 -- 172.21.15.61:0/1101255345 wait complete. 2023-10-18T09:16:22.024 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 13 2023-10-18T09:16:22.041 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:22.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.261+0000 7f5080fff700 1 -- 172.21.15.61:0/1387171589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c0731d0 msgr2=0x7f507c073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.261+0000 7f5080fff700 1 --2- 172.21.15.61:0/1387171589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c0731d0 0x7f507c073650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5070009990 tx=0x7f5070009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.261+0000 7f5080fff700 1 -- 172.21.15.61:0/1387171589 shutdown_connections 2023-10-18T09:16:22.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.261+0000 7f5080fff700 1 --2- 172.21.15.61:0/1387171589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c108630 0x7f507c10aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.262+0000 7f5080fff700 1 --2- 172.21.15.61:0/1387171589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0747b0 0x7f507c072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.262+0000 7f5080fff700 1 --2- 172.21.15.61:0/1387171589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c0731d0 0x7f507c073650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.262+0000 7f5080fff700 1 -- 172.21.15.61:0/1387171589 >> 172.21.15.61:0/1387171589 conn(0x7f507c0fbc90 msgr2=0x7f507c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:22.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.262+0000 7f5080fff700 1 -- 172.21.15.61:0/1387171589 shutdown_connections 2023-10-18T09:16:22.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.262+0000 7f5080fff700 1 -- 172.21.15.61:0/1387171589 wait complete. 2023-10-18T09:16:22.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.263+0000 7f5080fff700 1 Processor -- start 2023-10-18T09:16:22.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.263+0000 7f5080fff700 1 -- start start 2023-10-18T09:16:22.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0731d0 0x7f507c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c108630 0x7f507c19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a03c0 con 0x7f507c108630 2023-10-18T09:16:22.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a0500 con 0x7f507c0747b0 2023-10-18T09:16:22.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5080fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1a0690 con 0x7f507c0731d0 2023-10-18T09:16:22.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5079d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5079d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:41958/0 (socket says 172.21.15.61:41958) 2023-10-18T09:16:22.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f5079d9b700 1 -- 172.21.15.61:0/1959598763 learned_addr learned my addr 172.21.15.61:0/1959598763 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:22.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f507ad9d700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c108630 0x7f507c19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.264+0000 7f507a59c700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0731d0 0x7f507c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 -- 172.21.15.61:0/1959598763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0731d0 msgr2=0x7f507c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0731d0 0x7f507c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 -- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c108630 msgr2=0x7f507c19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c108630 0x7f507c19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 -- 172.21.15.61:0/1959598763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5074009c40 con 0x7f507c0747b0 2023-10-18T09:16:22.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5079d9b700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5070005910 tx=0x7f5070009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:22.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507001d070 con 0x7f507c0747b0 2023-10-18T09:16:22.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5070021410 con 0x7f507c0747b0 2023-10-18T09:16:22.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700095f0 con 0x7f507c0747b0 2023-10-18T09:16:22.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070017650 con 0x7f507c0747b0 2023-10-18T09:16:22.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.265+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c1a0ba0 con 0x7f507c0747b0 2023-10-18T09:16:22.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.266+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c107440 con 0x7f507c0747b0 2023-10-18T09:16:22.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.268+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5070021580 con 0x7f507c0747b0 2023-10-18T09:16:22.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.269+0000 7f506b7fe700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 0x7f50640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.270+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f50700cdbb0 con 0x7f507c0747b0 2023-10-18T09:16:22.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.270+0000 7f507a59c700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 0x7f50640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.270+0000 7f507a59c700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 0x7f50640a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f506c009c60 tx=0x7f506c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:22.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.273+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f50700976a0 con 0x7f507c0747b0 2023-10-18T09:16:22.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.507+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f507c067530 con 0x7f507c0747b0 2023-10-18T09:16:22.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.508+0000 7f506b7fe700 1 -- 172.21.15.61:0/1959598763 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18387 (secure 0 0 0) 0x7f507009acc0 con 0x7f507c0747b0 2023-10-18T09:16:22.511 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:22.511 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":80,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:15:37.907737+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:22.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 msgr2=0x7f50640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 0x7f50640a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f506c009c60 tx=0x7f506c009260 comp rx=0 tx=0).stop 2023-10-18T09:16:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 msgr2=0x7f507c198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5070005910 tx=0x7f5070009580 comp rx=0 tx=0).stop 2023-10-18T09:16:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 shutdown_connections 2023-10-18T09:16:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f50640a1480 0x7f50640a3940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f507c108630 0x7f507c19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f507c0731d0 0x7f507c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 --2- 172.21.15.61:0/1959598763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f507c0747b0 0x7f507c198dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 >> 172.21.15.61:0/1959598763 conn(0x7f507c0fbc90 msgr2=0x7f507c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 shutdown_connections 2023-10-18T09:16:22.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.511+0000 7f5080fff700 1 -- 172.21.15.61:0/1959598763 wait complete. 2023-10-18T09:16:22.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-18T09:16:22.534 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:16:22.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 -- 172.21.15.61:0/3820742882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 msgr2=0x7f7e6c107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 --2- 172.21.15.61:0/3820742882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c107c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0098f0 tx=0x7f7e5c009c00 comp rx=0 tx=0).stop 2023-10-18T09:16:22.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 -- 172.21.15.61:0/3820742882 shutdown_connections 2023-10-18T09:16:22.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 --2- 172.21.15.61:0/3820742882 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 --2- 172.21.15.61:0/3820742882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c107c10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 --2- 172.21.15.61:0/3820742882 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 0x7f7e6c10eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 -- 172.21.15.61:0/3820742882 >> 172.21.15.61:0/3820742882 conn(0x7f7e6c06d7d0 msgr2=0x7f7e6c06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:22.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 -- 172.21.15.61:0/3820742882 shutdown_connections 2023-10-18T09:16:22.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.748+0000 7f7e70bff700 1 -- 172.21.15.61:0/3820742882 wait complete. 2023-10-18T09:16:22.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.749+0000 7f7e70bff700 1 Processor -- start 2023-10-18T09:16:22.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 -- start start 2023-10-18T09:16:22.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c10a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 0x7f7e6c108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1097a0 con 0x7f7e6c108150 2023-10-18T09:16:22.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c109910 con 0x7f7e6c1077f0 2023-10-18T09:16:22.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.750+0000 7f7e70bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1a8fb0 con 0x7f7e6c10f070 2023-10-18T09:16:22.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c10a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44768/0 (socket says 172.21.15.61:44768) 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6ad9d700 1 -- 172.21.15.61:0/190594688 learned_addr learned my addr 172.21.15.61:0/190594688 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e69d9b700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 0x7f7e6c108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 -- 172.21.15.61:0/190594688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 msgr2=0x7f7e6c109000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 -- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 msgr2=0x7f7e6c108ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 0x7f7e6c108ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 -- 172.21.15.61:0/190594688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64009d00 con 0x7f7e6c1077f0 2023-10-18T09:16:22.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6ad9d700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:22.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.751+0000 7f7e6a59c700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c10a430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009230 tx=0x7f7e5c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:22.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.752+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c01c070 con 0x7f7e6c1077f0 2023-10-18T09:16:22.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.752+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c009520 con 0x7f7e6c1077f0 2023-10-18T09:16:22.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.752+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e5c00f7d0 con 0x7f7e6c1077f0 2023-10-18T09:16:22.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.752+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0038e0 con 0x7f7e6c1077f0 2023-10-18T09:16:22.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.752+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c1a9470 con 0x7f7e6c1077f0 2023-10-18T09:16:22.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.753+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c10e2c0 con 0x7f7e6c1077f0 2023-10-18T09:16:22.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.755+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7e5c020030 con 0x7f7e6c1077f0 2023-10-18T09:16:22.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.756+0000 7f7e5b7fe700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 0x7f7e540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:22.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.756+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7e5c0cd740 con 0x7f7e6c1077f0 2023-10-18T09:16:22.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.756+0000 7f7e69d9b700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 0x7f7e540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:22.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.757+0000 7f7e69d9b700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 0x7f7e540a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009c60 tx=0x7f7e60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:22.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.760+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7e5c097230 con 0x7f7e6c1077f0 2023-10-18T09:16:22.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.984+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e6c1147d0 con 0x7f7e6c1077f0 2023-10-18T09:16:22.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.985+0000 7f7e5b7fe700 1 -- 172.21.15.61:0/190594688 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e5c005600 con 0x7f7e6c1077f0 2023-10-18T09:16:22.987 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:16:22.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 msgr2=0x7f7e540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 0x7f7e540a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009c60 tx=0x7f7e60009260 comp rx=0 tx=0).stop 2023-10-18T09:16:22.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 msgr2=0x7f7e6c10a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:22.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c10a430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009230 tx=0x7f7e5c00b790 comp rx=0 tx=0).stop 2023-10-18T09:16:22.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 shutdown_connections 2023-10-18T09:16:22.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7e540a1480 0x7f7e540a3940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e6c10f070 0x7f7e6c109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e6c1077f0 0x7f7e6c10a430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 --2- 172.21.15.61:0/190594688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7e6c108150 0x7f7e6c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 >> 172.21.15.61:0/190594688 conn(0x7f7e6c06d7d0 msgr2=0x7f7e6c06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 shutdown_connections 2023-10-18T09:16:22.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:22.987+0000 7f7e70bff700 1 -- 172.21.15.61:0/190594688 wait complete. 2023-10-18T09:16:23.008 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-18T09:16:23.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:22 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1101255345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:22 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1959598763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:22 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1101255345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:22 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1959598763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 -- 172.21.15.61:0/3494139512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff000 msgr2=0x7f3d0c0ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 --2- 172.21.15.61:0/3494139512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c0ff420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009990 tx=0x7f3cfc009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 -- 172.21.15.61:0/3494139512 shutdown_connections 2023-10-18T09:16:23.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 --2- 172.21.15.61:0/3494139512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 0x7f3d0c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 --2- 172.21.15.61:0/3494139512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c0ff420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.221+0000 7f3d10fff700 1 --2- 172.21.15.61:0/3494139512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.222+0000 7f3d10fff700 1 -- 172.21.15.61:0/3494139512 >> 172.21.15.61:0/3494139512 conn(0x7f3d0c0faba0 msgr2=0x7f3d0c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:23.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.222+0000 7f3d10fff700 1 -- 172.21.15.61:0/3494139512 shutdown_connections 2023-10-18T09:16:23.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.222+0000 7f3d10fff700 1 -- 172.21.15.61:0/3494139512 wait complete. 2023-10-18T09:16:23.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 Processor -- start 2023-10-18T09:16:23.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 -- start start 2023-10-18T09:16:23.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 0x7f3d0c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1a0360 con 0x7f3d0c0ff000 2023-10-18T09:16:23.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1a04a0 con 0x7f3d0c0ff960 2023-10-18T09:16:23.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.223+0000 7f3d10fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c1a0630 con 0x7f3d0c100320 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d09d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49898/0 (socket says 172.21.15.61:49898) 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 -- 172.21.15.61:0/1057292537 learned_addr learned my addr 172.21.15.61:0/1057292537 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d09d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:42000/0 (socket says 172.21.15.61:42000) 2023-10-18T09:16:23.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0ad9d700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 0x7f3d0c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 -- 172.21.15.61:0/1057292537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 msgr2=0x7f3d0c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 0x7f3d0c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 -- 172.21.15.61:0/1057292537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 msgr2=0x7f3d0c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 -- 172.21.15.61:0/1057292537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00009520 con 0x7f3d0c0ff000 2023-10-18T09:16:23.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d09d9b700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:23.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.224+0000 7f3d0a59c700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005910 tx=0x7f3cfc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:23.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.225+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc01d070 con 0x7f3d0c0ff000 2023-10-18T09:16:23.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.226+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc0095f0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.226+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c1a0ba0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.226+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3cfc021410 con 0x7f3d0c0ff000 2023-10-18T09:16:23.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.226+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc017650 con 0x7f3d0c0ff000 2023-10-18T09:16:23.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.227+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c109fb0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.229+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3cfc021840 con 0x7f3d0c0ff000 2023-10-18T09:16:23.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.230+0000 7f3cfb7fe700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 0x7f3cf40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.230+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3cfc0cd9b0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.230+0000 7f3d09d9b700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 0x7f3cf40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.231+0000 7f3d09d9b700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 0x7f3cf40a39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:23.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.234+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3cfc0974a0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:22 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1101255345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:22 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1959598763' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:23.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.456+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3d0c067530 con 0x7f3d0c0ff000 2023-10-18T09:16:23.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.457+0000 7f3cfb7fe700 1 -- 172.21.15.61:0/1057292537 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3cfc09aac0 con 0x7f3d0c0ff000 2023-10-18T09:16:23.460 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:23.460 INFO:teuthology.orchestra.run.smithi061.stdout:[ 2023-10-18T09:16:23.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:16:23.460 INFO:teuthology.orchestra.run.smithi061.stdout: "poolnum": 1, 2023-10-18T09:16:23.460 INFO:teuthology.orchestra.run.smithi061.stdout: "poolname": ".mgr" 2023-10-18T09:16:23.461 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:16:23.461 INFO:teuthology.orchestra.run.smithi061.stdout:] 2023-10-18T09:16:23.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 msgr2=0x7f3cf40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 0x7f3cf40a39c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d00009260 comp rx=0 tx=0).stop 2023-10-18T09:16:23.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 msgr2=0x7f3d0c198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005910 tx=0x7f3cfc005740 comp rx=0 tx=0).stop 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 shutdown_connections 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3cf40a1500 0x7f3cf40a39c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d0c100320 0x7f3d0c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d0c0ff000 0x7f3d0c198830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 --2- 172.21.15.61:0/1057292537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0ff960 0x7f3d0c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 >> 172.21.15.61:0/1057292537 conn(0x7f3d0c0faba0 msgr2=0x7f3d0c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:23.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.459+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 shutdown_connections 2023-10-18T09:16:23.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.460+0000 7f3d10fff700 1 -- 172.21.15.61:0/1057292537 wait complete. 2023-10-18T09:16:23.482 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-18T09:16:23.482 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-18T09:16:23.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 -- 172.21.15.61:0/1438599367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d41007a0 msgr2=0x7ff6d4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/1438599367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d41007a0 0x7ff6d4100bc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40099c0 tx=0x7ff6c4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 -- 172.21.15.61:0/1438599367 shutdown_connections 2023-10-18T09:16:23.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/1438599367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d4102330 0x7ff6d4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/1438599367 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d41007a0 0x7ff6d4100bc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/1438599367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4101990 0x7ff6d4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 -- 172.21.15.61:0/1438599367 >> 172.21.15.61:0/1438599367 conn(0x7ff6d40fbcf0 msgr2=0x7ff6d40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 -- 172.21.15.61:0/1438599367 shutdown_connections 2023-10-18T09:16:23.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.706+0000 7ff6d93ff700 1 -- 172.21.15.61:0/1438599367 wait complete. 2023-10-18T09:16:23.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 Processor -- start 2023-10-18T09:16:23.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 -- start start 2023-10-18T09:16:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d4101990 0x7ff6d4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 0x7ff6d419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0450 con 0x7ff6d4101990 2023-10-18T09:16:23.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0590 con 0x7ff6d41007a0 2023-10-18T09:16:23.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.708+0000 7ff6d93ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41a0720 con 0x7ff6d4102330 2023-10-18T09:16:23.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:42016/0 (socket says 172.21.15.61:42016) 2023-10-18T09:16:23.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 -- 172.21.15.61:0/3651570906 learned_addr learned my addr 172.21.15.61:0/3651570906 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:23.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d27fc700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d4101990 0x7ff6d4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d37fe700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 0x7ff6d419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 -- 172.21.15.61:0/3651570906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 msgr2=0x7ff6d419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 0x7ff6d419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 -- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d4101990 msgr2=0x7ff6d4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d4101990 0x7ff6d4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:23.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 -- 172.21.15.61:0/3651570906 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c40095f0 con 0x7ff6d41007a0 2023-10-18T09:16:23.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d37fe700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 0x7ff6d419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:23.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.709+0000 7ff6d2ffd700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4006010 tx=0x7ff6c400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:23.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.710+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c401d070 con 0x7ff6d41007a0 2023-10-18T09:16:23.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.710+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff6c4005020 con 0x7ff6d41007a0 2023-10-18T09:16:23.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.710+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d41a09a0 con 0x7ff6d41007a0 2023-10-18T09:16:23.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.710+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4004bb0 con 0x7ff6d41007a0 2023-10-18T09:16:23.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.710+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d41a0d60 con 0x7ff6d41007a0 2023-10-18T09:16:23.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.711+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40675a0 con 0x7ff6d41007a0 2023-10-18T09:16:23.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.713+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff6c4005190 con 0x7ff6d41007a0 2023-10-18T09:16:23.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.714+0000 7ff6bbfff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 0x7ff6bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:23.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.714+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6c40cd360 con 0x7ff6d41007a0 2023-10-18T09:16:23.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.714+0000 7ff6d27fc700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 0x7ff6bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:23.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.715+0000 7ff6d27fc700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 0x7ff6bc0a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:23.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.718+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff6c4096e50 con 0x7ff6d41007a0 2023-10-18T09:16:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:23.937+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff6d41a10e0 con 0x7ff6d41007a0 2023-10-18T09:16:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/190594688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:24.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:23 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1057292537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T09:16:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/190594688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:23 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1057292537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T09:16:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/190594688' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:24.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:23 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1057292537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-18T09:16:24.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.839+0000 7ff6bbfff700 1 -- 172.21.15.61:0/3651570906 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff6c409a470 con 0x7ff6d41007a0 2023-10-18T09:16:24.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.844+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 msgr2=0x7ff6bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:24.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.844+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 0x7ff6bc0a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009c60 tx=0x7ff6c8009260 comp rx=0 tx=0).stop 2023-10-18T09:16:24.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.844+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 msgr2=0x7ff6d4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:24.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.844+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4006010 tx=0x7ff6c400f690 comp rx=0 tx=0).stop 2023-10-18T09:16:24.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 shutdown_connections 2023-10-18T09:16:24.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff6bc0a1480 0x7ff6bc0a3940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:24.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6d4102330 0x7ff6d419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:24.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6d41007a0 0x7ff6d4198920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:24.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 --2- 172.21.15.61:0/3651570906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff6d4101990 0x7ff6d4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:24.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 >> 172.21.15.61:0/3651570906 conn(0x7ff6d40fbcf0 msgr2=0x7ff6d4072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:24.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 shutdown_connections 2023-10-18T09:16:24.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:24.845+0000 7ff6d93ff700 1 -- 172.21.15.61:0/3651570906 wait complete. 2023-10-18T09:16:24.849 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_metadata' created 2023-10-18T09:16:24.867 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-18T09:16:25.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 -- 172.21.15.61:0/3244834074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8864101990 msgr2=0x7f8864101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:25.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 --2- 172.21.15.61:0/3244834074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8864101990 0x7f8864101df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8858009990 tx=0x7f8858009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 -- 172.21.15.61:0/3244834074 shutdown_connections 2023-10-18T09:16:25.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 --2- 172.21.15.61:0/3244834074 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864102330 0x7f8864108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 --2- 172.21.15.61:0/3244834074 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88641007a0 0x7f8864100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 --2- 172.21.15.61:0/3244834074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8864101990 0x7f8864101df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 -- 172.21.15.61:0/3244834074 >> 172.21.15.61:0/3244834074 conn(0x7f88640fbcf0 msgr2=0x7f88640fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:25.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 -- 172.21.15.61:0/3244834074 shutdown_connections 2023-10-18T09:16:25.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.079+0000 7f88697ff700 1 -- 172.21.15.61:0/3244834074 wait complete. 2023-10-18T09:16:25.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 Processor -- start 2023-10-18T09:16:25.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 -- start start 2023-10-18T09:16:25.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 0x7f8864196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:25.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864101990 0x7f8864196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:25.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 0x7f886419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:25.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419e250 con 0x7f8864102330 2023-10-18T09:16:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419e390 con 0x7f8864101990 2023-10-18T09:16:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.081+0000 7f88697ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419e520 con 0x7f88641007a0 2023-10-18T09:16:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f88637fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 0x7f886419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f88637fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 0x7f886419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49926/0 (socket says 172.21.15.61:49926) 2023-10-18T09:16:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 0x7f8864196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:25.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f88637fe700 1 -- 172.21.15.61:0/3575463711 learned_addr learned my addr 172.21.15.61:0/3575463711 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:25.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f88627fc700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864101990 0x7f8864196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:25.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 -- 172.21.15.61:0/3575463711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864101990 msgr2=0x7f8864196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:25.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864101990 0x7f8864196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 -- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 msgr2=0x7f886419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:25.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 0x7f886419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 -- 172.21.15.61:0/3575463711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88580095f0 con 0x7f88641007a0 2023-10-18T09:16:25.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f8862ffd700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 0x7f8864196720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8854009c20 tx=0x7f8854009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:25.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.082+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854015070 con 0x7f88641007a0 2023-10-18T09:16:25.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.083+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8854004020 con 0x7f88641007a0 2023-10-18T09:16:25.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.083+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8864072a80 con 0x7f88641007a0 2023-10-18T09:16:25.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.083+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854003bc0 con 0x7f88641007a0 2023-10-18T09:16:25.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.083+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8864072fe0 con 0x7f88641007a0 2023-10-18T09:16:25.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.084+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8864107bc0 con 0x7f88641007a0 2023-10-18T09:16:25.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.086+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8854004190 con 0x7f88641007a0 2023-10-18T09:16:25.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.087+0000 7f884bfff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 0x7f884c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:25.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.087+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f88540d04a0 con 0x7f88641007a0 2023-10-18T09:16:25.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.088+0000 7f88627fc700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 0x7f884c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:25.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.088+0000 7f88627fc700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 0x7f884c0a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f885800b2e0 tx=0x7f8858005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:25.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.090+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8854099d90 con 0x7f88641007a0 2023-10-18T09:16:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3651570906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:24 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3651570906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:24 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3651570906' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:24 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-18T09:16:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:25.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.317+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f88640675a0 con 0x7f88641007a0 2023-10-18T09:16:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.852+0000 7f884bfff700 1 -- 172.21.15.61:0/3575463711 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7f885409d3b0 con 0x7f88641007a0 2023-10-18T09:16:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 msgr2=0x7f884c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 0x7f884c0a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f885800b2e0 tx=0x7f8858005fb0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 msgr2=0x7f8864196720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:25.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 0x7f8864196720 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8854009c20 tx=0x7f8854009f30 comp rx=0 tx=0).stop 2023-10-18T09:16:25.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 shutdown_connections 2023-10-18T09:16:25.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f884c0a1500 0x7f884c0a39c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8864102330 0x7f886419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88641007a0 0x7f8864196720 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 --2- 172.21.15.61:0/3575463711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8864101990 0x7f8864196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:25.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.863+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 >> 172.21.15.61:0/3575463711 conn(0x7f88640fbcf0 msgr2=0x7f8864104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:25.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.864+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 shutdown_connections 2023-10-18T09:16:25.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:25.867+0000 7f88697ff700 1 -- 172.21.15.61:0/3575463711 wait complete. 2023-10-18T09:16:25.876 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_data' created 2023-10-18T09:16:25.898 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-18T09:16:26.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.119+0000 7f48837ff700 1 -- 172.21.15.61:0/897219142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c0ff060 msgr2=0x7f487c0ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:26.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.119+0000 7f48837ff700 1 --2- 172.21.15.61:0/897219142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c0ff060 0x7f487c0ff480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48780099c0 tx=0x7f4878009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 -- 172.21.15.61:0/897219142 shutdown_connections 2023-10-18T09:16:26.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 --2- 172.21.15.61:0/897219142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c100380 0x7f487c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 --2- 172.21.15.61:0/897219142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c0ff060 0x7f487c0ff480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 --2- 172.21.15.61:0/897219142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 -- 172.21.15.61:0/897219142 >> 172.21.15.61:0/897219142 conn(0x7f487c0fac00 msgr2=0x7f487c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:26.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 -- 172.21.15.61:0/897219142 shutdown_connections 2023-10-18T09:16:26.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.120+0000 7f48837ff700 1 -- 172.21.15.61:0/897219142 wait complete. 2023-10-18T09:16:26.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.121+0000 7f48837ff700 1 Processor -- start 2023-10-18T09:16:26.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 -- start start 2023-10-18T09:16:26.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:26.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 0x7f487c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a0410 con 0x7f487c0ff9c0 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a0550 con 0x7f487c100380 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f48837ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c1a06e0 con 0x7f487c0ff060 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f488159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:26.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f4880d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:26.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f488159b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44856/0 (socket says 172.21.15.61:44856) 2023-10-18T09:16:26.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f488159b700 1 -- 172.21.15.61:0/2870819141 learned_addr learned my addr 172.21.15.61:0/2870819141 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:26.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.122+0000 7f4881d9c700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 0x7f487c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:26.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 -- 172.21.15.61:0/2870819141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 msgr2=0x7f487c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:26.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 0x7f487c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 -- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 msgr2=0x7f487c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 -- 172.21.15.61:0/2870819141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48780095f0 con 0x7f487c0ff060 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f4881d9c700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 0x7f487c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f4880d9a700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:26.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f488159b700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4878003730 tx=0x7f487800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:26.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487801d070 con 0x7f487c0ff060 2023-10-18T09:16:26.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487c1a0960 con 0x7f487c0ff060 2023-10-18T09:16:26.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4878005050 con 0x7f487c0ff060 2023-10-18T09:16:26.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878004b10 con 0x7f487c0ff060 2023-10-18T09:16:26.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.123+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c1a0de0 con 0x7f487c0ff060 2023-10-18T09:16:26.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.125+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c10a010 con 0x7f487c0ff060 2023-10-18T09:16:26.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.127+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f48780051c0 con 0x7f487c0ff060 2023-10-18T09:16:26.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.128+0000 7f48727fc700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 0x7f48680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:26.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.128+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f48780cda90 con 0x7f487c0ff060 2023-10-18T09:16:26.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.128+0000 7f4880d9a700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 0x7f48680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:26.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.129+0000 7f4880d9a700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 0x7f48680a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f486c011490 tx=0x7f486c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:26.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.131+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f48780973b0 con 0x7f487c0ff060 2023-10-18T09:16:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T09:16:26.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T09:16:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3575463711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:26.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T09:16:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T09:16:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3575463711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:26.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-18T09:16:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: osdmap e81: 12 total, 12 up, 12 in 2023-10-18T09:16:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3575463711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-18T09:16:26.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:26.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:26.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.394+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f487c0675a0 con 0x7f487c0ff060 2023-10-18T09:16:26.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.849+0000 7f48727fc700 1 -- 172.21.15.61:0/2870819141 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f487809a9d0 con 0x7f487c0ff060 2023-10-18T09:16:26.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 msgr2=0x7f48680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:26.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 0x7f48680a3940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f486c011490 tx=0x7f486c009260 comp rx=0 tx=0).stop 2023-10-18T09:16:26.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 msgr2=0x7f487c1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:26.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4878003730 tx=0x7f487800f690 comp rx=0 tx=0).stop 2023-10-18T09:16:26.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 shutdown_connections 2023-10-18T09:16:26.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f48680a1480 0x7f48680a3940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f487c100380 0x7f487c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f487c0ff060 0x7f487c1988e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 --2- 172.21.15.61:0/2870819141 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f487c0ff9c0 0x7f487c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:26.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.852+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 >> 172.21.15.61:0/2870819141 conn(0x7f487c0fac00 msgr2=0x7f487c0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:26.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.854+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 shutdown_connections 2023-10-18T09:16:26.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:26.854+0000 7f48837ff700 1 -- 172.21.15.61:0/2870819141 wait complete. 2023-10-18T09:16:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:new fs with metadata pool 2 and data pool 3 2023-10-18T09:16:26.878 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-18T09:16:26.878 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-18T09:16:27.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/1906896938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641019b0 msgr2=0x7f3d64101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/1906896938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641019b0 0x7f3d64101e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d540099c0 tx=0x7f3d54009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/1906896938 shutdown_connections 2023-10-18T09:16:27.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/1906896938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 0x7f3d64108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/1906896938 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641007c0 0x7f3d64100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/1906896938 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641019b0 0x7f3d64101e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/1906896938 >> 172.21.15.61:0/1906896938 conn(0x7f3d640fbd30 msgr2=0x7f3d640fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:27.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/1906896938 shutdown_connections 2023-10-18T09:16:27.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.099+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/1906896938 wait complete. 2023-10-18T09:16:27.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.100+0000 7f3d6b5ff700 1 Processor -- start 2023-10-18T09:16:27.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.100+0000 7f3d6b5ff700 1 -- start start 2023-10-18T09:16:27.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.100+0000 7f3d6b5ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 0x7f3d64198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6b5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6b5ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 0x7f3d6419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6b5ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a0470 con 0x7f3d641007c0 2023-10-18T09:16:27.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6b5ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a05b0 con 0x7f3d64102350 2023-10-18T09:16:27.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6b5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641a0740 con 0x7f3d641019b0 2023-10-18T09:16:27.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6939b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 0x7f3d64198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44878/0 (socket says 172.21.15.61:44878) 2023-10-18T09:16:27.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 -- 172.21.15.61:0/3691527947 learned_addr learned my addr 172.21.15.61:0/3691527947 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:27.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d69b9c700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 0x7f3d6419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 -- 172.21.15.61:0/3691527947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 msgr2=0x7f3d6419fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 0x7f3d6419fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 -- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 msgr2=0x7f3d64198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 0x7f3d64198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 -- 172.21.15.61:0/3691527947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d60009520 con 0x7f3d641019b0 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d6939b700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 0x7f3d64198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.101+0000 7f3d68b9a700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003730 tx=0x7f3d5400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.102+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5401d070 con 0x7f3d641019b0 2023-10-18T09:16:27.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.102+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d5400bda0 con 0x7f3d641019b0 2023-10-18T09:16:27.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.102+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004bd0 con 0x7f3d641019b0 2023-10-18T09:16:27.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.102+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d540095f0 con 0x7f3d641019b0 2023-10-18T09:16:27.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.102+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d641a0c50 con 0x7f3d641019b0 2023-10-18T09:16:27.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.103+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d64066d10 con 0x7f3d641019b0 2023-10-18T09:16:27.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.105+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3d54004d30 con 0x7f3d641019b0 2023-10-18T09:16:27.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.106+0000 7f3d5a7fc700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 0x7f3d500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.106+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3d540cdbe0 con 0x7f3d641019b0 2023-10-18T09:16:27.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.106+0000 7f3d6939b700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 0x7f3d500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.107+0000 7f3d6939b700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 0x7f3d500a39c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009fd0 tx=0x7f3d60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:27.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.108+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3d540974a0 con 0x7f3d641019b0 2023-10-18T09:16:27.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:27.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T09:16:27.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T09:16:27.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:27.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2870819141' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T09:16:27.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:26 smithi170 ceph-mon[135150]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T09:16:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T09:16:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T09:16:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2870819141' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T09:16:27.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:26 smithi110 ceph-mon[134818]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T09:16:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-18T09:16:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: osdmap e82: 12 total, 12 up, 12 in 2023-10-18T09:16:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:16:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2870819141' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-18T09:16:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-18T09:16:27.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:26 smithi061 ceph-mon[98573]: osdmap e83: 12 total, 12 up, 12 in 2023-10-18T09:16:27.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.335+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f3d6410c110 con 0x7f3d641019b0 2023-10-18T09:16:27.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.501+0000 7f3d5a7fc700 1 -- 172.21.15.61:0/3691527947 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f3d54026020 con 0x7f3d641019b0 2023-10-18T09:16:27.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.504+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 msgr2=0x7f3d500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.504+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 0x7f3d500a39c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009fd0 tx=0x7f3d60009260 comp rx=0 tx=0).stop 2023-10-18T09:16:27.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.504+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 msgr2=0x7f3d64198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.504+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003730 tx=0x7f3d5400fa00 comp rx=0 tx=0).stop 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 shutdown_connections 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3d500a1500 0x7f3d500a39c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d64102350 0x7f3d6419fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3d641007c0 0x7f3d64198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 --2- 172.21.15.61:0/3691527947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3d641019b0 0x7f3d64198e80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 >> 172.21.15.61:0/3691527947 conn(0x7f3d640fbd30 msgr2=0x7f3d641071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:27.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 shutdown_connections 2023-10-18T09:16:27.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.505+0000 7f3d6b5ff700 1 -- 172.21.15.61:0/3691527947 wait complete. 2023-10-18T09:16:27.527 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-18T09:16:27.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 -- 172.21.15.61:0/3155869246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 msgr2=0x7f6450100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 --2- 172.21.15.61:0/3155869246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450100af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6440009990 tx=0x7f6440009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 -- 172.21.15.61:0/3155869246 shutdown_connections 2023-10-18T09:16:27.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 --2- 172.21.15.61:0/3155869246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f6450108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 --2- 172.21.15.61:0/3155869246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450100af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 --2- 172.21.15.61:0/3155869246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 0x7f6450101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 -- 172.21.15.61:0/3155869246 >> 172.21.15.61:0/3155869246 conn(0x7f64500fbc60 msgr2=0x7f64500fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:27.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 -- 172.21.15.61:0/3155869246 shutdown_connections 2023-10-18T09:16:27.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.751+0000 7f64569ff700 1 -- 172.21.15.61:0/3155869246 wait complete. 2023-10-18T09:16:27.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.753+0000 7f64569ff700 1 Processor -- start 2023-10-18T09:16:27.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.753+0000 7f64569ff700 1 -- start start 2023-10-18T09:16:27.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.753+0000 7f64569ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.753+0000 7f64569ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 0x7f6450196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f64569ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f645019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f64569ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645019e170 con 0x7f64501018c0 2023-10-18T09:16:27.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f64569ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645019e2b0 con 0x7f6450102260 2023-10-18T09:16:27.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f64569ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645019e440 con 0x7f64501006d0 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f644ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f644f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 0x7f6450196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f644ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44900/0 (socket says 172.21.15.61:44900) 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f644ffff700 1 -- 172.21.15.61:0/646756944 learned_addr learned my addr 172.21.15.61:0/646756944 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f645019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 -- 172.21.15.61:0/646756944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 msgr2=0x7f6450196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 -- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 msgr2=0x7f6450196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:27.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 0x7f6450196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:27.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 -- 172.21.15.61:0/646756944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009520 con 0x7f6450102260 2023-10-18T09:16:27.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f644ffff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.754+0000 7f6454f9c700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f645019dba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f644800efc0 tx=0x7f644800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.755+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448018070 con 0x7f6450102260 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.755+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64400095f0 con 0x7f6450102260 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.755+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f644800f040 con 0x7f6450102260 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.755+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800cd40 con 0x7f6450102260 2023-10-18T09:16:27.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.755+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645019ea40 con 0x7f6450102260 2023-10-18T09:16:27.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.756+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6450107b50 con 0x7f6450102260 2023-10-18T09:16:27.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.758+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6448003a00 con 0x7f6450102260 2023-10-18T09:16:27.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.759+0000 7f644d7fa700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 0x7f64380a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:27.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.759+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f64480cc920 con 0x7f6450102260 2023-10-18T09:16:27.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.759+0000 7f644ffff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 0x7f64380a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:27.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.760+0000 7f644ffff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 0x7f64380a3930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f644000b2e0 tx=0x7f6440005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:27.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.764+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6448095fd0 con 0x7f6450102260 2023-10-18T09:16:28.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:27.999+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f645002cfb0 con 0x7f6450102260 2023-10-18T09:16:28.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: pgmap v286: 97 pgs: 16 creating+peering, 7 active+clean, 74 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T09:16:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T09:16:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T09:16:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3691527947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T09:16:28.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:27 smithi170 ceph-mon[135150]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T09:16:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: pgmap v286: 97 pgs: 16 creating+peering, 7 active+clean, 74 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T09:16:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T09:16:28.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T09:16:28.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3691527947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T09:16:28.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:27 smithi110 ceph-mon[134818]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T09:16:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: pgmap v286: 97 pgs: 16 creating+peering, 7 active+clean, 74 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-18T09:16:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-18T09:16:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-18T09:16:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3691527947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-18T09:16:28.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-18T09:16:28.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:27 smithi061 ceph-mon[98573]: osdmap e84: 12 total, 12 up, 12 in 2023-10-18T09:16:29.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:active 2023-10-18T09:16:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/646756944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:28 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:active 2023-10-18T09:16:29.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/646756944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:28 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] up:active 2023-10-18T09:16:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/646756944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-18T09:16:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:28 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:29.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.874+0000 7f644d7fa700 1 -- 172.21.15.61:0/646756944 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f64480995f0 con 0x7f6450102260 2023-10-18T09:16:29.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 msgr2=0x7f64380a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:29.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 0x7f64380a3930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f644000b2e0 tx=0x7f6440005fb0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 msgr2=0x7f645019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:29.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f645019dba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f644800efc0 tx=0x7f644800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 shutdown_connections 2023-10-18T09:16:29.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f64380a1470 0x7f64380a3930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6450102260 0x7f645019dba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64501006d0 0x7f6450196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 --2- 172.21.15.61:0/646756944 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f64501018c0 0x7f6450196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:29.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 >> 172.21.15.61:0/646756944 conn(0x7f64500fbc60 msgr2=0x7f6450104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:29.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 shutdown_connections 2023-10-18T09:16:29.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:29.881+0000 7f64569ff700 1 -- 172.21.15.61:0/646756944 wait complete. 2023-10-18T09:16:29.886 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_data_ec' created 2023-10-18T09:16:29.904 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-18T09:16:30.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 -- 172.21.15.61:0/3378379331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 msgr2=0x7f80c4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:30.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 --2- 172.21.15.61:0/3378379331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4101df0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0098f0 tx=0x7f80ac009c00 comp rx=0 tx=0).stop 2023-10-18T09:16:30.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 -- 172.21.15.61:0/3378379331 shutdown_connections 2023-10-18T09:16:30.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 --2- 172.21.15.61:0/3378379331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 0x7f80c4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 --2- 172.21.15.61:0/3378379331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 0x7f80c4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 --2- 172.21.15.61:0/3378379331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4101df0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 -- 172.21.15.61:0/3378379331 >> 172.21.15.61:0/3378379331 conn(0x7f80c40fbcf0 msgr2=0x7f80c40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:30.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 -- 172.21.15.61:0/3378379331 shutdown_connections 2023-10-18T09:16:30.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.112+0000 7f80c9bff700 1 -- 172.21.15.61:0/3378379331 wait complete. 2023-10-18T09:16:30.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.113+0000 7f80c9bff700 1 Processor -- start 2023-10-18T09:16:30.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.113+0000 7f80c9bff700 1 -- start start 2023-10-18T09:16:30.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 0x7f80c4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:30.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:30.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 0x7f80c4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:30.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40738c0 con 0x7f80c4102330 2023-10-18T09:16:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4073a30 con 0x7f80c41007a0 2023-10-18T09:16:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c9bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41a07a0 con 0x7f80c4101990 2023-10-18T09:16:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:33846/0 (socket says 172.21.15.61:33846) 2023-10-18T09:16:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 -- 172.21.15.61:0/4107521056 learned_addr learned my addr 172.21.15.61:0/4107521056 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:30.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c37fe700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 0x7f80c4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:30.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80c3fff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 0x7f80c4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:30.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 -- 172.21.15.61:0/4107521056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 msgr2=0x7f80c4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:30.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 0x7f80c4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 -- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 msgr2=0x7f80c4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:30.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 0x7f80c4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.114+0000 7f80bbfff700 1 -- 172.21.15.61:0/4107521056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc009d00 con 0x7f80c4101990 2023-10-18T09:16:30.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80bbfff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009810 tx=0x7f80ac00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:30.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac01c070 con 0x7f80c4101990 2023-10-18T09:16:30.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac009520 con 0x7f80c4101990 2023-10-18T09:16:30.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80ac00bc10 con 0x7f80c4101990 2023-10-18T09:16:30.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0039c0 con 0x7f80c4101990 2023-10-18T09:16:30.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.115+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c41a0c00 con 0x7f80c4101990 2023-10-18T09:16:30.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.116+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c4107c70 con 0x7f80c4101990 2023-10-18T09:16:30.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.118+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80ac020030 con 0x7f80c4101990 2023-10-18T09:16:30.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.119+0000 7f80c17fa700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 0x7f80b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:30.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.120+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f80ac0ce2f0 con 0x7f80c4101990 2023-10-18T09:16:30.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.120+0000 7f80c37fe700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 0x7f80b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:30.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.120+0000 7f80c37fe700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 0x7f80b00a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f80c41017f0 tx=0x7f80b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:30.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.122+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f80ac0977b0 con 0x7f80c4101990 2023-10-18T09:16:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: pgmap v288: 97 pgs: 16 creating+peering, 43 active+clean, 38 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T09:16:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:30.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:30.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: pgmap v288: 97 pgs: 16 creating+peering, 43 active+clean, 38 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T09:16:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:30.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:30.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: pgmap v288: 97 pgs: 16 creating+peering, 43 active+clean, 38 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:16:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: osdmap e85: 12 total, 12 up, 12 in 2023-10-18T09:16:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:16:30.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:16:30.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:16:30.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.333+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f80c40675a0 con 0x7f80c4101990 2023-10-18T09:16:30.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.883+0000 7f80c17fa700 1 -- 172.21.15.61:0/4107521056 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f80ac09add0 con 0x7f80c4101990 2023-10-18T09:16:30.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 msgr2=0x7f80b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:30.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 0x7f80b00a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f80c41017f0 tx=0x7f80b4009260 comp rx=0 tx=0).stop 2023-10-18T09:16:30.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 msgr2=0x7f80c4072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:30.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009810 tx=0x7f80ac00b560 comp rx=0 tx=0).stop 2023-10-18T09:16:30.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 shutdown_connections 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b00a1480 0x7f80b00a3940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4102330 0x7f80c4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c41007a0 0x7f80c4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 --2- 172.21.15.61:0/4107521056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101990 0x7f80c4072be0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 >> 172.21.15.61:0/4107521056 conn(0x7f80c40fbcf0 msgr2=0x7f80c4104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 shutdown_connections 2023-10-18T09:16:30.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:30.887+0000 7f80c9bff700 1 -- 172.21.15.61:0/4107521056 wait complete. 2023-10-18T09:16:30.892 INFO:teuthology.orchestra.run.smithi061.stderr:set pool 4 allow_ec_overwrites to true 2023-10-18T09:16:30.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-18T09:16:31.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 -- 172.21.15.61:0/4131711961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c1007a0 msgr2=0x7fb75c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:31.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 --2- 172.21.15.61:0/4131711961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c1007a0 0x7fb75c100bc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0099c0 tx=0x7fb74c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 -- 172.21.15.61:0/4131711961 shutdown_connections 2023-10-18T09:16:31.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 --2- 172.21.15.61:0/4131711961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c102330 0x7fb75c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 --2- 172.21.15.61:0/4131711961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c1007a0 0x7fb75c100bc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 --2- 172.21.15.61:0/4131711961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c101990 0x7fb75c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.131+0000 7fb7613ff700 1 -- 172.21.15.61:0/4131711961 >> 172.21.15.61:0/4131711961 conn(0x7fb75c0fbcf0 msgr2=0x7fb75c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:31.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.132+0000 7fb7613ff700 1 -- 172.21.15.61:0/4131711961 shutdown_connections 2023-10-18T09:16:31.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.132+0000 7fb7613ff700 1 -- 172.21.15.61:0/4131711961 wait complete. 2023-10-18T09:16:31.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.132+0000 7fb7613ff700 1 Processor -- start 2023-10-18T09:16:31.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 -- start start 2023-10-18T09:16:31.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:31.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:31.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c102330 0x7fb75c19fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:31.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c1a04c0 con 0x7fb75c101990 2023-10-18T09:16:31.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c1a0600 con 0x7fb75c102330 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb7613ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c1a0790 con 0x7fb75c1007a0 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55638/0 (socket says 172.21.15.61:55638) 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75a7fc700 1 -- 172.21.15.61:0/422665631 learned_addr learned my addr 172.21.15.61:0/422665631 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:31.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75affd700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:33856/0 (socket says 172.21.15.61:33856) 2023-10-18T09:16:31.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.133+0000 7fb75b7fe700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c102330 0x7fb75c19fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:31.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 -- 172.21.15.61:0/422665631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 msgr2=0x7fb75c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:31.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 -- 172.21.15.61:0/422665631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c102330 msgr2=0x7fb75c19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:31.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c102330 0x7fb75c19fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 -- 172.21.15.61:0/422665631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c0095f0 con 0x7fb75c101990 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75affd700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb75a7fc700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9a0 tx=0x7fb75000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750009790 con 0x7fb75c101990 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb750010430 con 0x7fb75c101990 2023-10-18T09:16:31.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c1a0a70 con 0x7fb75c101990 2023-10-18T09:16:31.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.134+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750018680 con 0x7fb75c101990 2023-10-18T09:16:31.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.135+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c1a0f30 con 0x7fb75c101990 2023-10-18T09:16:31.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:30 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T09:16:31.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:30 smithi170 ceph-mon[135150]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T09:16:31.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:30 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4107521056' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:30 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.136+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c0675a0 con 0x7fb75c101990 2023-10-18T09:16:31.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.138+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb7500105a0 con 0x7fb75c101990 2023-10-18T09:16:31.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.139+0000 7fb743fff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:31.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.139+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fb750014070 con 0x7fb75c101990 2023-10-18T09:16:31.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.140+0000 7fb75affd700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:31.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.140+0000 7fb75affd700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb74c003730 tx=0x7fb74c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:31.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.142+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb750096700 con 0x7fb75c101990 2023-10-18T09:16:31.157 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:30 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T09:16:31.157 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:30 smithi110 ceph-mon[134818]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T09:16:31.157 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:30 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4107521056' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.158 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:30 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:30 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-18T09:16:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:30 smithi061 ceph-mon[98573]: osdmap e86: 12 total, 12 up, 12 in 2023-10-18T09:16:31.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4107521056' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:30 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-18T09:16:31.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.405+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb75c10c000 con 0x7fb75c101990 2023-10-18T09:16:31.462 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: 2023-10-18T09:16:31.153+0000 7f8e6fe88700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-18T09:16:31.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.884+0000 7fb743fff700 1 -- 172.21.15.61:0/422665631 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb750099d20 con 0x7fb75c101990 2023-10-18T09:16:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 msgr2=0x7fb7440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb74c003730 tx=0x7fb74c00b540 comp rx=0 tx=0).stop 2023-10-18T09:16:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 msgr2=0x7fb75c198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:31.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb75000e9a0 tx=0x7fb75000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 shutdown_connections 2023-10-18T09:16:31.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb7440a1480 0x7fb7440a3940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb75c102330 0x7fb75c19fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb75c1007a0 0x7fb75c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 --2- 172.21.15.61:0/422665631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb75c101990 0x7fb75c198f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 >> 172.21.15.61:0/422665631 conn(0x7fb75c0fbcf0 msgr2=0x7fb75c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 shutdown_connections 2023-10-18T09:16:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:31.887+0000 7fb7613ff700 1 -- 172.21.15.61:0/422665631 wait complete. 2023-10-18T09:16:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:added data pool 4 to fsmap 2023-10-18T09:16:31.909 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 -- 172.21.15.61:0/889575993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 msgr2=0x7f4058108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 --2- 172.21.15.61:0/889575993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f4058108b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f405000b0d0 tx=0x7f405000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 -- 172.21.15.61:0/889575993 shutdown_connections 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 --2- 172.21.15.61:0/889575993 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f4058108b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 --2- 172.21.15.61:0/889575993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 0x7f4058100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 --2- 172.21.15.61:0/889575993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 0x7f4058101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 -- 172.21.15.61:0/889575993 >> 172.21.15.61:0/889575993 conn(0x7f40580fbc90 msgr2=0x7f40580fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 -- 172.21.15.61:0/889575993 shutdown_connections 2023-10-18T09:16:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.114+0000 7f405d3ff700 1 -- 172.21.15.61:0/889575993 wait complete. 2023-10-18T09:16:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.115+0000 7f405d3ff700 1 Processor -- start 2023-10-18T09:16:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 -- start start 2023-10-18T09:16:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 0x7f40581967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 0x7f4058196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819e2f0 con 0x7f40581022d0 2023-10-18T09:16:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819e430 con 0x7f4058100740 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f405d3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405819e5c0 con 0x7f4058101930 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40577fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40577fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55652/0 (socket says 172.21.15.61:55652) 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40577fe700 1 -- 172.21.15.61:0/2687958912 learned_addr learned my addr 172.21.15.61:0/2687958912 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40567fc700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 0x7f4058196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f4056ffd700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 0x7f40581967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40577fe700 1 -- 172.21.15.61:0/2687958912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 msgr2=0x7f4058196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.116+0000 7f40577fe700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 0x7f4058196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f40577fe700 1 -- 172.21.15.61:0/2687958912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 msgr2=0x7f40581967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f40577fe700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 0x7f40581967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f40577fe700 1 -- 172.21.15.61:0/2687958912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050009d00 con 0x7f40581022d0 2023-10-18T09:16:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f40577fe700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4050007b60 tx=0x7f4050008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405000e040 con 0x7f40581022d0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f405001d930 con 0x7f40581022d0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405001b9f0 con 0x7f40581022d0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405819e840 con 0x7f40581022d0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.117+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405819ec70 con 0x7f40581022d0 2023-10-18T09:16:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.118+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4058107af0 con 0x7f40581022d0 2023-10-18T09:16:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.120+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f405001daa0 con 0x7f40581022d0 2023-10-18T09:16:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.121+0000 7f403ffff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 0x7f40400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.122+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f40500ce310 con 0x7f40581022d0 2023-10-18T09:16:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.122+0000 7f4056ffd700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 0x7f40400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.122+0000 7f4056ffd700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 0x7f40400a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4058101790 tx=0x7f4048009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.125+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f40500977b0 con 0x7f40581022d0 2023-10-18T09:16:32.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: pgmap v291: 129 pgs: 18 creating+peering, 87 active+clean, 24 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T09:16:32.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T09:16:32.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T09:16:32.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T09:16:32.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T09:16:32.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:31 smithi170 ceph-mon[135150]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T09:16:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: pgmap v291: 129 pgs: 18 creating+peering, 87 active+clean, 24 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T09:16:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T09:16:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T09:16:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T09:16:32.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T09:16:32.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:31 smithi110 ceph-mon[134818]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T09:16:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: pgmap v291: 129 pgs: 18 creating+peering, 87 active+clean, 24 unknown; 577 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2023-10-18T09:16:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-18T09:16:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-18T09:16:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: osdmap e87: 12 total, 12 up, 12 in 2023-10-18T09:16:32.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-18T09:16:32.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:31 smithi061 ceph-mon[98573]: osdmap e88: 12 total, 12 up, 12 in 2023-10-18T09:16:32.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.387+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40581074d0 con 0x7f40581022d0 2023-10-18T09:16:32.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.387+0000 7f403ffff700 1 -- 172.21.15.61:0/2687958912 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11270 (secure 0 0 0) 0x7f405009add0 con 0x7f40581022d0 2023-10-18T09:16:32.390 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:32.390 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24668,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:31.407216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T09:16:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 msgr2=0x7f40400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 0x7f40400a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4058101790 tx=0x7f4048009260 comp rx=0 tx=0).stop 2023-10-18T09:16:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 msgr2=0x7f405819dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4050007b60 tx=0x7f4050008e40 comp rx=0 tx=0).stop 2023-10-18T09:16:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 shutdown_connections 2023-10-18T09:16:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40400a1480 0x7f40400a3940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40581022d0 0x7f405819dd20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4058100740 0x7f40581967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 --2- 172.21.15.61:0/2687958912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4058101930 0x7f4058196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 >> 172.21.15.61:0/2687958912 conn(0x7f40580fbc90 msgr2=0x7f4058104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:32.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 shutdown_connections 2023-10-18T09:16:32.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.390+0000 7f405d3ff700 1 -- 172.21.15.61:0/2687958912 wait complete. 2023-10-18T09:16:32.395 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 17 2023-10-18T09:16:32.412 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:32.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 -- 172.21.15.61:0/2737499385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c074830 msgr2=0x7f387c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 --2- 172.21.15.61:0/2737499385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c074830 0x7f387c072c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f386c009990 tx=0x7f386c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 -- 172.21.15.61:0/2737499385 shutdown_connections 2023-10-18T09:16:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 --2- 172.21.15.61:0/2737499385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c108600 0x7f387c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 --2- 172.21.15.61:0/2737499385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c074830 0x7f387c072c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 --2- 172.21.15.61:0/2737499385 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c073250 0x7f387c0736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 -- 172.21.15.61:0/2737499385 >> 172.21.15.61:0/2737499385 conn(0x7f387c0fbc90 msgr2=0x7f387c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.614+0000 7f3881dff700 1 -- 172.21.15.61:0/2737499385 shutdown_connections 2023-10-18T09:16:32.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.615+0000 7f3881dff700 1 -- 172.21.15.61:0/2737499385 wait complete. 2023-10-18T09:16:32.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.615+0000 7f3881dff700 1 Processor -- start 2023-10-18T09:16:32.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 -- start start 2023-10-18T09:16:32.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 0x7f387c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c108600 0x7f387c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a03e0 con 0x7f387c074830 2023-10-18T09:16:32.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a0520 con 0x7f387c073250 2023-10-18T09:16:32.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f3881dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a06b0 con 0x7f387c108600 2023-10-18T09:16:32.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f387affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 0x7f387c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43246/0 (socket says 172.21.15.61:43246) 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f387b7fe700 1 -- 172.21.15.61:0/1121830551 learned_addr learned my addr 172.21.15.61:0/1121830551 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.616+0000 7f387bfff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c108600 0x7f387c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 -- 172.21.15.61:0/1121830551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c108600 msgr2=0x7f387c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c108600 0x7f387c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 -- 172.21.15.61:0/1121830551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 msgr2=0x7f387c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 -- 172.21.15.61:0/1121830551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3870009520 con 0x7f387c074830 2023-10-18T09:16:32.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387b7fe700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:32.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f387affd700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 0x7f387c198e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f387000e9c0 tx=0x7f387000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:32.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387000cb70 con 0x7f387c074830 2023-10-18T09:16:32.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f387000ee60 con 0x7f387c074830 2023-10-18T09:16:32.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c0095f0 con 0x7f387c074830 2023-10-18T09:16:32.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870018680 con 0x7f387c074830 2023-10-18T09:16:32.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.617+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1a0d60 con 0x7f387c074830 2023-10-18T09:16:32.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.618+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387c1073c0 con 0x7f387c074830 2023-10-18T09:16:32.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.620+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3870010430 con 0x7f387c074830 2023-10-18T09:16:32.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.622+0000 7f3878ff9700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 0x7f38640a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:32.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.623+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3870014070 con 0x7f387c074830 2023-10-18T09:16:32.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.623+0000 7f387b7fe700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 0x7f38640a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:32.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.623+0000 7f387b7fe700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 0x7f38640a39c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f386c00b2e0 tx=0x7f386c00b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:32.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.626+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3870096640 con 0x7f387c074830 2023-10-18T09:16:32.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.839+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f387c067530 con 0x7f387c074830 2023-10-18T09:16:32.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.841+0000 7f3878ff9700 1 -- 172.21.15.61:0/1121830551 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f387000ce60 con 0x7f387c074830 2023-10-18T09:16:32.844 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:32.844 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:32.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 msgr2=0x7f38640a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 0x7f38640a39c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f386c00b2e0 tx=0x7f386c00b750 comp rx=0 tx=0).stop 2023-10-18T09:16:32.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 msgr2=0x7f387c198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:32.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 0x7f387c198e40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f387000e9c0 tx=0x7f387000ecd0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 shutdown_connections 2023-10-18T09:16:32.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38640a1500 0x7f38640a39c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c108600 0x7f387c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f387c073250 0x7f387c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.843+0000 7f3881dff700 1 --2- 172.21.15.61:0/1121830551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f387c074830 0x7f387c198e40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:32.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.844+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 >> 172.21.15.61:0/1121830551 conn(0x7f387c0fbc90 msgr2=0x7f387c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:32.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.844+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 shutdown_connections 2023-10-18T09:16:32.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:32.844+0000 7f3881dff700 1 -- 172.21.15.61:0/1121830551 wait complete. 2023-10-18T09:16:32.866 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:33.072 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 -- 172.21.15.61:0/3995991632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa4100740 msgr2=0x7f9fa4100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.072 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/3995991632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa4100740 0x7f9fa4100b60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009990 tx=0x7f9fa0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.072 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 -- 172.21.15.61:0/3995991632 shutdown_connections 2023-10-18T09:16:33.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/3995991632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa41022d0 0x7f9fa4108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/3995991632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa4100740 0x7f9fa4100b60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/3995991632 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4101930 0x7f9fa4101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.069+0000 7f9fac5ff700 1 -- 172.21.15.61:0/3995991632 >> 172.21.15.61:0/3995991632 conn(0x7f9fa40fbc90 msgr2=0x7f9fa40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:33.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.070+0000 7f9fac5ff700 1 -- 172.21.15.61:0/3995991632 shutdown_connections 2023-10-18T09:16:33.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.070+0000 7f9fac5ff700 1 -- 172.21.15.61:0/3995991632 wait complete. 2023-10-18T09:16:33.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 Processor -- start 2023-10-18T09:16:33.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 -- start start 2023-10-18T09:16:33.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 0x7f9fa4196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 0x7f9fa419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa419e1f0 con 0x7f9fa4100740 2023-10-18T09:16:33.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa419e330 con 0x7f9fa4101930 2023-10-18T09:16:33.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fac5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa419e4c0 con 0x7f9fa41022d0 2023-10-18T09:16:33.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9faa39b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9faa39b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55674/0 (socket says 172.21.15.61:55674) 2023-10-18T09:16:33.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9faa39b700 1 -- 172.21.15.61:0/2832961900 learned_addr learned my addr 172.21.15.61:0/2832961900 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:33.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9fa9b9a700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 0x7f9fa4196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.071+0000 7f9faab9c700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 0x7f9fa419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 -- 172.21.15.61:0/2832961900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 msgr2=0x7f9fa419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 0x7f9fa419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 -- 172.21.15.61:0/2832961900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 msgr2=0x7f9fa4196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 0x7f9fa4196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 -- 172.21.15.61:0/2832961900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00095f0 con 0x7f9fa4100740 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9fa9b9a700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 0x7f9fa4196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faab9c700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 0x7f9fa419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9faa39b700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00098e0 tx=0x7f9fa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:33.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa001d070 con 0x7f9fa4100740 2023-10-18T09:16:33.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fa0022470 con 0x7f9fa4100740 2023-10-18T09:16:33.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0017650 con 0x7f9fa4100740 2023-10-18T09:16:33.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa419e740 con 0x7f9fa4100740 2023-10-18T09:16:33.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.072+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa419eb10 con 0x7f9fa4100740 2023-10-18T09:16:33.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.073+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa4107c10 con 0x7f9fa4100740 2023-10-18T09:16:33.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.075+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9fa0022870 con 0x7f9fa4100740 2023-10-18T09:16:33.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.077+0000 7f9f9b7fe700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 0x7f9f900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.077+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9fa00ce4e0 con 0x7f9fa4100740 2023-10-18T09:16:33.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.077+0000 7f9fa9b9a700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 0x7f9f900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.077+0000 7f9fa9b9a700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 0x7f9f900a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009cb0 tx=0x7f9f94009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:33.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.080+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fa0097980 con 0x7f9fa4100740 2023-10-18T09:16:33.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:32 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T09:16:33.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:32 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:33.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:32 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2687958912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:32 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1121830551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:32 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T09:16:33.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:32 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:33.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:32 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2687958912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:32 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1121830551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:32 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/422665631' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-18T09:16:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:32 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:32 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2687958912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:32 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1121830551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:33.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.295+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa4067530 con 0x7f9fa4100740 2023-10-18T09:16:33.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.297+0000 7f9f9b7fe700 1 -- 172.21.15.61:0/2832961900 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f9fa009afa0 con 0x7f9fa4100740 2023-10-18T09:16:33.299 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:33.300 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 msgr2=0x7f9f900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 0x7f9f900a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009cb0 tx=0x7f9f94009260 comp rx=0 tx=0).stop 2023-10-18T09:16:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 msgr2=0x7f9fa41966c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00098e0 tx=0x7f9fa00057f0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 shutdown_connections 2023-10-18T09:16:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9f900a1480 0x7f9f900a3940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fa41022d0 0x7f9fa419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9fa4100740 0x7f9fa41966c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 --2- 172.21.15.61:0/2832961900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9fa4101930 0x7f9fa4196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 >> 172.21.15.61:0/2832961900 conn(0x7f9fa40fbc90 msgr2=0x7f9fa4104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:33.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.299+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 shutdown_connections 2023-10-18T09:16:33.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.300+0000 7f9fac5ff700 1 -- 172.21.15.61:0/2832961900 wait complete. 2023-10-18T09:16:33.336 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:16:33.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 -- 172.21.15.61:0/1901996956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81181022d0 msgr2=0x7f8118108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 --2- 172.21.15.61:0/1901996956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81181022d0 0x7f8118108b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f811000b0d0 tx=0x7f811000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 -- 172.21.15.61:0/1901996956 shutdown_connections 2023-10-18T09:16:33.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 --2- 172.21.15.61:0/1901996956 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f81181022d0 0x7f8118108b80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 --2- 172.21.15.61:0/1901996956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8118100740 0x7f8118100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 --2- 172.21.15.61:0/1901996956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 0x7f8118101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.538+0000 7f81203ff700 1 -- 172.21.15.61:0/1901996956 >> 172.21.15.61:0/1901996956 conn(0x7f81180fbc90 msgr2=0x7f81180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:33.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.539+0000 7f81203ff700 1 -- 172.21.15.61:0/1901996956 shutdown_connections 2023-10-18T09:16:33.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.539+0000 7f81203ff700 1 -- 172.21.15.61:0/1901996956 wait complete. 2023-10-18T09:16:33.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.539+0000 7f81203ff700 1 Processor -- start 2023-10-18T09:16:33.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 -- start start 2023-10-18T09:16:33.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 0x7f8118196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 0x7f811819dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819e2f0 con 0x7f8118100740 2023-10-18T09:16:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819e430 con 0x7f8118101930 2023-10-18T09:16:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f81203ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811819e5c0 con 0x7f81181022d0 2023-10-18T09:16:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f811e19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f811e19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55702/0 (socket says 172.21.15.61:55702) 2023-10-18T09:16:33.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f811e19b700 1 -- 172.21.15.61:0/898233586 learned_addr learned my addr 172.21.15.61:0/898233586 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:33.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f811d99a700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 0x7f8118196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.540+0000 7f811e99c700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 0x7f811819dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 -- 172.21.15.61:0/898233586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 msgr2=0x7f811819dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 0x7f811819dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 -- 172.21.15.61:0/898233586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 msgr2=0x7f8118196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 0x7f8118196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 -- 172.21.15.61:0/898233586 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009d00 con 0x7f8118100740 2023-10-18T09:16:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e99c700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 0x7f811819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f811e19b700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f811400e970 tx=0x7f811400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:33.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114009790 con 0x7f8118100740 2023-10-18T09:16:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811400eec0 con 0x7f8118100740 2023-10-18T09:16:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114003cf0 con 0x7f8118100740 2023-10-18T09:16:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811819e8a0 con 0x7f8118100740 2023-10-18T09:16:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.541+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811819edb0 con 0x7f8118100740 2023-10-18T09:16:33.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.542+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8118107af0 con 0x7f8118100740 2023-10-18T09:16:33.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.544+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f811400ccd0 con 0x7f8118100740 2023-10-18T09:16:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.545+0000 7f810f7fe700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 0x7f81040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.546+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8114014070 con 0x7f8118100740 2023-10-18T09:16:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.546+0000 7f811d99a700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 0x7f81040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.546+0000 7f811d99a700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 0x7f81040a39c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8108009c60 tx=0x7f8108009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:33.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.549+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8114096550 con 0x7f8118100740 2023-10-18T09:16:33.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.809+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8118067530 con 0x7f8118100740 2023-10-18T09:16:33.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.810+0000 7f810f7fe700 1 -- 172.21.15.61:0/898233586 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11270 (secure 0 0 0) 0x7f8114099b70 con 0x7f8118100740 2023-10-18T09:16:33.812 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:33.813 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14688,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24589,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24593,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24629,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24653,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24668,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:31.407216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-18T09:16:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 msgr2=0x7f81040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 0x7f81040a39c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8108009c60 tx=0x7f8108009260 comp rx=0 tx=0).stop 2023-10-18T09:16:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 msgr2=0x7f81181967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f811400e970 tx=0x7f811400ec80 comp rx=0 tx=0).stop 2023-10-18T09:16:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 shutdown_connections 2023-10-18T09:16:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f81040a1500 0x7f81040a39c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81181022d0 0x7f811819dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8118100740 0x7f81181967c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 --2- 172.21.15.61:0/898233586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8118101930 0x7f8118196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.812+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 >> 172.21.15.61:0/898233586 conn(0x7f81180fbc90 msgr2=0x7f8118104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.813+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 shutdown_connections 2023-10-18T09:16:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:33.813+0000 7f81203ff700 1 -- 172.21.15.61:0/898233586 wait complete. 2023-10-18T09:16:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 17 2023-10-18T09:16:33.834 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:34.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 -- 172.21.15.61:0/59063295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab0101930 msgr2=0x7fbab0101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 --2- 172.21.15.61:0/59063295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab0101930 0x7fbab0101d90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40099c0 tx=0x7fbaa4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 -- 172.21.15.61:0/59063295 shutdown_connections 2023-10-18T09:16:34.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 --2- 172.21.15.61:0/59063295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab01022d0 0x7fbab0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 --2- 172.21.15.61:0/59063295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0100740 0x7fbab0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 --2- 172.21.15.61:0/59063295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab0101930 0x7fbab0101d90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 -- 172.21.15.61:0/59063295 >> 172.21.15.61:0/59063295 conn(0x7fbab00fbc90 msgr2=0x7fbab00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 -- 172.21.15.61:0/59063295 shutdown_connections 2023-10-18T09:16:34.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.040+0000 7fbab69ff700 1 -- 172.21.15.61:0/59063295 wait complete. 2023-10-18T09:16:34.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 Processor -- start 2023-10-18T09:16:34.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 -- start start 2023-10-18T09:16:34.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 0x7fbab0196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0101930 0x7fbab0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab019e1c0 con 0x7fbab01022d0 2023-10-18T09:16:34.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab019e300 con 0x7fbab0100740 2023-10-18T09:16:34.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.041+0000 7fbab69ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab019e490 con 0x7fbab0101930 2023-10-18T09:16:34.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbaaf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0101930 0x7fbab0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbaaffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 0x7fbab0196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55718/0 (socket says 172.21.15.61:55718) 2023-10-18T09:16:34.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 -- 172.21.15.61:0/1105402962 learned_addr learned my addr 172.21.15.61:0/1105402962 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 -- 172.21.15.61:0/1105402962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0101930 msgr2=0x7fbab0196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0101930 0x7fbab0196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 -- 172.21.15.61:0/1105402962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 msgr2=0x7fbab0196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 0x7fbab0196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 -- 172.21.15.61:0/1105402962 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40095f0 con 0x7fbab01022d0 2023-10-18T09:16:34.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbaaffff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 0x7fbab0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:34.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab4f9c700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800ef30 tx=0x7fbaa800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8018070 con 0x7fbab01022d0 2023-10-18T09:16:34.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.042+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab0072a20 con 0x7fbab01022d0 2023-10-18T09:16:34.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.043+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaa800f040 con 0x7fbab01022d0 2023-10-18T09:16:34.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.043+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800cd00 con 0x7fbab01022d0 2023-10-18T09:16:34.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.043+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab0072f30 con 0x7fbab01022d0 2023-10-18T09:16:34.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.044+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab0107c10 con 0x7fbab01022d0 2023-10-18T09:16:34.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.046+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbaa8003a00 con 0x7fbab01022d0 2023-10-18T09:16:34.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.047+0000 7fbaad7fa700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 0x7fba980a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.047+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbaa80cca10 con 0x7fbab01022d0 2023-10-18T09:16:34.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.047+0000 7fbaaffff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 0x7fba980a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.048+0000 7fbaaffff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 0x7fba980a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbab0101790 tx=0x7fbaa0007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.051+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbaa8095eb0 con 0x7fbab01022d0 2023-10-18T09:16:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:33 smithi110 ceph-mon[134818]: pgmap v294: 129 pgs: 8 creating+peering, 105 active+clean, 16 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-10-18T09:16:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:33 smithi110 ceph-mon[134818]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T09:16:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:33 smithi110 ceph-mon[134818]: Cluster is now healthy 2023-10-18T09:16:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:33 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2832961900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:33 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/898233586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:33 smithi061 ceph-mon[98573]: pgmap v294: 129 pgs: 8 creating+peering, 105 active+clean, 16 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-10-18T09:16:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:33 smithi061 ceph-mon[98573]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T09:16:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:33 smithi061 ceph-mon[98573]: Cluster is now healthy 2023-10-18T09:16:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:33 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2832961900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:33 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/898233586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.265+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab0067500 con 0x7fbab01022d0 2023-10-18T09:16:34.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.266+0000 7fbaad7fa700 1 -- 172.21.15.61:0/1105402962 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7fbaa80994d0 con 0x7fbab01022d0 2023-10-18T09:16:34.269 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:34.269 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:34.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.268+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 msgr2=0x7fba980a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.268+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 0x7fba980a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbab0101790 tx=0x7fbaa0007400 comp rx=0 tx=0).stop 2023-10-18T09:16:34.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.268+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 msgr2=0x7fbab019dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.268+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800ef30 tx=0x7fbaa800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 shutdown_connections 2023-10-18T09:16:34.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fba980a1480 0x7fba980a3940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbab01022d0 0x7fbab019dbf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbab0100740 0x7fbab0196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 --2- 172.21.15.61:0/1105402962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbab0101930 0x7fbab0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 >> 172.21.15.61:0/1105402962 conn(0x7fbab00fbc90 msgr2=0x7fbab0104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 shutdown_connections 2023-10-18T09:16:34.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.269+0000 7fbab69ff700 1 -- 172.21.15.61:0/1105402962 wait complete. 2023-10-18T09:16:34.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-18T09:16:26.396655+0000', 'modified': '2023-10-18T09:16:31.407216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24692}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24692': {'gid': 24692, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6839/1483417844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 1483417844}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 1483417844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-18T09:16:34.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-18T09:16:34.292 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T09:16:34.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:33 smithi170 ceph-mon[135150]: pgmap v294: 129 pgs: 8 creating+peering, 105 active+clean, 16 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 10 op/s 2023-10-18T09:16:34.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:33 smithi170 ceph-mon[135150]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-18T09:16:34.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:33 smithi170 ceph-mon[135150]: Cluster is now healthy 2023-10-18T09:16:34.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:33 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2832961900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:33 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/898233586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:16:34.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.494+0000 7efe4f7ff700 1 -- 172.21.15.61:0/3804004321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 msgr2=0x7efe481035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.494+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/3804004321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe481035f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efe44009990 tx=0x7efe44009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 -- 172.21.15.61:0/3804004321 shutdown_connections 2023-10-18T09:16:34.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/3804004321 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 0x7efe48108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/3804004321 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe481035f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/3804004321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 0x7efe48105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 -- 172.21.15.61:0/3804004321 >> 172.21.15.61:0/3804004321 conn(0x7efe480fab90 msgr2=0x7efe480fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 -- 172.21.15.61:0/3804004321 shutdown_connections 2023-10-18T09:16:34.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.495+0000 7efe4f7ff700 1 -- 172.21.15.61:0/3804004321 wait complete. 2023-10-18T09:16:34.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.496+0000 7efe4f7ff700 1 Processor -- start 2023-10-18T09:16:34.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.496+0000 7efe4f7ff700 1 -- start start 2023-10-18T09:16:34.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 0x7efe48198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 0x7efe4819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481a03c0 con 0x7efe48101200 2023-10-18T09:16:34.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481a0500 con 0x7efe48106460 2023-10-18T09:16:34.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4f7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe481a0690 con 0x7efe48103b30 2023-10-18T09:16:34.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55740/0 (socket says 172.21.15.61:55740) 2023-10-18T09:16:34.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 -- 172.21.15.61:0/1017378782 learned_addr learned my addr 172.21.15.61:0/1017378782 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 0x7efe48198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 0x7efe4819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 -- 172.21.15.61:0/1017378782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 msgr2=0x7efe48198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 0x7efe48198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 -- 172.21.15.61:0/1017378782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 msgr2=0x7efe4819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.497+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 0x7efe4819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe4d59b700 1 -- 172.21.15.61:0/1017378782 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440095f0 con 0x7efe48101200 2023-10-18T09:16:34.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe44006010 tx=0x7efe44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe3e7fc700 1 -- 172.21.15.61:0/1017378782 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4401d070 con 0x7efe48101200 2023-10-18T09:16:34.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe3e7fc700 1 -- 172.21.15.61:0/1017378782 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe44005030 con 0x7efe48101200 2023-10-18T09:16:34.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe3e7fc700 1 -- 172.21.15.61:0/1017378782 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44004b40 con 0x7efe48101200 2023-10-18T09:16:34.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe481a0910 con 0x7efe48101200 2023-10-18T09:16:34.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.498+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe481a0ce0 con 0x7efe48101200 2023-10-18T09:16:34.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.501+0000 7efe3e7fc700 1 -- 172.21.15.61:0/1017378782 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efe44004d40 con 0x7efe48101200 2023-10-18T09:16:34.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 Processor -- start 2023-10-18T09:16:34.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe3e7fc700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 0x7efe340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 -- start start 2023-10-18T09:16:34.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe3e7fc700 1 -- 172.21.15.61:0/1017378782 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efe440cdf90 con 0x7efe48101200 2023-10-18T09:16:34.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 0x7efe340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 0x55b5ef0416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041be0 0x55b5ef040040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eefb3690 con 0x55b5ef040580 2023-10-18T09:16:34.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eef99c10 con 0x55b5ef041280 2023-10-18T09:16:34.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe630a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eef97530 con 0x55b5ef041be0 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.503+0000 7efe4dd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55746/0 (socket says 172.21.15.61:55746) 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 -- 172.21.15.61:0/3369528271 learned_addr learned my addr 172.21.15.61:0/3369528271 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4d59b700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 0x55b5ef0416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041be0 0x55b5ef040040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 -- 172.21.15.61:0/3369528271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041be0 msgr2=0x55b5ef040040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041be0 0x55b5ef040040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 -- 172.21.15.61:0/3369528271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 msgr2=0x55b5ef0416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 0x55b5ef0416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 0x7efe340a39c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe38009cb0 tx=0x7efe38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 -- 172.21.15.61:0/3369528271 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5eef503a0 con 0x55b5ef040580 2023-10-18T09:16:34.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4d59b700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 0x55b5ef0416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:34.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe400098f0 tx=0x7efe40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569b35e17571571e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.504+0000 7efe337fe700 1 -- 172.21.15.61:0/3369528271 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40019070 con 0x55b5ef040580 2023-10-18T09:16:34.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe337fe700 1 -- 172.21.15.61:0/3369528271 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe4000f040 con 0x55b5ef040580 2023-10-18T09:16:34.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe337fe700 1 -- 172.21.15.61:0/3369528271 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40007a60 con 0x55b5ef040580 2023-10-18T09:16:34.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 -- 172.21.15.61:0/3369528271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 msgr2=0x55b5ef040a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efe400098f0 tx=0x7efe40009c00 comp rx=0 tx=0).stop 2023-10-18T09:16:34.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 -- 172.21.15.61:0/3369528271 shutdown_connections 2023-10-18T09:16:34.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef040580 0x55b5ef040a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041280 0x55b5ef0416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 --2- 172.21.15.61:0/3369528271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041be0 0x55b5ef040040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 -- 172.21.15.61:0/3369528271 >> 172.21.15.61:0/3369528271 conn(0x55b5eefb46d0 msgr2=0x55b5ef03ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 -- 172.21.15.61:0/3369528271 shutdown_connections 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.505+0000 7efe630a9b80 1 -- 172.21.15.61:0/3369528271 wait complete. 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.506+0000 7efe630a9b80 1 -- start start 2023-10-18T09:16:34.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.506+0000 7efe630a9b80 1 Processor -- start 2023-10-18T09:16:34.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.506+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.506+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef049fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef051000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe630a9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eef92e20 con 0x55b5ef041be0 2023-10-18T09:16:34.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe630a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eef2f6c0 con 0x55b5ef040580 2023-10-18T09:16:34.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe630a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eef2f360 con 0x55b5ef041280 2023-10-18T09:16:34.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe630a9b80 10 client.0 _ll_register_callbacks cb 0x55b5eee79a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:16:34.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4d59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef051000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4d59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43354/0 (socket says 172.21.15.61:43354) 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4d59b700 1 -- 172.21.15.61:0/1261458250 learned_addr learned my addr 172.21.15.61:0/1261458250 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef049fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 -- 172.21.15.61:0/1261458250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 msgr2=0x55b5ef049fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef049fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 -- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 msgr2=0x55b5ef049a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4dd9c700 1 -- 172.21.15.61:0/1261458250 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe481a0ce0 con 0x55b5ef041be0 2023-10-18T09:16:34.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.507+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:34.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef051000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe40003bf0 tx=0x7efe40004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d716bcf6cf6b51c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:16:34.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40023070 con 0x55b5ef041be0 2023-10-18T09:16:34.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5eef97ea0 con 0x55b5ef041be0 2023-10-18T09:16:34.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efe40007a50 con 0x55b5ef041be0 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe40019040 con 0x55b5ef041be0 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe630a9b80 10 client.14805 fetch_fsmap learned FSMap version 17 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7efe40003c50 con 0x55b5ef041be0 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.508+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5eef97ea0 con 0x55b5ef041be0 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efe40026020 con 0x55b5ef041be0 2023-10-18T09:16:34.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 10 client.14805 fetch_fsmap finished waiting for FSMap version 17 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 10 client.14805 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 20 client.14805 populate_metadata read hostname 'smithi061' 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0555e0 0x55b5ef0540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 4 client.14805 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5ef00f860 con 0x55b5ef0555e0 2023-10-18T09:16:34.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.509+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0555e0 0x55b5ef0540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.510+0000 7efe4d59b700 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0555e0 0x55b5ef0540f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a2c41c43684e04 server_cookie=2b3efe1f3c40019e in_seq=0 out_seq=0 2023-10-18T09:16:34.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.510+0000 7efe30ff9700 10 client.14805 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:16:34.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe30ff9700 1 -- 172.21.15.61:0/1261458250 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b5ef00f860 con 0x55b5ef0555e0 2023-10-18T09:16:34.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe30ff9700 10 client.14805 handle_command_reply: tid=0 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 client.14805 shutdown 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 msgr2=0x55b5ef051000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef051000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe40003bf0 tx=0x7efe40004780 comp rx=0 tx=0).stop 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 shutdown_connections 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0555e0 0x55b5ef0540f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef051000 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef049a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 --2- 172.21.15.61:0/1261458250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef049fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe30ff9700 0 client.14805 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:16:34.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 >> 172.21.15.61:0/1261458250 conn(0x55b5eefb46d0 msgr2=0x55b5ef03aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 shutdown_connections 2023-10-18T09:16:34.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 1 -- 172.21.15.61:0/1261458250 wait complete. 2023-10-18T09:16:34.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.511+0000 7efe630a9b80 20 client.14805 trim_cache size 0 max 16384 2023-10-18T09:16:34.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.532+0000 7efe630a9b80 1 Processor -- start 2023-10-18T09:16:34.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.532+0000 7efe630a9b80 1 -- start start 2023-10-18T09:16:34.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef040040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef050f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5eee138d0 con 0x55b5ef041be0 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ef04a980 con 0x55b5ef040580 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe630a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ef00f860 con 0x55b5ef041280 2023-10-18T09:16:34.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55764/0 (socket says 172.21.15.61:55764) 2023-10-18T09:16:34.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 -- 172.21.15.61:0/4017468501 learned_addr learned my addr 172.21.15.61:0/4017468501 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef050f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef040040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 -- 172.21.15.61:0/4017468501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 msgr2=0x55b5ef050f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef050f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 -- 172.21.15.61:0/4017468501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 msgr2=0x55b5ef040040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef040040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4dd9c700 1 -- 172.21.15.61:0/4017468501 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5eedbdd90 con 0x55b5ef041be0 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.533+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef040040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:34.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.534+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe400094d0 tx=0x7efe400097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4917b2bfc152fb6d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.534+0000 7efe33fff700 1 -- 172.21.15.61:0/4017468501 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40023070 con 0x55b5ef041be0 2023-10-18T09:16:34.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.534+0000 7efe33fff700 1 -- 172.21.15.61:0/4017468501 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe40015940 con 0x55b5ef041be0 2023-10-18T09:16:34.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.534+0000 7efe33fff700 1 -- 172.21.15.61:0/4017468501 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40005560 con 0x55b5ef041be0 2023-10-18T09:16:34.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- 172.21.15.61:0/4017468501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 msgr2=0x55b5ef03aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe400094d0 tx=0x7efe400097e0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- 172.21.15.61:0/4017468501 shutdown_connections 2023-10-18T09:16:34.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041be0 0x55b5ef03aad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef040580 0x55b5ef040040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4017468501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef041280 0x55b5ef050f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- 172.21.15.61:0/4017468501 >> 172.21.15.61:0/4017468501 conn(0x55b5eefb46d0 msgr2=0x55b5ef040960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- 172.21.15.61:0/4017468501 shutdown_connections 2023-10-18T09:16:34.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- 172.21.15.61:0/4017468501 wait complete. 2023-10-18T09:16:34.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 -- start start 2023-10-18T09:16:34.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.535+0000 7efe630a9b80 1 Processor -- start 2023-10-18T09:16:34.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 0x55b5ef034c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041be0 0x55b5ef0504a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ef050bb0 con 0x55b5ef041280 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ef050d20 con 0x55b5ef041be0 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ef050ee0 con 0x55b5ef040580 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe630a9b80 10 client.0 _ll_register_callbacks cb 0x55b5eee79a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:16:34.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55776/0 (socket says 172.21.15.61:55776) 2023-10-18T09:16:34.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 -- 172.21.15.61:0/4163558351 learned_addr learned my addr 172.21.15.61:0/4163558351 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:34.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 0x55b5ef034c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4dd9c700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041be0 0x55b5ef0504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 -- 172.21.15.61:0/4163558351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 msgr2=0x55b5ef034c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 0x55b5ef034c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 -- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041be0 msgr2=0x55b5ef0504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:16:34.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.536+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041be0 0x55b5ef0504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe4cd9a700 1 -- 172.21.15.61:0/4163558351 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe40003c50 con 0x55b5ef041280 2023-10-18T09:16:34.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 0x55b5ef034c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:34.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe4cd9a700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe3801da20 tx=0x7efe380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a218eba19bec8ff server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:34.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:16:34.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38021030 con 0x55b5ef041280 2023-10-18T09:16:34.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe4000f040 con 0x55b5ef041280 2023-10-18T09:16:34.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efe3801bad0 con 0x55b5ef041280 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe3801bc90 con 0x55b5ef041280 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe630a9b80 10 client.14811 fetch_fsmap learned FSMap version 17 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.537+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55b5eeda6c10 con 0x55b5ef041280 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe4000f040 con 0x55b5ef041280 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7efe38004d10 con 0x55b5ef041280 2023-10-18T09:16:34.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 10 client.14811 fetch_fsmap finished waiting for FSMap version 17 2023-10-18T09:16:34.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 10 client.14811 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:16:34.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 20 client.14811 populate_metadata read hostname 'smithi061' 2023-10-18T09:16:34.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0adf00 0x55b5ef0aa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:34.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 4 client.14811 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:16:34.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5eef8be00 con 0x55b5ef0adf00 2023-10-18T09:16:34.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.538+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0adf00 0x55b5ef0aa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:34.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe4d59b700 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0adf00 0x55b5ef0aa340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513de225269da7a0 server_cookie=c4b0362706a8f684 in_seq=0 out_seq=0 2023-10-18T09:16:34.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe30ff9700 10 client.14811 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:16:34.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe30ff9700 1 -- 172.21.15.61:0/4163558351 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55b5eef8be00 con 0x55b5ef0adf00 2023-10-18T09:16:34.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe30ff9700 10 client.14811 handle_command_reply: tid=0 2023-10-18T09:16:34.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 client.14811 shutdown 2023-10-18T09:16:34.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 msgr2=0x55b5ef04ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efe3801da20 tx=0x7efe380047c0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 shutdown_connections 2023-10-18T09:16:34.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b5ef0adf00 0x55b5ef0aa340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5ef041be0 0x55b5ef0504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b5ef040580 0x55b5ef034c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe30ff9700 0 client.14811 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 --2- 172.21.15.61:0/4163558351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b5ef041280 0x55b5ef04ff60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 >> 172.21.15.61:0/4163558351 conn(0x55b5eefb46d0 msgr2=0x55b5ef03aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 shutdown_connections 2023-10-18T09:16:34.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.539+0000 7efe630a9b80 1 -- 172.21.15.61:0/4163558351 wait complete. 2023-10-18T09:16:34.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe630a9b80 20 client.14811 trim_cache size 0 max 16384 2023-10-18T09:16:34.556 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:16:34.556 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:16:34.557 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 0, 2023-10-18T09:16:34.557 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24692, 2023-10-18T09:16:34.557 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:16:34.557 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:16:34.557 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 7.6914398950000002, 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 17, 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 88, 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 14.331828464000001 2023-10-18T09:16:34.558 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:16:34.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 msgr2=0x7efe340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 0x7efe340a39c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efe38009cb0 tx=0x7efe38009260 comp rx=0 tx=0).stop 2023-10-18T09:16:34.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 msgr2=0x7efe48198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:34.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe44006010 tx=0x7efe44004730 comp rx=0 tx=0).stop 2023-10-18T09:16:34.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 shutdown_connections 2023-10-18T09:16:34.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efe340a1500 0x7efe340a39c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe48106460 0x7efe4819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efe48101200 0x7efe48198890 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 --2- 172.21.15.61:0/1017378782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe48103b30 0x7efe48198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 >> 172.21.15.61:0/1017378782 conn(0x7efe480fab90 msgr2=0x7efe48100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 shutdown_connections 2023-10-18T09:16:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:34.540+0000 7efe4f7ff700 1 -- 172.21.15.61:0/1017378782 wait complete. 2023-10-18T09:16:34.562 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T09:16:34.563 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:34.564 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-18T09:16:35.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:34 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1105402962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:35.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1105402962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:35.304 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:16:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:34 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1105402962' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:36.162 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:35 smithi061 ceph-mon[98573]: pgmap v295: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-10-18T09:16:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:35 smithi110 ceph-mon[134818]: pgmap v295: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-10-18T09:16:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:35 smithi170 ceph-mon[135150]: pgmap v295: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2023-10-18T09:16:36.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 -- 172.21.15.61:0/1604658099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd0101260 msgr2=0x7fddd010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:36.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 --2- 172.21.15.61:0/1604658099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd0101260 0x7fddd010d650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b0a0 tx=0x7fddc800b3b0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 -- 172.21.15.61:0/1604658099 shutdown_connections 2023-10-18T09:16:36.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 --2- 172.21.15.61:0/1604658099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd0101260 0x7fddd010d650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 --2- 172.21.15.61:0/1604658099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01057c0 0x7fddd0105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 --2- 172.21.15.61:0/1604658099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd01008a0 0x7fddd0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 -- 172.21.15.61:0/1604658099 >> 172.21.15.61:0/1604658099 conn(0x7fddd00779e0 msgr2=0x7fddd0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:36.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 -- 172.21.15.61:0/1604658099 shutdown_connections 2023-10-18T09:16:36.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.781+0000 7fddd8271700 1 -- 172.21.15.61:0/1604658099 wait complete. 2023-10-18T09:16:36.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 Processor -- start 2023-10-18T09:16:36.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 -- start start 2023-10-18T09:16:36.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:36.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0101260 0x7fddd0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:36.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 0x7fddd0073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:36.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a5050 con 0x7fddd01057c0 2023-10-18T09:16:36.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a51c0 con 0x7fddd01008a0 2023-10-18T09:16:36.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.782+0000 7fddd8271700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01a5380 con 0x7fddd0101260 2023-10-18T09:16:36.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd600d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:36.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 0x7fddd0073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:36.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd600d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43396/0 (socket says 172.21.15.61:43396) 2023-10-18T09:16:36.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd600d700 1 -- 172.21.15.61:0/2950927281 learned_addr learned my addr 172.21.15.61:0/2950927281 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:36.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd580c700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0101260 0x7fddd0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:36.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 -- 172.21.15.61:0/2950927281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0101260 msgr2=0x7fddd0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:36.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0101260 0x7fddd0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 -- 172.21.15.61:0/2950927281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 msgr2=0x7fddd0072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:36.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:36.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 -- 172.21.15.61:0/2950927281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc8009d00 con 0x7fddd01057c0 2023-10-18T09:16:36.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd600d700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:36.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.783+0000 7fddd680e700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 0x7fddd0073570 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b070 tx=0x7fddc8007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:36.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.784+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc800e040 con 0x7fddd01057c0 2023-10-18T09:16:36.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.784+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fddc801d470 con 0x7fddd01057c0 2023-10-18T09:16:36.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.784+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd01a5600 con 0x7fddd01057c0 2023-10-18T09:16:36.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.784+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8005280 con 0x7fddd01057c0 2023-10-18T09:16:36.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.784+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd01a5a20 con 0x7fddd01057c0 2023-10-18T09:16:36.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.785+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0101f30 con 0x7fddd01057c0 2023-10-18T09:16:36.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.787+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fddc801d5e0 con 0x7fddd01057c0 2023-10-18T09:16:36.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.788+0000 7fddc77fe700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 0x7fddbc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:36.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.788+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fddc80ce250 con 0x7fddd01057c0 2023-10-18T09:16:36.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.788+0000 7fddd600d700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 0x7fddbc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:36.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.789+0000 7fddd600d700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 0x7fddbc0a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fddcc005950 tx=0x7fddcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:36.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:36.792+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fddc80976f0 con 0x7fddd01057c0 2023-10-18T09:16:37.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.050+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fddd0110780 con 0x7fddd01057c0 2023-10-18T09:16:37.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.051+0000 7fddc77fe700 1 -- 172.21.15.61:0/2950927281 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fddc809ad10 con 0x7fddd01057c0 2023-10-18T09:16:37.053 INFO:teuthology.orchestra.run.smithi061.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T09:16:37.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 msgr2=0x7fddbc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:37.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 0x7fddbc0a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fddcc005950 tx=0x7fddcc009260 comp rx=0 tx=0).stop 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 msgr2=0x7fddd0073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 0x7fddd0073570 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b070 tx=0x7fddc8007860 comp rx=0 tx=0).stop 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 shutdown_connections 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fddbc0a1480 0x7fddbc0a3940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fddd01057c0 0x7fddd0073570 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:37.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddd01008a0 0x7fddd0072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:37.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 --2- 172.21.15.61:0/2950927281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0101260 0x7fddd0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:37.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 >> 172.21.15.61:0/2950927281 conn(0x7fddd00779e0 msgr2=0x7fddd00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:37.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 shutdown_connections 2023-10-18T09:16:37.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:37.053+0000 7fddd8271700 1 -- 172.21.15.61:0/2950927281 wait complete. 2023-10-18T09:16:37.490 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-18T09:16:37.490 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:16:37.490 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi061.front.sepia.ceph.com 2023-10-18T09:16:37.491 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-18T09:16:37.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:37.491 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-18T09:16:37.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T09:16:37.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:16:37.492 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T09:16:37.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:37.493 DEBUG:teuthology.orchestra.run.smithi061:> ip addr 2023-10-18T09:16:37.540 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:16:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:16:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:16:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:16:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-10-18T09:16:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:16:37.541 INFO:teuthology.orchestra.run.smithi061.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:16:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0c brd ff:ff:ff:ff:ff:ff 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: altname enp7s0f0 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0d brd ff:ff:ff:ff:ff:ff 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: altname enp7s0f1 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:16:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b8 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b9 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: inet 172.21.15.61/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft 41689sec preferred_lft 41689sec 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::e081:b724:74b9:7104/64 scope link noprefixroute 2023-10-18T09:16:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:16:37.544 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T09:16:37.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:37.545 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:16:37.545 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:16:37.546 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add name ceph-brx type bridge 2023-10-18T09:16:37.546 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr flush dev ceph-brx 2023-10-18T09:16:37.546 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set ceph-brx up 2023-10-18T09:16:37.547 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T09:16:37.547 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:16:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:37 smithi061 ceph-mon[98573]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-10-18T09:16:37.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:37 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2950927281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:16:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:37 smithi170 ceph-mon[135150]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-10-18T09:16:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:37 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2950927281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:16:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:37 smithi110 ceph-mon[134818]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2023-10-18T09:16:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:37 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2950927281' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:16:37.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:37.986 DEBUG:teuthology.orchestra.run.smithi061:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:16:38.059 INFO:teuthology.orchestra.run.smithi061.stdout:1 2023-10-18T09:16:38.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:38.063 DEBUG:teuthology.orchestra.run.smithi061:> route 2023-10-18T09:16:38.128 INFO:teuthology.orchestra.run.smithi061.stdout:Kernel IP routing table 2023-10-18T09:16:38.129 INFO:teuthology.orchestra.run.smithi061.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:16:38.129 INFO:teuthology.orchestra.run.smithi061.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:16:38.129 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:16:38.129 INFO:teuthology.orchestra.run.smithi061.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:16:38.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:38.131 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:16:38.132 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:16:38.132 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:16:38.132 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:16:38.133 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:16:38.133 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:16:38.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:38.809 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-10-18T09:16:38.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:38.855 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list-id 2023-10-18T09:16:38.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:38.944 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:16:38.945 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:16:38.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:16:38.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-18T09:16:38.946 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:16:39.093 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-18T09:16:39.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:39.094 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:16:39.094 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:16:39.094 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-18T09:16:39.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T09:16:39.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-18T09:16:39.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-18T09:16:39.096 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-18T09:16:39.096 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:16:39.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:39.388 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:16:39.389 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:16:39.389 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set brx.0 up 2023-10-18T09:16:39.389 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T09:16:39.390 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:16:39.567 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-18T09:16:39.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:16:39.568 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:39.612 INFO:teuthology.orchestra.run.smithi061.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-18T09:16:39.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:16:39.612 DEBUG:teuthology.orchestra.run.smithi061:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:39.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-10-18T09:16:39.807 DEBUG:teuthology.orchestra.run.smithi061:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T09:16:39.853 INFO:teuthology.orchestra.run.smithi061.stdout:/sys 2023-10-18T09:16:39.853 INFO:teuthology.orchestra.run.smithi061.stdout:/proc 2023-10-18T09:16:39.853 INFO:teuthology.orchestra.run.smithi061.stdout:/dev 2023-10-18T09:16:39.853 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/security 2023-10-18T09:16:39.854 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/shm 2023-10-18T09:16:39.854 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/pts 2023-10-18T09:16:39.854 INFO:teuthology.orchestra.run.smithi061.stdout:/run 2023-10-18T09:16:39.854 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup 2023-10-18T09:16:39.854 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/systemd 2023-10-18T09:16:39.855 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/pstore 2023-10-18T09:16:39.855 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-18T09:16:39.855 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/cpuset 2023-10-18T09:16:39.855 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-18T09:16:39.855 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/hugetlb 2023-10-18T09:16:39.856 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/memory 2023-10-18T09:16:39.856 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/pids 2023-10-18T09:16:39.856 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/perf_event 2023-10-18T09:16:39.856 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/devices 2023-10-18T09:16:39.856 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/blkio 2023-10-18T09:16:39.857 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/freezer 2023-10-18T09:16:39.857 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/tracing 2023-10-18T09:16:39.857 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/config 2023-10-18T09:16:39.857 INFO:teuthology.orchestra.run.smithi061.stdout:/ 2023-10-18T09:16:39.857 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/selinux 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/mqueue 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/hugepages 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/debug 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/fuse/connections 2023-10-18T09:16:39.858 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T09:16:39.859 INFO:teuthology.orchestra.run.smithi061.stdout:/run/user/1000 2023-10-18T09:16:39.859 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/ceph 2023-10-18T09:16:39.859 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay 2023-10-18T09:16:39.859 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/e33befa65d2c4303106dad032df09fdc7bd5c2ac09762877fe4d1c0dccbcb320/merged 2023-10-18T09:16:39.859 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/a17f708cbc5d115fe53af034ce97d2b1956e4a554559e15260f365a4aab39e3e/merged 2023-10-18T09:16:39.860 INFO:teuthology.orchestra.run.smithi061.stdout:/run/user/0 2023-10-18T09:16:39.860 INFO:teuthology.orchestra.run.smithi061.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T09:16:39.860 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/8593a1638b83b829bd73b9a1c5da1eaa6a0932e74cc8562ac5ca26ea9405fa12/merged 2023-10-18T09:16:39.860 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/566e43660e99231e261829e6127a91a101ee13be2b3d690af95dae04a88462d6/merged 2023-10-18T09:16:39.860 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/2b6739b09efc3fe0985d2ed74e8703f5c87283881611279c9e0cafaf53b405b0/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/21f3cfb88dc721328cc64c2023126d83609639df7162b8e263970d6ecd24d75a/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/86a567c9012d8fa141815ae13369639b7d60d2307675a099b3df2956a8463b16/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/35011c6cae15a9284b32d1457cb3d59f844134fd225100a281b010985e21eecd/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/0610bb67148c5c8564eab5d009889424111c6ade0fc1cd67d8be796ef8581ca1/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/04d5220655a032b165e26267146fed967509be99b305afd81e9876aa4ad52ad3/merged 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns 2023-10-18T09:16:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:16:39.862 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:16:39.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:39.863 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-10-18T09:16:39.908 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-18T09:16:39.908 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-18T09:16:39.952 DEBUG:teuthology.orchestra.run.smithi061:> sudo modprobe fuse 2023-10-18T09:16:40.025 DEBUG:teuthology.orchestra.run.smithi061:> cat /proc/self/mounts | awk '{print $2}' 2023-10-18T09:16:40.072 INFO:teuthology.orchestra.run.smithi061.stdout:/sys 2023-10-18T09:16:40.072 INFO:teuthology.orchestra.run.smithi061.stdout:/proc 2023-10-18T09:16:40.072 INFO:teuthology.orchestra.run.smithi061.stdout:/dev 2023-10-18T09:16:40.072 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/security 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/shm 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/pts 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/run 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/systemd 2023-10-18T09:16:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/pstore 2023-10-18T09:16:40.074 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-18T09:16:40.074 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/cpuset 2023-10-18T09:16:40.074 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-18T09:16:40.074 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/hugetlb 2023-10-18T09:16:40.074 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/memory 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/pids 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/perf_event 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/devices 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/blkio 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/cgroup/freezer 2023-10-18T09:16:40.075 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/tracing 2023-10-18T09:16:40.076 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/config 2023-10-18T09:16:40.076 INFO:teuthology.orchestra.run.smithi061.stdout:/ 2023-10-18T09:16:40.076 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/selinux 2023-10-18T09:16:40.076 INFO:teuthology.orchestra.run.smithi061.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T09:16:40.076 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/mqueue 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/dev/hugepages 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/kernel/debug 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/sys/fs/fuse/connections 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/nfs/rpc_pipefs 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/run/user/1000 2023-10-18T09:16:40.077 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/ceph 2023-10-18T09:16:40.078 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay 2023-10-18T09:16:40.078 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/e33befa65d2c4303106dad032df09fdc7bd5c2ac09762877fe4d1c0dccbcb320/merged 2023-10-18T09:16:40.078 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/a17f708cbc5d115fe53af034ce97d2b1956e4a554559e15260f365a4aab39e3e/merged 2023-10-18T09:16:40.078 INFO:teuthology.orchestra.run.smithi061.stdout:/run/user/0 2023-10-18T09:16:40.078 INFO:teuthology.orchestra.run.smithi061.stdout:/proc/sys/fs/binfmt_misc 2023-10-18T09:16:40.079 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/8593a1638b83b829bd73b9a1c5da1eaa6a0932e74cc8562ac5ca26ea9405fa12/merged 2023-10-18T09:16:40.079 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/566e43660e99231e261829e6127a91a101ee13be2b3d690af95dae04a88462d6/merged 2023-10-18T09:16:40.079 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/2b6739b09efc3fe0985d2ed74e8703f5c87283881611279c9e0cafaf53b405b0/merged 2023-10-18T09:16:40.079 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/21f3cfb88dc721328cc64c2023126d83609639df7162b8e263970d6ecd24d75a/merged 2023-10-18T09:16:40.079 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/86a567c9012d8fa141815ae13369639b7d60d2307675a099b3df2956a8463b16/merged 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/35011c6cae15a9284b32d1457cb3d59f844134fd225100a281b010985e21eecd/merged 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/0610bb67148c5c8564eab5d009889424111c6ade0fc1cd67d8be796ef8581ca1/merged 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/var/lib/containers/storage/overlay/04d5220655a032b165e26267146fed967509be99b305afd81e9876aa4ad52ad3/merged 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:16:40.080 INFO:teuthology.orchestra.run.smithi061.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:16:40.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:40.082 DEBUG:teuthology.orchestra.run.smithi061:> ls /sys/fs/fuse/connections 2023-10-18T09:16:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:39 smithi170 ceph-mon[135150]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-10-18T09:16:40.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi061.stderr:2023-10-18T09:16:40.151+0000 7f55ce1385c0 -1 init, newargv = 0x55e1ad693a40 newargc=15 2023-10-18T09:16:40.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi061.stderr:ceph-fuse[153707]: starting ceph client 2023-10-18T09:16:40.164 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi061.stderr:ceph-fuse[153707]: starting fuse 2023-10-18T09:16:40.170 INFO:teuthology.orchestra.run.smithi061.stdout:121 2023-10-18T09:16:40.170 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-10-18T09:16:40.171 DEBUG:teuthology.orchestra.run.smithi061:> sudo stdin-killer -- python3 -c ' 2023-10-18T09:16:40.171 DEBUG:teuthology.orchestra.run.smithi061:> import glob 2023-10-18T09:16:40.172 DEBUG:teuthology.orchestra.run.smithi061:> import re 2023-10-18T09:16:40.172 DEBUG:teuthology.orchestra.run.smithi061:> import os 2023-10-18T09:16:40.172 DEBUG:teuthology.orchestra.run.smithi061:> import subprocess 2023-10-18T09:16:40.173 DEBUG:teuthology.orchestra.run.smithi061:> 2023-10-18T09:16:40.173 DEBUG:teuthology.orchestra.run.smithi061:> def _find_admin_socket(client_name): 2023-10-18T09:16:40.173 DEBUG:teuthology.orchestra.run.smithi061:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-18T09:16:40.174 DEBUG:teuthology.orchestra.run.smithi061:> files = glob.glob(asok_path) 2023-10-18T09:16:40.174 DEBUG:teuthology.orchestra.run.smithi061:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-18T09:16:40.175 DEBUG:teuthology.orchestra.run.smithi061:> 2023-10-18T09:16:40.175 DEBUG:teuthology.orchestra.run.smithi061:> # Given a non-glob path, it better be there 2023-10-18T09:16:40.175 DEBUG:teuthology.orchestra.run.smithi061:> if "*" not in asok_path: 2023-10-18T09:16:40.176 DEBUG:teuthology.orchestra.run.smithi061:> assert(len(files) == 1) 2023-10-18T09:16:40.176 DEBUG:teuthology.orchestra.run.smithi061:> return files[0] 2023-10-18T09:16:40.176 DEBUG:teuthology.orchestra.run.smithi061:> 2023-10-18T09:16:40.177 DEBUG:teuthology.orchestra.run.smithi061:> for f in files: 2023-10-18T09:16:40.177 DEBUG:teuthology.orchestra.run.smithi061:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-18T09:16:40.177 DEBUG:teuthology.orchestra.run.smithi061:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-18T09:16:40.178 DEBUG:teuthology.orchestra.run.smithi061:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-18T09:16:40.178 DEBUG:teuthology.orchestra.run.smithi061:> contents = proc_f.read() 2023-10-18T09:16:40.178 DEBUG:teuthology.orchestra.run.smithi061:> if mountpoint in contents: 2023-10-18T09:16:40.179 DEBUG:teuthology.orchestra.run.smithi061:> return f 2023-10-18T09:16:40.179 DEBUG:teuthology.orchestra.run.smithi061:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-18T09:16:40.180 DEBUG:teuthology.orchestra.run.smithi061:> 2023-10-18T09:16:40.180 DEBUG:teuthology.orchestra.run.smithi061:> print(_find_admin_socket("client.admin")) 2023-10-18T09:16:40.180 DEBUG:teuthology.orchestra.run.smithi061:> ' 2023-10-18T09:16:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:39 smithi110 ceph-mon[134818]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-10-18T09:16:40.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:39 smithi061 ceph-mon[98573]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 1 op/s 2023-10-18T09:16:40.357 INFO:teuthology.orchestra.run.smithi061.stdout:/var/run/ceph/ceph-client.admin.153707.asok 2023-10-18T09:16:40.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-18T09:16:40.371 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.153707.asok 2023-10-18T09:16:40.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:40.371 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.153707.asok status 2023-10-18T09:16:40.604 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:16:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "metadata": { 2023-10-18T09:16:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_sha1": "f7420d7f8e55ceea786dcbf536c338ce920c21eb", 2023-10-18T09:16:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_version": "ceph version 18.0.0-6786-gf7420d7f (f7420d7f8e55ceea786dcbf536c338ce920c21eb) reef (dev)", 2023-10-18T09:16:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "entity_id": "admin", 2023-10-18T09:16:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": "smithi061", 2023-10-18T09:16:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-18T09:16:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "pid": "153707", 2023-10-18T09:16:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "root": "/" 2023-10-18T09:16:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:16:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_count": 0, 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "dentry_pinned_count": 0, 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 14823, 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "inst": { 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "name": { 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "client", 2023-10-18T09:16:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 14823 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.1:0", 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 1322786106 2023-10-18T09:16:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": { 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "192.168.0.1:0", 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 1322786106 2023-10-18T09:16:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:16:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "inst_str": "client.14823 192.168.0.1:0/1322786106", 2023-10-18T09:16:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "addr_str": "192.168.0.1:0/1322786106", 2023-10-18T09:16:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "inode_count": 1, 2023-10-18T09:16:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_epoch": 17, 2023-10-18T09:16:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch": 88, 2023-10-18T09:16:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_epoch_barrier": 0, 2023-10-18T09:16:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklisted": false, 2023-10-18T09:16:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs" 2023-10-18T09:16:40.611 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:16:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:40.620 DEBUG:teuthology.orchestra.run.smithi061:> stat --file-system '--printf=%T 2023-10-18T09:16:40.620 DEBUG:teuthology.orchestra.run.smithi061:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:40.665 INFO:teuthology.orchestra.run.smithi061.stdout:fuseblk 2023-10-18T09:16:40.665 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:40.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:40.665 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:40.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:40.777 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-18T09:16:40.906 INFO:teuthology.orchestra.run.smithi061.stdout:# file: . 2023-10-18T09:16:40.907 INFO:teuthology.orchestra.run.smithi061.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-18T09:16:40.907 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:40.908 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:16:40.908 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:16:40.953 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:41.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 -- 172.21.15.61:0/65397278 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1624108610 msgr2=0x7f162410aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 --2- 172.21.15.61:0/65397278 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1624108610 0x7f162410aa80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b0a0 tx=0x7f161c00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 -- 172.21.15.61:0/65397278 shutdown_connections 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 --2- 172.21.15.61:0/65397278 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1624108610 0x7f162410aa80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 --2- 172.21.15.61:0/65397278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16240747d0 0x7f1624072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 --2- 172.21.15.61:0/65397278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240731f0 0x7f1624073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.195+0000 7f1628fff700 1 -- 172.21.15.61:0/65397278 >> 172.21.15.61:0/65397278 conn(0x7f16240fbc90 msgr2=0x7f16240fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:41.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.196+0000 7f1628fff700 1 -- 172.21.15.61:0/65397278 shutdown_connections 2023-10-18T09:16:41.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.196+0000 7f1628fff700 1 -- 172.21.15.61:0/65397278 wait complete. 2023-10-18T09:16:41.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.197+0000 7f1628fff700 1 Processor -- start 2023-10-18T09:16:41.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.197+0000 7f1628fff700 1 -- start start 2023-10-18T09:16:41.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240747d0 0x7f1624198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1624108610 0x7f162419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241a04a0 con 0x7f16240731f0 2023-10-18T09:16:41.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241a05e0 con 0x7f16240747d0 2023-10-18T09:16:41.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1628fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241a0770 con 0x7f1624108610 2023-10-18T09:16:41.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49112/0 (socket says 172.21.15.61:49112) 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 -- 172.21.15.61:0/1579245845 learned_addr learned my addr 172.21.15.61:0/1579245845 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1621d9b700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240747d0 0x7f1624198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f1622d9d700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1624108610 0x7f162419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 -- 172.21.15.61:0/1579245845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1624108610 msgr2=0x7f162419fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1624108610 0x7f162419fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 -- 172.21.15.61:0/1579245845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240747d0 msgr2=0x7f1624198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240747d0 0x7f1624198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.198+0000 7f162259c700 1 -- 172.21.15.61:0/1579245845 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009d00 con 0x7f16240731f0 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f162259c700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9a0 tx=0x7f161400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614009790 con 0x7f16240731f0 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16241a0a50 con 0x7f16240731f0 2023-10-18T09:16:41.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f161400cbd0 con 0x7f16240731f0 2023-10-18T09:16:41.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614018680 con 0x7f16240731f0 2023-10-18T09:16:41.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.199+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16241a0fb0 con 0x7f16240731f0 2023-10-18T09:16:41.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.200+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624107420 con 0x7f16240731f0 2023-10-18T09:16:41.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.202+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f161400cd40 con 0x7f16240731f0 2023-10-18T09:16:41.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.203+0000 7f16137fe700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 0x7f160c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.204+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1614014070 con 0x7f16240731f0 2023-10-18T09:16:41.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.204+0000 7f1621d9b700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 0x7f160c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.204+0000 7f1621d9b700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 0x7f160c0a39c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1618009c60 tx=0x7f1618009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:41.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.206+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1614096850 con 0x7f16240731f0 2023-10-18T09:16:41.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.420+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1624067530 con 0x7f16240731f0 2023-10-18T09:16:41.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.422+0000 7f16137fe700 1 -- 172.21.15.61:0/1579245845 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f1614026090 con 0x7f16240731f0 2023-10-18T09:16:41.425 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:41.425 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:41.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 msgr2=0x7f160c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 0x7f160c0a39c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1618009c60 tx=0x7f1618009260 comp rx=0 tx=0).stop 2023-10-18T09:16:41.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 msgr2=0x7f1624198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f161400e9a0 tx=0x7f161400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 shutdown_connections 2023-10-18T09:16:41.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f160c0a1500 0x7f160c0a39c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1624108610 0x7f162419fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f16240731f0 0x7f1624198970 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.424+0000 7f1628fff700 1 --2- 172.21.15.61:0/1579245845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f16240747d0 0x7f1624198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.425+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 >> 172.21.15.61:0/1579245845 conn(0x7f16240fbc90 msgr2=0x7f1624104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:41.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.425+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 shutdown_connections 2023-10-18T09:16:41.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.425+0000 7f1628fff700 1 -- 172.21.15.61:0/1579245845 wait complete. 2023-10-18T09:16:41.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:16:41.447 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:16:41.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 -- 172.21.15.61:0/2526327159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c100880 msgr2=0x7ff92c100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 --2- 172.21.15.61:0/2526327159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c100880 0x7ff92c100ce0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff9200099c0 tx=0x7ff920009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 -- 172.21.15.61:0/2526327159 shutdown_connections 2023-10-18T09:16:41.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 --2- 172.21.15.61:0/2526327159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c101220 0x7ff92c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 --2- 172.21.15.61:0/2526327159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 0x7ff92c0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 --2- 172.21.15.61:0/2526327159 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c100880 0x7ff92c100ce0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 -- 172.21.15.61:0/2526327159 >> 172.21.15.61:0/2526327159 conn(0x7ff92c0fac00 msgr2=0x7ff92c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:41.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 -- 172.21.15.61:0/2526327159 shutdown_connections 2023-10-18T09:16:41.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.653+0000 7ff9313ff700 1 -- 172.21.15.61:0/2526327159 wait complete. 2023-10-18T09:16:41.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.654+0000 7ff9313ff700 1 Processor -- start 2023-10-18T09:16:41.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.654+0000 7ff9313ff700 1 -- start start 2023-10-18T09:16:41.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 0x7ff92c1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 0x7ff92c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c102940 con 0x7ff92c101220 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c102ab0 con 0x7ff92c100880 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff9313ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1a0740 con 0x7ff92c0ff690 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff92b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49122/0 (socket says 172.21.15.61:49122) 2023-10-18T09:16:41.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff92b7fe700 1 -- 172.21.15.61:0/1410941175 learned_addr learned my addr 172.21.15.61:0/1410941175 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:41.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff92a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 0x7ff92c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.655+0000 7ff92affd700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 0x7ff92c1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 -- 172.21.15.61:0/1410941175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 msgr2=0x7ff92c1035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 0x7ff92c1035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 -- 172.21.15.61:0/1410941175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 msgr2=0x7ff92c101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 0x7ff92c101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 -- 172.21.15.61:0/1410941175 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9200095f0 con 0x7ff92c101220 2023-10-18T09:16:41.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92a7fc700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 0x7ff92c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:41.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff92b7fe700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff92400ef30 tx=0x7ff92400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:41.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924018070 con 0x7ff92c101220 2023-10-18T09:16:41.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff92400f040 con 0x7ff92c101220 2023-10-18T09:16:41.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c1a0970 con 0x7ff92c101220 2023-10-18T09:16:41.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400cca0 con 0x7ff92c101220 2023-10-18T09:16:41.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.656+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1a0e80 con 0x7ff92c101220 2023-10-18T09:16:41.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.657+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c0675a0 con 0x7ff92c101220 2023-10-18T09:16:41.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.660+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff924003a00 con 0x7ff92c101220 2023-10-18T09:16:41.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.661+0000 7ff913fff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 0x7ff9140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:41.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.661+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff9240cd350 con 0x7ff92c101220 2023-10-18T09:16:41.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.661+0000 7ff92affd700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 0x7ff9140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:41.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.662+0000 7ff92affd700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 0x7ff9140a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1006e0 tx=0x7ff91c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:41.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.663+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff9240967f0 con 0x7ff92c101220 2023-10-18T09:16:41.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.877+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff92c10b080 con 0x7ff92c101220 2023-10-18T09:16:41.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.878+0000 7ff913fff700 1 -- 172.21.15.61:0/1410941175 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff924099e10 con 0x7ff92c101220 2023-10-18T09:16:41.881 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:16:41.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 msgr2=0x7ff9140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 0x7ff9140a39c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff92c1006e0 tx=0x7ff91c009260 comp rx=0 tx=0).stop 2023-10-18T09:16:41.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 msgr2=0x7ff92c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:41.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff92400ef30 tx=0x7ff92400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 shutdown_connections 2023-10-18T09:16:41.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9140a1500 0x7ff9140a39c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff92c101220 0x7ff92c1021a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff92c0ff690 0x7ff92c1035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 --2- 172.21.15.61:0/1410941175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff92c100880 0x7ff92c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:41.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 >> 172.21.15.61:0/1410941175 conn(0x7ff92c0fac00 msgr2=0x7ff92c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:41.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 shutdown_connections 2023-10-18T09:16:41.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:41.881+0000 7ff9313ff700 1 -- 172.21.15.61:0/1410941175 wait complete. 2023-10-18T09:16:41.904 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:16:42.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/2657617167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88074830 msgr2=0x7f2e88072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/2657617167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88074830 0x7f2e88072c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009990 tx=0x7f2e7c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/2657617167 shutdown_connections 2023-10-18T09:16:42.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/2657617167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 0x7f2e8810a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/2657617167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88074830 0x7f2e88072c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/2657617167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88073250 0x7f2e880736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/2657617167 >> 172.21.15.61:0/2657617167 conn(0x7f2e880fbbd0 msgr2=0x7f2e880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:42.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/2657617167 shutdown_connections 2023-10-18T09:16:42.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.108+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/2657617167 wait complete. 2023-10-18T09:16:42.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.109+0000 7f2e8dd9d700 1 Processor -- start 2023-10-18T09:16:42.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 -- start start 2023-10-18T09:16:42.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88074830 0x7f2e88198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 0x7f2e8819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881a02e0 con 0x7f2e88073250 2023-10-18T09:16:42.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881a0420 con 0x7f2e88108500 2023-10-18T09:16:42.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8dd9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e881a05b0 con 0x7f2e88074830 2023-10-18T09:16:42.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8cd9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8cd9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49138/0 (socket says 172.21.15.61:49138) 2023-10-18T09:16:42.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8cd9b700 1 -- 172.21.15.61:0/1020635551 learned_addr learned my addr 172.21.15.61:0/1020635551 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:42.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e87fff700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88074830 0x7f2e88198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.110+0000 7f2e8d59c700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 0x7f2e8819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 -- 172.21.15.61:0/1020635551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88074830 msgr2=0x7f2e88198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88074830 0x7f2e88198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 -- 172.21.15.61:0/1020635551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 msgr2=0x7f2e8819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 0x7f2e8819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 -- 172.21.15.61:0/1020635551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c0095f0 con 0x7f2e88073250 2023-10-18T09:16:42.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8cd9b700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c005910 tx=0x7f2e7c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:42.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c01d070 con 0x7f2e88073250 2023-10-18T09:16:42.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e7c021410 con 0x7f2e88073250 2023-10-18T09:16:42.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e881a0830 con 0x7f2e88073250 2023-10-18T09:16:42.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.111+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c017650 con 0x7f2e88073250 2023-10-18T09:16:42.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.112+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e881a0c50 con 0x7f2e88073250 2023-10-18T09:16:42.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.113+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e881072c0 con 0x7f2e88073250 2023-10-18T09:16:42.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.115+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2e7c021580 con 0x7f2e88073250 2023-10-18T09:16:42.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.116+0000 7f2e85ffb700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 0x7f2e780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.116+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2e7c0ce250 con 0x7f2e88073250 2023-10-18T09:16:42.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.116+0000 7f2e87fff700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 0x7f2e780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.117+0000 7f2e87fff700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 0x7f2e780a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e88074550 tx=0x7f2e74009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:42.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.120+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2e7c097750 con 0x7f2e88073250 2023-10-18T09:16:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:41 smithi110 ceph-mon[134818]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 1 op/s 2023-10-18T09:16:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1579245845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1410941175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:41 smithi061 ceph-mon[98573]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 1 op/s 2023-10-18T09:16:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1579245845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:42.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1410941175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:42.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.333+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e88106ca0 con 0x7f2e88073250 2023-10-18T09:16:42.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.333+0000 7f2e85ffb700 1 -- 172.21.15.61:0/1020635551 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e7c09ad70 con 0x7f2e88073250 2023-10-18T09:16:42.336 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:16:42.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 msgr2=0x7f2e780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 0x7f2e780a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e88074550 tx=0x7f2e74009260 comp rx=0 tx=0).stop 2023-10-18T09:16:42.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 msgr2=0x7f2e88198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c005910 tx=0x7f2e7c009580 comp rx=0 tx=0).stop 2023-10-18T09:16:42.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 shutdown_connections 2023-10-18T09:16:42.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e780a1480 0x7f2e780a3940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e88108500 0x7f2e8819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e88073250 0x7f2e88198800 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 --2- 172.21.15.61:0/1020635551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e88074830 0x7f2e88198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 >> 172.21.15.61:0/1020635551 conn(0x7f2e880fbbd0 msgr2=0x7f2e88104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:42.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 shutdown_connections 2023-10-18T09:16:42.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.336+0000 7f2e8dd9d700 1 -- 172.21.15.61:0/1020635551 wait complete. 2023-10-18T09:16:42.357 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:16:42.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:41 smithi170 ceph-mon[135150]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 308 B/s wr, 1 op/s 2023-10-18T09:16:42.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1579245845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:42.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1410941175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:42.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 -- 172.21.15.61:0/3208744760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2780101990 msgr2=0x7f2780101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 --2- 172.21.15.61:0/3208744760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2780101990 0x7f2780101df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27740099c0 tx=0x7f2774009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 -- 172.21.15.61:0/3208744760 shutdown_connections 2023-10-18T09:16:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 --2- 172.21.15.61:0/3208744760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780102330 0x7f2780108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 --2- 172.21.15.61:0/3208744760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27801007a0 0x7f2780100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 --2- 172.21.15.61:0/3208744760 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2780101990 0x7f2780101df0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 -- 172.21.15.61:0/3208744760 >> 172.21.15.61:0/3208744760 conn(0x7f27800fbcf0 msgr2=0x7f27800fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 -- 172.21.15.61:0/3208744760 shutdown_connections 2023-10-18T09:16:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.565+0000 7f27859ff700 1 -- 172.21.15.61:0/3208744760 wait complete. 2023-10-18T09:16:42.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.566+0000 7f27859ff700 1 Processor -- start 2023-10-18T09:16:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.566+0000 7f27859ff700 1 -- start start 2023-10-18T09:16:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780101990 0x7f2780072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2780102330 0x7f2780073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27800738c0 con 0x7f27801007a0 2023-10-18T09:16:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780073a30 con 0x7f2780102330 2023-10-18T09:16:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f27859ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27801a0760 con 0x7f2780101990 2023-10-18T09:16:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f277effd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f277effd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49156/0 (socket says 172.21.15.61:49156) 2023-10-18T09:16:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f277effd700 1 -- 172.21.15.61:0/1764583338 learned_addr learned my addr 172.21.15.61:0/1764583338 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f277e7fc700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780101990 0x7f2780072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.567+0000 7f277f7fe700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2780102330 0x7f2780073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 -- 172.21.15.61:0/1764583338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780101990 msgr2=0x7f2780072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780101990 0x7f2780072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 -- 172.21.15.61:0/1764583338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2780102330 msgr2=0x7f2780073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2780102330 0x7f2780073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 -- 172.21.15.61:0/1764583338 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2770009520 con 0x7f27801007a0 2023-10-18T09:16:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f277effd700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f277000ea50 tx=0x7f277000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277000cb70 con 0x7f27801007a0 2023-10-18T09:16:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277000ccd0 con 0x7f27801007a0 2023-10-18T09:16:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27740095f0 con 0x7f27801007a0 2023-10-18T09:16:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770018680 con 0x7f27801007a0 2023-10-18T09:16:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.568+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27801a0d20 con 0x7f27801007a0 2023-10-18T09:16:42.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.569+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2780107c40 con 0x7f27801007a0 2023-10-18T09:16:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.571+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2770010430 con 0x7f27801007a0 2023-10-18T09:16:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.572+0000 7f27849fd700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 0x7f27680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.573+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2770014070 con 0x7f27801007a0 2023-10-18T09:16:42.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.573+0000 7f277e7fc700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 0x7f27680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:42.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.573+0000 7f277e7fc700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 0x7f27680a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2774006010 tx=0x7f277400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:42.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.576+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2770096740 con 0x7f27801007a0 2023-10-18T09:16:42.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.787+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f278002cf10 con 0x7f27801007a0 2023-10-18T09:16:42.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.788+0000 7f27849fd700 1 -- 172.21.15.61:0/1764583338 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2770099d60 con 0x7f27801007a0 2023-10-18T09:16:42.791 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:16:42.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.790+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 msgr2=0x7f27680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.790+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 0x7f27680a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2774006010 tx=0x7f277400b540 comp rx=0 tx=0).stop 2023-10-18T09:16:42.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 msgr2=0x7f2780074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:42.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f277000ea50 tx=0x7f277000ed60 comp rx=0 tx=0).stop 2023-10-18T09:16:42.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 shutdown_connections 2023-10-18T09:16:42.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27680a1480 0x7f27680a3940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2780102330 0x7f2780073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27801007a0 0x7f2780074550 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 --2- 172.21.15.61:0/1764583338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2780101990 0x7f2780072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:42.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 >> 172.21.15.61:0/1764583338 conn(0x7f27800fbcf0 msgr2=0x7f2780104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:42.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 shutdown_connections 2023-10-18T09:16:42.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:42.791+0000 7f27859ff700 1 -- 172.21.15.61:0/1764583338 wait complete. 2023-10-18T09:16:42.812 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:16:43.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 -- 172.21.15.61:0/1322315282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c1007a0 msgr2=0x7f890c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 --2- 172.21.15.61:0/1322315282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c1007a0 0x7f890c100bc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f89000099c0 tx=0x7f8900009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 -- 172.21.15.61:0/1322315282 shutdown_connections 2023-10-18T09:16:43.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 --2- 172.21.15.61:0/1322315282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c102330 0x7f890c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 --2- 172.21.15.61:0/1322315282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c1007a0 0x7f890c100bc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 --2- 172.21.15.61:0/1322315282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c101990 0x7f890c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 -- 172.21.15.61:0/1322315282 >> 172.21.15.61:0/1322315282 conn(0x7f890c0fbcf0 msgr2=0x7f890c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:43.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 -- 172.21.15.61:0/1322315282 shutdown_connections 2023-10-18T09:16:43.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.018+0000 7f89131ff700 1 -- 172.21.15.61:0/1322315282 wait complete. 2023-10-18T09:16:43.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 Processor -- start 2023-10-18T09:16:43.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 -- start start 2023-10-18T09:16:43.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101990 0x7f890c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 0x7f890c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1a0490 con 0x7f890c102330 2023-10-18T09:16:43.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1a05d0 con 0x7f890c101990 2023-10-18T09:16:43.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f89131ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1a0760 con 0x7f890c1007a0 2023-10-18T09:16:43.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f8910f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.020+0000 7f8910f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:56472/0 (socket says 172.21.15.61:56472) 2023-10-18T09:16:43.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 -- 172.21.15.61:0/2835305858 learned_addr learned my addr 172.21.15.61:0/2835305858 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:43.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f891179c700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 0x7f890c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f890bfff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101990 0x7f890c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 -- 172.21.15.61:0/2835305858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101990 msgr2=0x7f890c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101990 0x7f890c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 -- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 msgr2=0x7f890c19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 0x7f890c19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 -- 172.21.15.61:0/2835305858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89000095f0 con 0x7f890c1007a0 2023-10-18T09:16:43.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f891179c700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 0x7f890c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:43.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8910f9b700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8900003730 tx=0x7f890000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:43.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890001d070 con 0x7f890c1007a0 2023-10-18T09:16:43.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c1a09e0 con 0x7f890c1007a0 2023-10-18T09:16:43.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.021+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f890000bda0 con 0x7f890c1007a0 2023-10-18T09:16:43.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.022+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000173f0 con 0x7f890c1007a0 2023-10-18T09:16:43.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.022+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c1a0e00 con 0x7f890c1007a0 2023-10-18T09:16:43.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.023+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c0675a0 con 0x7f890c1007a0 2023-10-18T09:16:43.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.025+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8900022470 con 0x7f890c1007a0 2023-10-18T09:16:43.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.026+0000 7f8909ffb700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 0x7f88fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.026+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f89000cdff0 con 0x7f890c1007a0 2023-10-18T09:16:43.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.026+0000 7f890bfff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 0x7f88fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.027+0000 7f890bfff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 0x7f88fc0a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009c60 tx=0x7f88f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:43.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.028+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f89000974f0 con 0x7f890c1007a0 2023-10-18T09:16:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1020635551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1764583338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.239+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f890c1a10f0 con 0x7f890c1007a0 2023-10-18T09:16:43.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.239+0000 7f8909ffb700 1 -- 172.21.15.61:0/2835305858 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f890009ab10 con 0x7f890c1007a0 2023-10-18T09:16:43.242 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:16:43.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 msgr2=0x7f88fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 0x7f88fc0a3940 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009c60 tx=0x7f88f8009260 comp rx=0 tx=0).stop 2023-10-18T09:16:43.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 msgr2=0x7f890c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8900003730 tx=0x7f890000fa00 comp rx=0 tx=0).stop 2023-10-18T09:16:43.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 shutdown_connections 2023-10-18T09:16:43.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88fc0a1480 0x7f88fc0a3940 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f890c102330 0x7f890c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f890c1007a0 0x7f890c198960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 --2- 172.21.15.61:0/2835305858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f890c101990 0x7f890c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 >> 172.21.15.61:0/2835305858 conn(0x7f890c0fbcf0 msgr2=0x7f890c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:43.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 shutdown_connections 2023-10-18T09:16:43.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.242+0000 7f89131ff700 1 -- 172.21.15.61:0/2835305858 wait complete. 2023-10-18T09:16:43.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1020635551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1764583338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.263 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:16:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1020635551' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1764583338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:43.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/3697690080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4100720 msgr2=0x7f6cf4100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/3697690080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4100720 0x7f6cf4100b40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40099c0 tx=0x7f6ce4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/3697690080 shutdown_connections 2023-10-18T09:16:43.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/3697690080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 0x7f6cf4108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/3697690080 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4100720 0x7f6cf4100b40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/3697690080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4101910 0x7f6cf4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/3697690080 >> 172.21.15.61:0/3697690080 conn(0x7f6cf40fbc90 msgr2=0x7f6cf40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:43.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/3697690080 shutdown_connections 2023-10-18T09:16:43.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.466+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/3697690080 wait complete. 2023-10-18T09:16:43.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.467+0000 7f6cfa9ff700 1 Processor -- start 2023-10-18T09:16:43.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 -- start start 2023-10-18T09:16:43.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 0x7f6cf41966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 0x7f6cf419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf419e200 con 0x7f6cf4101910 2023-10-18T09:16:43.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf419e340 con 0x7f6cf41022b0 2023-10-18T09:16:43.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cfa9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf419e4d0 con 0x7f6cf4100720 2023-10-18T09:16:43.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cf37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 0x7f6cf41966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cf37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49196/0 (socket says 172.21.15.61:49196) 2023-10-18T09:16:43.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cf37fe700 1 -- 172.21.15.61:0/4217914834 learned_addr learned my addr 172.21.15.61:0/4217914834 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:43.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.468+0000 7f6cf8f9c700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 0x7f6cf419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 -- 172.21.15.61:0/4217914834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 msgr2=0x7f6cf41966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 0x7f6cf41966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 -- 172.21.15.61:0/4217914834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 msgr2=0x7f6cf419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 0x7f6cf419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 -- 172.21.15.61:0/4217914834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40095f0 con 0x7f6cf4101910 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf3fff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 0x7f6cf41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf37fe700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9a0 tx=0x7f6ce800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8009790 con 0x7f6cf4101910 2023-10-18T09:16:43.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf419e7b0 con 0x7f6cf4101910 2023-10-18T09:16:43.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ce8010430 con 0x7f6cf4101910 2023-10-18T09:16:43.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8018680 con 0x7f6cf4101910 2023-10-18T09:16:43.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.469+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf419ecc0 con 0x7f6cf4101910 2023-10-18T09:16:43.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.470+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf4107bc0 con 0x7f6cf4101910 2023-10-18T09:16:43.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.472+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6ce80105a0 con 0x7f6cf4101910 2023-10-18T09:16:43.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.474+0000 7f6cf17fa700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 0x7f6cdc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:43.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.474+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6ce8014070 con 0x7f6cf4101910 2023-10-18T09:16:43.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.474+0000 7f6cf3fff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 0x7f6cdc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:43.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.475+0000 7f6cf3fff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 0x7f6cdc0a3940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4003730 tx=0x7f6ce400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:43.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.478+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6ce8052100 con 0x7f6cf4101910 2023-10-18T09:16:43.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.707+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6cf4067530 con 0x7f6cf4101910 2023-10-18T09:16:43.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.708+0000 7f6cf17fa700 1 -- 172.21.15.61:0/4217914834 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+591 (secure 0 0 0) 0x7f6ce8096510 con 0x7f6cf4101910 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/3728584080 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1938489455 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2303013428 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:43.711 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:43.712 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1518173664 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:43.712 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:43.712 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2840133191 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:43.712 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/623274229 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:43.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 msgr2=0x7f6cdc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 0x7f6cdc0a3940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4003730 tx=0x7f6ce400b540 comp rx=0 tx=0).stop 2023-10-18T09:16:43.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 msgr2=0x7f6cf4196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:43.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800e9a0 tx=0x7f6ce800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 shutdown_connections 2023-10-18T09:16:43.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6cdc0a1480 0x7f6cdc0a3940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6cf41022b0 0x7f6cf419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6cf4100720 0x7f6cf41966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 --2- 172.21.15.61:0/4217914834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6cf4101910 0x7f6cf4196c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:43.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.710+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 >> 172.21.15.61:0/4217914834 conn(0x7f6cf40fbc90 msgr2=0x7f6cf4104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:43.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.711+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 shutdown_connections 2023-10-18T09:16:43.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:43.711+0000 7f6cfa9ff700 1 -- 172.21.15.61:0/4217914834 wait complete. 2023-10-18T09:16:43.716 INFO:teuthology.orchestra.run.smithi061.stderr:listed 10 entries 2023-10-18T09:16:43.731 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:16:43.732 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:16:43.776 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:43 smithi061 ceph-mon[98573]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 270 B/s wr, 0 op/s 2023-10-18T09:16:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2835305858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4217914834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:44.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 -- 172.21.15.61:0/2798138952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5894100740 msgr2=0x7f5894100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 --2- 172.21.15.61:0/2798138952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5894100740 0x7f5894100b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58840099c0 tx=0x7f5884009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 -- 172.21.15.61:0/2798138952 shutdown_connections 2023-10-18T09:16:44.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 --2- 172.21.15.61:0/2798138952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58941022d0 0x7f5894108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 --2- 172.21.15.61:0/2798138952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5894100740 0x7f5894100b60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.021+0000 7f58999ff700 1 --2- 172.21.15.61:0/2798138952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 0x7f5894101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.022+0000 7f58999ff700 1 -- 172.21.15.61:0/2798138952 >> 172.21.15.61:0/2798138952 conn(0x7f58940fbc90 msgr2=0x7f58940fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:44.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.022+0000 7f58999ff700 1 -- 172.21.15.61:0/2798138952 shutdown_connections 2023-10-18T09:16:44.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.022+0000 7f58999ff700 1 -- 172.21.15.61:0/2798138952 wait complete. 2023-10-18T09:16:44.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 Processor -- start 2023-10-18T09:16:44.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 -- start start 2023-10-18T09:16:44.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 0x7f5894072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58941022d0 0x7f5894073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58940738c0 con 0x7f58941022d0 2023-10-18T09:16:44.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894073a30 con 0x7f5894101930 2023-10-18T09:16:44.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58999ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58941a0730 con 0x7f5894100740 2023-10-18T09:16:44.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f5892ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f5892ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:56502/0 (socket says 172.21.15.61:56502) 2023-10-18T09:16:44.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f58937fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58941022d0 0x7f5894073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.023+0000 7f5892ffd700 1 -- 172.21.15.61:0/1753161400 learned_addr learned my addr 172.21.15.61:0/1753161400 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:44.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f58927fc700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 0x7f5894072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 -- 172.21.15.61:0/1753161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 msgr2=0x7f5894072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 0x7f5894072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 -- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58941022d0 msgr2=0x7f5894073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58941022d0 0x7f5894073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 -- 172.21.15.61:0/1753161400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840095f0 con 0x7f5894100740 2023-10-18T09:16:44.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f5892ffd700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5884006010 tx=0x7f588400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588401d070 con 0x7f5894100740 2023-10-18T09:16:44.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5884005020 con 0x7f5894100740 2023-10-18T09:16:44.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58941a0960 con 0x7f5894100740 2023-10-18T09:16:44.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.024+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884004b10 con 0x7f5894100740 2023-10-18T09:16:44.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.025+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58941a0cd0 con 0x7f5894100740 2023-10-18T09:16:44.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.025+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5894107bc0 con 0x7f5894100740 2023-10-18T09:16:44.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.028+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5884005190 con 0x7f5894100740 2023-10-18T09:16:44.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.029+0000 7f58989fd700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 0x7f587c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.029+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f58840cd8b0 con 0x7f5894100740 2023-10-18T09:16:44.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.029+0000 7f58927fc700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 0x7f587c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.030+0000 7f58927fc700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 0x7f587c0a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5888009c60 tx=0x7f5888009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.031+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5884096db0 con 0x7f5894100740 2023-10-18T09:16:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:43 smithi110 ceph-mon[134818]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 270 B/s wr, 0 op/s 2023-10-18T09:16:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2835305858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4217914834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:44.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.245+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5894067530 con 0x7f5894100740 2023-10-18T09:16:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.248+0000 7f58989fd700 1 -- 172.21.15.61:0/1753161400 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f588409a3d0 con 0x7f5894100740 2023-10-18T09:16:44.250 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:44.250 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:44.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 msgr2=0x7f587c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 0x7f587c0a3940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5888009c60 tx=0x7f5888009260 comp rx=0 tx=0).stop 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 msgr2=0x7f5894074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5884006010 tx=0x7f588400f690 comp rx=0 tx=0).stop 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 shutdown_connections 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f587c0a1480 0x7f587c0a3940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58941022d0 0x7f5894073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5894100740 0x7f5894074550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 --2- 172.21.15.61:0/1753161400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5894101930 0x7f5894072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 >> 172.21.15.61:0/1753161400 conn(0x7f58940fbc90 msgr2=0x7f5894104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:44.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 shutdown_connections 2023-10-18T09:16:44.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.250+0000 7f58999ff700 1 -- 172.21.15.61:0/1753161400 wait complete. 2023-10-18T09:16:44.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:16:44.271 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:16:44.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:43 smithi170 ceph-mon[135150]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 270 B/s wr, 0 op/s 2023-10-18T09:16:44.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2835305858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:44.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4217914834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:44.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 -- 172.21.15.61:0/3138955261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 msgr2=0x7f8c5c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 --2- 172.21.15.61:0/3138955261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c101df0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0099c0 tx=0x7f8c4c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 -- 172.21.15.61:0/3138955261 shutdown_connections 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 --2- 172.21.15.61:0/3138955261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c102330 0x7f8c5c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 --2- 172.21.15.61:0/3138955261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c1007a0 0x7f8c5c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 --2- 172.21.15.61:0/3138955261 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c101df0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 -- 172.21.15.61:0/3138955261 >> 172.21.15.61:0/3138955261 conn(0x7f8c5c0fbcf0 msgr2=0x7f8c5c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 -- 172.21.15.61:0/3138955261 shutdown_connections 2023-10-18T09:16:44.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.477+0000 7f8c63bff700 1 -- 172.21.15.61:0/3138955261 wait complete. 2023-10-18T09:16:44.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.478+0000 7f8c63bff700 1 Processor -- start 2023-10-18T09:16:44.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.478+0000 7f8c63bff700 1 -- start start 2023-10-18T09:16:44.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c1007a0 0x7f8c5c196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c102330 0x7f8c5c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19e230 con 0x7f8c5c101990 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19e370 con 0x7f8c5c102330 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c63bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c19e500 con 0x7f8c5c1007a0 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c6119a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c6119a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49218/0 (socket says 172.21.15.61:49218) 2023-10-18T09:16:44.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c6119a700 1 -- 172.21.15.61:0/2789274266 learned_addr learned my addr 172.21.15.61:0/2789274266 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c6199b700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c1007a0 0x7f8c5c196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.479+0000 7f8c6219c700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c102330 0x7f8c5c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 -- 172.21.15.61:0/2789274266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c1007a0 msgr2=0x7f8c5c196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c1007a0 0x7f8c5c196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 -- 172.21.15.61:0/2789274266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c102330 msgr2=0x7f8c5c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c102330 0x7f8c5c19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 -- 172.21.15.61:0/2789274266 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0095f0 con 0x7f8c5c101990 2023-10-18T09:16:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c6119a700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c01d070 con 0x7f8c5c101990 2023-10-18T09:16:44.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c19e780 con 0x7f8c5c101990 2023-10-18T09:16:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c4c005030 con 0x7f8c5c101990 2023-10-18T09:16:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004b10 con 0x7f8c5c101990 2023-10-18T09:16:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.480+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c19ec00 con 0x7f8c5c101990 2023-10-18T09:16:44.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.481+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c107c40 con 0x7f8c5c101990 2023-10-18T09:16:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.483+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8c4c0051a0 con 0x7f8c5c101990 2023-10-18T09:16:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.485+0000 7f8c52ffd700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 0x7f8c480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.485+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8c4c0cdfd0 con 0x7f8c5c101990 2023-10-18T09:16:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.485+0000 7f8c6199b700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 0x7f8c480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.486+0000 7f8c6199b700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 0x7f8c480a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c1017f0 tx=0x7f8c58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.487+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8c4c097470 con 0x7f8c5c101990 2023-10-18T09:16:44.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.699+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c5c10c110 con 0x7f8c5c101990 2023-10-18T09:16:44.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.699+0000 7f8c52ffd700 1 -- 172.21.15.61:0/2789274266 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c4c09aa90 con 0x7f8c5c101990 2023-10-18T09:16:44.702 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:16:44.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 msgr2=0x7f8c480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 0x7f8c480a3940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c1017f0 tx=0x7f8c58009260 comp rx=0 tx=0).stop 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 msgr2=0x7f8c5c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c00f690 comp rx=0 tx=0).stop 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 shutdown_connections 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c480a1480 0x7f8c480a3940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c5c102330 0x7f8c5c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c5c1007a0 0x7f8c5c196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 --2- 172.21.15.61:0/2789274266 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c5c101990 0x7f8c5c196c40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 >> 172.21.15.61:0/2789274266 conn(0x7f8c5c0fbcf0 msgr2=0x7f8c5c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:44.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 shutdown_connections 2023-10-18T09:16:44.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.702+0000 7f8c63bff700 1 -- 172.21.15.61:0/2789274266 wait complete. 2023-10-18T09:16:44.722 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:16:44.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 -- 172.21.15.61:0/1620815339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 msgr2=0x7f4ca4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/1620815339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4100bc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4c940099c0 tx=0x7f4c94009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 -- 172.21.15.61:0/1620815339 shutdown_connections 2023-10-18T09:16:44.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/1620815339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4102330 0x7f4ca4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/1620815339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4100bc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/1620815339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4101990 0x7f4ca4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 -- 172.21.15.61:0/1620815339 >> 172.21.15.61:0/1620815339 conn(0x7f4ca40fbcf0 msgr2=0x7f4ca40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:44.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 -- 172.21.15.61:0/1620815339 shutdown_connections 2023-10-18T09:16:44.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.930+0000 7f4ca93ff700 1 -- 172.21.15.61:0/1620815339 wait complete. 2023-10-18T09:16:44.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.931+0000 7f4ca93ff700 1 Processor -- start 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.931+0000 7f4ca93ff700 1 -- start start 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 0x7f4ca4198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca41a0480 con 0x7f4ca4102330 2023-10-18T09:16:44.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca41a05c0 con 0x7f4ca41007a0 2023-10-18T09:16:44.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca93ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca41a0750 con 0x7f4ca4101990 2023-10-18T09:16:44.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49234/0 (socket says 172.21.15.61:49234) 2023-10-18T09:16:44.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 -- 172.21.15.61:0/3973837652 learned_addr learned my addr 172.21.15.61:0/3973837652 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:44.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4c9bfff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 0x7f4ca4198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 -- 172.21.15.61:0/3973837652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 msgr2=0x7f4ca4198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 0x7f4ca4198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 -- 172.21.15.61:0/3973837652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 msgr2=0x7f4ca4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:44.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca37fe700 1 -- 172.21.15.61:0/3973837652 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940095f0 con 0x7f4ca4102330 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4c9bfff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 0x7f4ca4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.932+0000 7f4ca2ffd700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca37fe700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00ef30 tx=0x7f4c9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c018070 con 0x7f4ca4102330 2023-10-18T09:16:44.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca41a09d0 con 0x7f4ca4102330 2023-10-18T09:16:44.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c9c00f040 con 0x7f4ca4102330 2023-10-18T09:16:44.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c00cdb0 con 0x7f4ca4102330 2023-10-18T09:16:44.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.933+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca41a0e90 con 0x7f4ca4102330 2023-10-18T09:16:44.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.934+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca40675a0 con 0x7f4ca4102330 2023-10-18T09:16:44.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.936+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4c9c003a00 con 0x7f4ca4102330 2023-10-18T09:16:44.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.937+0000 7f4ca0ff9700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 0x7f4c900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:44.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.938+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4c9c0ccb40 con 0x7f4ca4102330 2023-10-18T09:16:44.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.938+0000 7f4ca2ffd700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 0x7f4c900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:44.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.938+0000 7f4ca2ffd700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 0x7f4c900a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009990 tx=0x7f4c9401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:44.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:44.942+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4c9c095fe0 con 0x7f4ca4102330 2023-10-18T09:16:45.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:44 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1753161400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:45.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:44 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2789274266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:45.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.153+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ca41a10f0 con 0x7f4ca4102330 2023-10-18T09:16:45.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.153+0000 7f4ca0ff9700 1 -- 172.21.15.61:0/3973837652 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c9c099600 con 0x7f4ca4102330 2023-10-18T09:16:45.156 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:16:45.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.155+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 msgr2=0x7f4c900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.155+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 0x7f4c900a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009990 tx=0x7f4c9401a040 comp rx=0 tx=0).stop 2023-10-18T09:16:45.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.155+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 msgr2=0x7f4ca419feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.155+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00ef30 tx=0x7f4c9c00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 shutdown_connections 2023-10-18T09:16:45.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c900a1480 0x7f4c900a3940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4ca4102330 0x7f4ca419feb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4ca41007a0 0x7f4ca4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 --2- 172.21.15.61:0/3973837652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ca4101990 0x7f4ca4198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 >> 172.21.15.61:0/3973837652 conn(0x7f4ca40fbcf0 msgr2=0x7f4ca41071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:45.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 shutdown_connections 2023-10-18T09:16:45.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.156+0000 7f4ca93ff700 1 -- 172.21.15.61:0/3973837652 wait complete. 2023-10-18T09:16:45.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:16:45.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:44 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1753161400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:45.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:44 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2789274266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:45.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 -- 172.21.15.61:0/1373359469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41019b0 msgr2=0x7f91d4101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1373359469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41019b0 0x7f91d4101e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009990 tx=0x7f91c4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 -- 172.21.15.61:0/1373359469 shutdown_connections 2023-10-18T09:16:45.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1373359469 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1373359469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41007c0 0x7f91d4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1373359469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41019b0 0x7f91d4101e10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 -- 172.21.15.61:0/1373359469 >> 172.21.15.61:0/1373359469 conn(0x7f91d40fbd50 msgr2=0x7f91d40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:45.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 -- 172.21.15.61:0/1373359469 shutdown_connections 2023-10-18T09:16:45.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.382+0000 7f91db9ff700 1 -- 172.21.15.61:0/1373359469 wait complete. 2023-10-18T09:16:45.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.383+0000 7f91db9ff700 1 Processor -- start 2023-10-18T09:16:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.383+0000 7f91db9ff700 1 -- start start 2023-10-18T09:16:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 0x7f91d4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 0x7f91d4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40738c0 con 0x7f91d4102350 2023-10-18T09:16:45.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d4073a30 con 0x7f91d41019b0 2023-10-18T09:16:45.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91db9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d41a0720 con 0x7f91d41007c0 2023-10-18T09:16:45.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 0x7f91d4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d979b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 0x7f91d4074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:56540/0 (socket says 172.21.15.61:56540) 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d979b700 1 -- 172.21.15.61:0/1185698963 learned_addr learned my addr 172.21.15.61:0/1185698963 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d8f9a700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 0x7f91d4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 -- 172.21.15.61:0/1185698963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 msgr2=0x7f91d4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 0x7f91d4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 -- 172.21.15.61:0/1185698963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 msgr2=0x7f91d4072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 0x7f91d4072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d9f9c700 1 -- 172.21.15.61:0/1185698963 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40095f0 con 0x7f91d4102350 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.384+0000 7f91d8f9a700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 0x7f91d4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91d9f9c700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4073120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00efc0 tx=0x7f91cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc018070 con 0x7f91d4102350 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d41a0950 con 0x7f91d4102350 2023-10-18T09:16:45.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc00f040 con 0x7f91d4102350 2023-10-18T09:16:45.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00cce0 con 0x7f91d4102350 2023-10-18T09:16:45.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.385+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d41a0e60 con 0x7f91d4102350 2023-10-18T09:16:45.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.386+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d4107b90 con 0x7f91d4102350 2023-10-18T09:16:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:44 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1753161400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:44 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2789274266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:16:45.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.388+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f91cc003a00 con 0x7f91d4102350 2023-10-18T09:16:45.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.389+0000 7f91ca7fc700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 0x7f91c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.390+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91cc0cd480 con 0x7f91d4102350 2023-10-18T09:16:45.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.390+0000 7f91d979b700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 0x7f91c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.390+0000 7f91d979b700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 0x7f91c00a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005950 tx=0x7f91d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:45.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.392+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91cc096920 con 0x7f91d4102350 2023-10-18T09:16:45.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.605+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91d410c0e0 con 0x7f91d4102350 2023-10-18T09:16:45.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.605+0000 7f91ca7fc700 1 -- 172.21.15.61:0/1185698963 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91cc099f40 con 0x7f91d4102350 2023-10-18T09:16:45.608 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:16:45.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 msgr2=0x7f91c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 0x7f91c00a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005950 tx=0x7f91d0009260 comp rx=0 tx=0).stop 2023-10-18T09:16:45.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 msgr2=0x7f91d4073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4073120 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00efc0 tx=0x7f91cc00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 shutdown_connections 2023-10-18T09:16:45.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91c00a1480 0x7f91c00a3940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91d4102350 0x7f91d4073120 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91d41007c0 0x7f91d4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 --2- 172.21.15.61:0/1185698963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91d41019b0 0x7f91d4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 >> 172.21.15.61:0/1185698963 conn(0x7f91d40fbd50 msgr2=0x7f91d4104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 shutdown_connections 2023-10-18T09:16:45.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.608+0000 7f91db9ff700 1 -- 172.21.15.61:0/1185698963 wait complete. 2023-10-18T09:16:45.628 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:16:45.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1781893890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 msgr2=0x7f1b78101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1781893890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78101df0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b60009920 tx=0x7f1b60009c30 comp rx=0 tx=0).stop 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1781893890 shutdown_connections 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1781893890 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1781893890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1781893890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78101df0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1781893890 >> 172.21.15.61:0/1781893890 conn(0x7f1b780fbcf0 msgr2=0x7f1b780fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:45.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1781893890 shutdown_connections 2023-10-18T09:16:45.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.834+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1781893890 wait complete. 2023-10-18T09:16:45.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 Processor -- start 2023-10-18T09:16:45.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 -- start start 2023-10-18T09:16:45.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78073820 con 0x7f1b78102330 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78073990 con 0x7f1b78101990 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.835+0000 7f1b7d7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a0890 con 0x7f1b781007a0 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b777fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b777fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49276/0 (socket says 172.21.15.61:49276) 2023-10-18T09:16:45.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:56548/0 (socket says 172.21.15.61:56548) 2023-10-18T09:16:45.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b777fe700 1 -- 172.21.15.61:0/1200770976 learned_addr learned my addr 172.21.15.61:0/1200770976 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:45.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b6ffff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 -- 172.21.15.61:0/1200770976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 msgr2=0x7f1b78072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 -- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 msgr2=0x7f1b78073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:45.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:45.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 -- 172.21.15.61:0/1200770976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60009520 con 0x7f1b781007a0 2023-10-18T09:16:45.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b777fe700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.836+0000 7f1b76ffd700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9c0 tx=0x7f1b6800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.837+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68004220 con 0x7f1b781007a0 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.837+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b68004380 con 0x7f1b781007a0 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.837+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b781a0af0 con 0x7f1b781007a0 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.837+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68003be0 con 0x7f1b781007a0 2023-10-18T09:16:45.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.837+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b781a1000 con 0x7f1b781007a0 2023-10-18T09:16:45.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.838+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b78107c10 con 0x7f1b781007a0 2023-10-18T09:16:45.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.840+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1b68003d40 con 0x7f1b781007a0 2023-10-18T09:16:45.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.841+0000 7f1b74ff9700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 0x7f1b640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:45.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.841+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1b68014070 con 0x7f1b781007a0 2023-10-18T09:16:45.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.842+0000 7f1b6ffff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 0x7f1b640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:45.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.842+0000 7f1b6ffff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 0x7f1b640a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000fa40 tx=0x7f1b60019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:45.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:45.843+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1b68096780 con 0x7f1b781007a0 2023-10-18T09:16:46.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.054+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1b7810c000 con 0x7f1b781007a0 2023-10-18T09:16:46.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.055+0000 7f1b74ff9700 1 -- 172.21.15.61:0/1200770976 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1b68099da0 con 0x7f1b781007a0 2023-10-18T09:16:46.058 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:16:46.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.057+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 msgr2=0x7f1b640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 0x7f1b640a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000fa40 tx=0x7f1b60019040 comp rx=0 tx=0).stop 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 msgr2=0x7f1b78074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e9c0 tx=0x7f1b6800ecd0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 shutdown_connections 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b640a1480 0x7f1b640a3940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b78102330 0x7f1b78073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b781007a0 0x7f1b78074500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 --2- 172.21.15.61:0/1200770976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b78101990 0x7f1b78072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 >> 172.21.15.61:0/1200770976 conn(0x7f1b780fbcf0 msgr2=0x7f1b78104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 shutdown_connections 2023-10-18T09:16:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.058+0000 7f1b7d7ff700 1 -- 172.21.15.61:0/1200770976 wait complete. 2023-10-18T09:16:46.079 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:16:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:45 smithi110 ceph-mon[134818]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3973837652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1185698963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:45 smithi061 ceph-mon[98573]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:46.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3973837652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1185698963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3618992951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940747d0 msgr2=0x7f0a94072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3618992951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940747d0 0x7f0a94072c20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009990 tx=0x7f0a84009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3618992951 shutdown_connections 2023-10-18T09:16:46.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3618992951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a941085f0 0x7f0a9410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3618992951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940747d0 0x7f0a94072c20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3618992951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a940731f0 0x7f0a94073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3618992951 >> 172.21.15.61:0/3618992951 conn(0x7f0a940fbc90 msgr2=0x7f0a940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:46.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3618992951 shutdown_connections 2023-10-18T09:16:46.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.283+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3618992951 wait complete. 2023-10-18T09:16:46.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 Processor -- start 2023-10-18T09:16:46.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 -- start start 2023-10-18T09:16:46.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:46.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940747d0 0x7f0a94198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:46.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 0x7f0a9419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:46.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941a03a0 con 0x7f0a940731f0 2023-10-18T09:16:46.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941a04e0 con 0x7f0a941085f0 2023-10-18T09:16:46.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.284+0000 7f0a9a9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a941a0670 con 0x7f0a940747d0 2023-10-18T09:16:46.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:46.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a98f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 0x7f0a9419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a98f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 0x7f0a9419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54088/0 (socket says 172.21.15.61:54088) 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49298/0 (socket says 172.21.15.61:49298) 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a98f9c700 1 -- 172.21.15.61:0/3114389349 learned_addr learned my addr 172.21.15.61:0/3114389349 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a937fe700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940747d0 0x7f0a94198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 -- 172.21.15.61:0/3114389349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940747d0 msgr2=0x7f0a94198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940747d0 0x7f0a94198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 -- 172.21.15.61:0/3114389349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 msgr2=0x7f0a9419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 0x7f0a9419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 -- 172.21.15.61:0/3114389349 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840095f0 con 0x7f0a940731f0 2023-10-18T09:16:46.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.285+0000 7f0a93fff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400b680 tx=0x7f0a84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:46.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.286+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8401d070 con 0x7f0a940731f0 2023-10-18T09:16:46.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.286+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a941a08f0 con 0x7f0a940731f0 2023-10-18T09:16:46.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.286+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a84021410 con 0x7f0a940731f0 2023-10-18T09:16:46.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.286+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a84017650 con 0x7f0a940731f0 2023-10-18T09:16:46.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.286+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a941a0d70 con 0x7f0a940731f0 2023-10-18T09:16:46.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.287+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a94107400 con 0x7f0a940731f0 2023-10-18T09:16:46.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.289+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0a84021580 con 0x7f0a940731f0 2023-10-18T09:16:46.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.290+0000 7f0a917fa700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 0x7f0a7c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:46.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.290+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0a840ce290 con 0x7f0a940731f0 2023-10-18T09:16:46.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.290+0000 7f0a937fe700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 0x7f0a7c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:46.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.291+0000 7f0a937fe700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 0x7f0a7c0a39c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c60 tx=0x7f0a88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:46.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.294+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a84097790 con 0x7f0a940731f0 2023-10-18T09:16:46.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:45 smithi170 ceph-mon[135150]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:46.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3973837652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1185698963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:16:46.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.520+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0a94067530 con 0x7f0a940731f0 2023-10-18T09:16:46.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.521+0000 7f0a917fa700 1 -- 172.21.15.61:0/3114389349 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+591 (secure 0 0 0) 0x7f0a8409adb0 con 0x7f0a940731f0 2023-10-18T09:16:46.524 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/3728584080 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:46.524 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1938489455 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:46.524 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:46.524 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:46.524 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2303013428 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:46.525 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:46.525 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1518173664 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:46.525 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:16:46.525 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2840133191 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:46.525 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/623274229 2023-10-19T09:08:53.425206+0000 2023-10-18T09:16:46.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.523+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 msgr2=0x7f0a7c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.523+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 0x7f0a7c0a39c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009c60 tx=0x7f0a88009260 comp rx=0 tx=0).stop 2023-10-18T09:16:46.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.523+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 msgr2=0x7f0a94198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:46.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.523+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a8400b680 tx=0x7f0a84004730 comp rx=0 tx=0).stop 2023-10-18T09:16:46.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.523+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 shutdown_connections 2023-10-18T09:16:46.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0a7c0a1500 0x7f0a7c0a39c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0a941085f0 0x7f0a9419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0a940731f0 0x7f0a94198870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 --2- 172.21.15.61:0/3114389349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a940747d0 0x7f0a94198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:46.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 >> 172.21.15.61:0/3114389349 conn(0x7f0a940fbc90 msgr2=0x7f0a94104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:46.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 shutdown_connections 2023-10-18T09:16:46.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:46.524+0000 7f0a9a9ff700 1 -- 172.21.15.61:0/3114389349 wait complete. 2023-10-18T09:16:46.529 INFO:teuthology.orchestra.run.smithi061.stderr:listed 10 entries 2023-10-18T09:16:46.544 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi061.front.sepia.ceph.com... 2023-10-18T09:16:46.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:46.545 DEBUG:teuthology.orchestra.run.smithi061:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-18T09:16:46.613 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi061.stderr:ceph-fuse[153707]: fuse finished with error 0 and tester_r 0 2023-10-18T09:16:46.616 INFO:teuthology.orchestra.run:waiting for 300 2023-10-18T09:16:47.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:46 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1200770976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:47.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:46 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3114389349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:47.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1200770976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:47.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3114389349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:47.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:46 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1200770976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:16:47.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:46 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3114389349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:16:48.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:47 smithi110 ceph-mon[134818]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:47 smithi061 ceph-mon[98573]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:48.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:47 smithi170 ceph-mon[135150]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:49 smithi110 ceph-mon[134818]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:50.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:49 smithi061 ceph-mon[98573]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:50.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:49 smithi170 ceph-mon[135150]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:50 smithi061 ceph-mon[98573]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:50 smithi170 ceph-mon[135150]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:50 smithi110 ceph-mon[134818]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:52.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:16:52.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:16:52.719 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-18T09:16:52.785 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:52.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 -- 172.21.15.61:0/1533227587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c0747d0 msgr2=0x7f431c072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 --2- 172.21.15.61:0/1533227587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c0747d0 0x7f431c072c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4318009990 tx=0x7f4318009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 -- 172.21.15.61:0/1533227587 shutdown_connections 2023-10-18T09:16:53.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 --2- 172.21.15.61:0/1533227587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c108610 0x7f431c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 --2- 172.21.15.61:0/1533227587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c0747d0 0x7f431c072c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 --2- 172.21.15.61:0/1533227587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 -- 172.21.15.61:0/1533227587 >> 172.21.15.61:0/1533227587 conn(0x7f431c0fbc90 msgr2=0x7f431c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:53.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 -- 172.21.15.61:0/1533227587 shutdown_connections 2023-10-18T09:16:53.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.996+0000 7f43237ff700 1 -- 172.21.15.61:0/1533227587 wait complete. 2023-10-18T09:16:53.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 Processor -- start 2023-10-18T09:16:53.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 -- start start 2023-10-18T09:16:53.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c108610 0x7f431c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0420 con 0x7f431c0731f0 2023-10-18T09:16:53.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0560 con 0x7f431c108610 2023-10-18T09:16:53.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f43237ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a06f0 con 0x7f431c0747d0 2023-10-18T09:16:53.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f432159b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.997+0000 7f4320d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39938/0 (socket says 172.21.15.61:39938) 2023-10-18T09:16:53.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 -- 172.21.15.61:0/2780935132 learned_addr learned my addr 172.21.15.61:0/2780935132 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:53.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f432159b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53538/0 (socket says 172.21.15.61:53538) 2023-10-18T09:16:53.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4321d9c700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c108610 0x7f431c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 -- 172.21.15.61:0/2780935132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c108610 msgr2=0x7f431c19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c108610 0x7f431c19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 -- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 msgr2=0x7f431c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 -- 172.21.15.61:0/2780935132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c009520 con 0x7f431c0747d0 2023-10-18T09:16:53.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f4320d9a700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f430c00eb30 tx=0x7f430c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c009880 con 0x7f431c0747d0 2023-10-18T09:16:53.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430c0044e0 con 0x7f431c0747d0 2023-10-18T09:16:53.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.998+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43180095f0 con 0x7f431c0747d0 2023-10-18T09:16:53.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.999+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c003d80 con 0x7f431c0747d0 2023-10-18T09:16:53.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.999+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a0d50 con 0x7f431c0747d0 2023-10-18T09:16:53.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:52.999+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c107420 con 0x7f431c0747d0 2023-10-18T09:16:53.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.002+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f430c003f20 con 0x7f431c0747d0 2023-10-18T09:16:53.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.003+0000 7f43127fc700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 0x7f43080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.003+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f430c014070 con 0x7f431c0747d0 2023-10-18T09:16:53.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.003+0000 7f432159b700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 0x7f43080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.004+0000 7f432159b700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 0x7f43080a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f431800b2e0 tx=0x7f4318005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.005+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f430c096c80 con 0x7f431c0747d0 2023-10-18T09:16:53.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.218+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431c067530 con 0x7f431c0747d0 2023-10-18T09:16:53.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.220+0000 7f43127fc700 1 -- 172.21.15.61:0/2780935132 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f430c09a2a0 con 0x7f431c0747d0 2023-10-18T09:16:53.224 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:53.225 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:53.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 msgr2=0x7f43080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 0x7f43080a3940 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f431800b2e0 tx=0x7f4318005fb0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 msgr2=0x7f431c198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f430c00eb30 tx=0x7f430c00ee40 comp rx=0 tx=0).stop 2023-10-18T09:16:53.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 shutdown_connections 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f43080a1480 0x7f43080a3940 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f431c108610 0x7f431c19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f431c0731f0 0x7f431c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 --2- 172.21.15.61:0/2780935132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f431c0747d0 0x7f431c198e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 >> 172.21.15.61:0/2780935132 conn(0x7f431c0fbc90 msgr2=0x7f431c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:53.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 shutdown_connections 2023-10-18T09:16:53.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.223+0000 7f43237ff700 1 -- 172.21.15.61:0/2780935132 wait complete. 2023-10-18T09:16:53.245 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:16:53.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 -- 172.21.15.61:0/2900866165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 msgr2=0x7fb308073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/2900866165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb308073670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009990 tx=0x7fb2f0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 -- 172.21.15.61:0/2900866165 shutdown_connections 2023-10-18T09:16:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/2900866165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 0x7fb30810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/2900866165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/2900866165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb308073670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 -- 172.21.15.61:0/2900866165 >> 172.21.15.61:0/2900866165 conn(0x7fb3080fbc90 msgr2=0x7fb3080fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 -- 172.21.15.61:0/2900866165 shutdown_connections 2023-10-18T09:16:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.447+0000 7fb30e1ff700 1 -- 172.21.15.61:0/2900866165 wait complete. 2023-10-18T09:16:53.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 Processor -- start 2023-10-18T09:16:53.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 -- start start 2023-10-18T09:16:53.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 0x7fb30819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a03e0 con 0x7fb3080731f0 2023-10-18T09:16:53.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a0520 con 0x7fb308108610 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.448+0000 7fb30e1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a06b0 con 0x7fb3080747d0 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53568/0 (socket says 172.21.15.61:53568) 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 -- 172.21.15.61:0/3636639434 learned_addr learned my addr 172.21.15.61:0/3636639434 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb2fffff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb307fff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 0x7fb30819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 -- 172.21.15.61:0/3636639434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 msgr2=0x7fb308198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 -- 172.21.15.61:0/3636639434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 msgr2=0x7fb30819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 0x7fb30819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 -- 172.21.15.61:0/3636639434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f00095f0 con 0x7fb3080731f0 2023-10-18T09:16:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb2fffff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3077fe700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800e9a0 tx=0x7fb2f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8009790 con 0x7fb3080731f0 2023-10-18T09:16:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.449+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f8010430 con 0x7fb3080731f0 2023-10-18T09:16:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.450+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8018680 con 0x7fb3080731f0 2023-10-18T09:16:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.450+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3081a0990 con 0x7fb3080731f0 2023-10-18T09:16:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.450+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3081a0e50 con 0x7fb3080731f0 2023-10-18T09:16:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.451+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308107420 con 0x7fb3080731f0 2023-10-18T09:16:53.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.453+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb2f80105a0 con 0x7fb3080731f0 2023-10-18T09:16:53.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.454+0000 7fb3057fa700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 0x7fb2f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.454+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2f8014070 con 0x7fb3080731f0 2023-10-18T09:16:53.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.454+0000 7fb2fffff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 0x7fb2f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.455+0000 7fb2fffff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 0x7fb2f40a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b2e0 tx=0x7fb2f0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.458+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb2f8052100 con 0x7fb3080731f0 2023-10-18T09:16:53.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.675+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb308067500 con 0x7fb3080731f0 2023-10-18T09:16:53.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.677+0000 7fb3057fa700 1 -- 172.21.15.61:0/3636639434 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7fb2f8096510 con 0x7fb3080731f0 2023-10-18T09:16:53.681 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:16:53.681 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:16:53.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.679+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 msgr2=0x7fb2f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.679+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 0x7fb2f40a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000b2e0 tx=0x7fb2f0005fd0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 msgr2=0x7fb3081988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb2f800e9a0 tx=0x7fb2f800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 shutdown_connections 2023-10-18T09:16:53.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2f40a1480 0x7fb2f40a3940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb308108610 0x7fb30819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3080731f0 0x7fb3081988b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 --2- 172.21.15.61:0/3636639434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3080747d0 0x7fb308198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 >> 172.21.15.61:0/3636639434 conn(0x7fb3080fbc90 msgr2=0x7fb308104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:53.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 shutdown_connections 2023-10-18T09:16:53.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.680+0000 7fb30e1ff700 1 -- 172.21.15.61:0/3636639434 wait complete. 2023-10-18T09:16:53.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-18T09:16:53.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 -- 172.21.15.61:0/1298226160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 msgr2=0x7fccdc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 --2- 172.21.15.61:0/1298226160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc101df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccc40099c0 tx=0x7fccc4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 -- 172.21.15.61:0/1298226160 shutdown_connections 2023-10-18T09:16:53.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 --2- 172.21.15.61:0/1298226160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 --2- 172.21.15.61:0/1298226160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 --2- 172.21.15.61:0/1298226160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc101df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 -- 172.21.15.61:0/1298226160 >> 172.21.15.61:0/1298226160 conn(0x7fccdc0fbcf0 msgr2=0x7fccdc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:53.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 -- 172.21.15.61:0/1298226160 shutdown_connections 2023-10-18T09:16:53.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.907+0000 7fcce1fff700 1 -- 172.21.15.61:0/1298226160 wait complete. 2023-10-18T09:16:53.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 Processor -- start 2023-10-18T09:16:53.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 -- start start 2023-10-18T09:16:53.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1a0420 con 0x7fccdc102330 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1a0560 con 0x7fccdc101990 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.908+0000 7fcce1fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc1a06f0 con 0x7fccdc1007a0 2023-10-18T09:16:53.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39982/0 (socket says 172.21.15.61:39982) 2023-10-18T09:16:53.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 -- 172.21.15.61:0/757562523 learned_addr learned my addr 172.21.15.61:0/757562523 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:53.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccd3fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 -- 172.21.15.61:0/757562523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 msgr2=0x7fccdc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 -- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 msgr2=0x7fccdc19fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:53.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc19fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:53.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 -- 172.21.15.61:0/757562523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40095f0 con 0x7fccdc1007a0 2023-10-18T09:16:53.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdbfff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:53.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.909+0000 7fccdb7fe700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00e970 tx=0x7fcccc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.910+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc009790 con 0x7fccdc1007a0 2023-10-18T09:16:53.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.910+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc1a09d0 con 0x7fccdc1007a0 2023-10-18T09:16:53.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.910+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcccc00c7e0 con 0x7fccdc1007a0 2023-10-18T09:16:53.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.910+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc003c70 con 0x7fccdc1007a0 2023-10-18T09:16:53.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.910+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc1a0f30 con 0x7fccdc1007a0 2023-10-18T09:16:53.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.911+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc0675a0 con 0x7fccdc1007a0 2023-10-18T09:16:53.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.913+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcccc00cc50 con 0x7fccdc1007a0 2023-10-18T09:16:53.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.914+0000 7fccd97fa700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 0x7fccc80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:53.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.914+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcccc014070 con 0x7fccdc1007a0 2023-10-18T09:16:53.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.915+0000 7fccd3fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 0x7fccc80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:53.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.915+0000 7fccd3fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 0x7fccc80a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006010 tx=0x7fccc400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:53.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:53.917+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcccc0968f0 con 0x7fccdc1007a0 2023-10-18T09:16:54.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:53 smithi061 ceph-mon[98573]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T09:16:54.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:53 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2780935132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:53 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3636639434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.176+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fccdc10c000 con 0x7fccdc1007a0 2023-10-18T09:16:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:53 smithi110 ceph-mon[134818]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T09:16:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:53 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2780935132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:53 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3636639434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:53 smithi170 ceph-mon[135150]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-18T09:16:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:53 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2780935132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:53 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3636639434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:16:54.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.902+0000 7fccd97fa700 1 -- 172.21.15.61:0/757562523 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcccc003dd0 con 0x7fccdc1007a0 2023-10-18T09:16:54.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.905+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 msgr2=0x7fccc80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:54.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.905+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 0x7fccc80a3940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006010 tx=0x7fccc400b540 comp rx=0 tx=0).stop 2023-10-18T09:16:54.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.905+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 msgr2=0x7fccdc1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:54.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.905+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00e970 tx=0x7fcccc00ec80 comp rx=0 tx=0).stop 2023-10-18T09:16:54.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 shutdown_connections 2023-10-18T09:16:54.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fccc80a1480 0x7fccc80a3940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:54.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fccdc102330 0x7fccdc19fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:54.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccdc1007a0 0x7fccdc1988f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:54.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 --2- 172.21.15.61:0/757562523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fccdc101990 0x7fccdc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:54.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 >> 172.21.15.61:0/757562523 conn(0x7fccdc0fbcf0 msgr2=0x7fccdc072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:54.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 shutdown_connections 2023-10-18T09:16:54.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:54.906+0000 7fcce1fff700 1 -- 172.21.15.61:0/757562523 wait complete. 2023-10-18T09:16:54.933 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-18T09:16:54.934 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-10-18T09:16:55.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.143+0000 7fe8e83ff700 1 -- 172.21.15.61:0/2396020285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e0101990 msgr2=0x7fe8e0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:55.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.143+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/2396020285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e0101990 0x7fe8e0101df0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00099c0 tx=0x7fe8d0009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.143+0000 7fe8e83ff700 1 -- 172.21.15.61:0/2396020285 shutdown_connections 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/2396020285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 0x7fe8e0108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/2396020285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e01007a0 0x7fe8e0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/2396020285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e0101990 0x7fe8e0101df0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 -- 172.21.15.61:0/2396020285 >> 172.21.15.61:0/2396020285 conn(0x7fe8e00fbcf0 msgr2=0x7fe8e00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 -- 172.21.15.61:0/2396020285 shutdown_connections 2023-10-18T09:16:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.144+0000 7fe8e83ff700 1 -- 172.21.15.61:0/2396020285 wait complete. 2023-10-18T09:16:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 Processor -- start 2023-10-18T09:16:55.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 -- start start 2023-10-18T09:16:55.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:55.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e0101990 0x7fe8e0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:55.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 0x7fe8e019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:55.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e019e230 con 0x7fe8e01007a0 2023-10-18T09:16:55.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e019e370 con 0x7fe8e0101990 2023-10-18T09:16:55.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e83ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e019e500 con 0x7fe8e0102330 2023-10-18T09:16:55.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e619b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:55.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e619b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53584/0 (socket says 172.21.15.61:53584) 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.145+0000 7fe8e619b700 1 -- 172.21.15.61:0/67559401 learned_addr learned my addr 172.21.15.61:0/67559401 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e599a700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e0101990 0x7fe8e0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e699c700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 0x7fe8e019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 -- 172.21.15.61:0/67559401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 msgr2=0x7fe8e019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 0x7fe8e019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 -- 172.21.15.61:0/67559401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e0101990 msgr2=0x7fe8e0196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:55.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e0101990 0x7fe8e0196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 -- 172.21.15.61:0/67559401 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00095f0 con 0x7fe8e01007a0 2023-10-18T09:16:55.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e619b700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e970 tx=0x7fe8dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:55.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc009790 con 0x7fe8e01007a0 2023-10-18T09:16:55.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8dc00c7e0 con 0x7fe8e01007a0 2023-10-18T09:16:55.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.146+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e019e7e0 con 0x7fe8e01007a0 2023-10-18T09:16:55.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.147+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003c70 con 0x7fe8e01007a0 2023-10-18T09:16:55.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.147+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e019ed40 con 0x7fe8e01007a0 2023-10-18T09:16:55.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.148+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e0107c40 con 0x7fe8e01007a0 2023-10-18T09:16:55.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.150+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe8dc00cc50 con 0x7fe8e01007a0 2023-10-18T09:16:55.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.151+0000 7fe8d77fe700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 0x7fe8cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:55.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.151+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe8dc014070 con 0x7fe8e01007a0 2023-10-18T09:16:55.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.151+0000 7fe8e599a700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 0x7fe8cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:55.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.152+0000 7fe8e599a700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 0x7fe8cc0a39c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006010 tx=0x7fe8d000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:55.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.155+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe8dc096490 con 0x7fe8e01007a0 2023-10-18T09:16:55.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:54 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/757562523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:54 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:54 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/757562523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:54 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:54 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/757562523' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:54 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-18T09:16:55.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.416+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fe8e0107620 con 0x7fe8e01007a0 2023-10-18T09:16:55.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.905+0000 7fe8d77fe700 1 -- 172.21.15.61:0/67559401 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe8dc003dd0 con 0x7fe8e01007a0 2023-10-18T09:16:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 msgr2=0x7fe8cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 0x7fe8cc0a39c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006010 tx=0x7fe8d000b540 comp rx=0 tx=0).stop 2023-10-18T09:16:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 msgr2=0x7fe8e0196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00e970 tx=0x7fe8dc00ec80 comp rx=0 tx=0).stop 2023-10-18T09:16:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 shutdown_connections 2023-10-18T09:16:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe8cc0a1500 0x7fe8cc0a39c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8e0102330 0x7fe8e019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe8e01007a0 0x7fe8e0196700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 --2- 172.21.15.61:0/67559401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8e0101990 0x7fe8e0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 >> 172.21.15.61:0/67559401 conn(0x7fe8e00fbcf0 msgr2=0x7fe8e0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 shutdown_connections 2023-10-18T09:16:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:55.909+0000 7fe8e83ff700 1 -- 172.21.15.61:0/67559401 wait complete. 2023-10-18T09:16:55.935 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-18T09:16:56.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 -- 172.21.15.61:0/1711284834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c1007a0 msgr2=0x7fbb1c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1711284834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c100bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0099c0 tx=0x7fbb0c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 -- 172.21.15.61:0/1711284834 shutdown_connections 2023-10-18T09:16:56.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1711284834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c102330 0x7fbb1c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1711284834 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c100bc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1711284834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c101990 0x7fbb1c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 -- 172.21.15.61:0/1711284834 >> 172.21.15.61:0/1711284834 conn(0x7fbb1c0fbcf0 msgr2=0x7fbb1c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:56.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 -- 172.21.15.61:0/1711284834 shutdown_connections 2023-10-18T09:16:56.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.141+0000 7fbb211ff700 1 -- 172.21.15.61:0/1711284834 wait complete. 2023-10-18T09:16:56.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 Processor -- start 2023-10-18T09:16:56.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 -- start start 2023-10-18T09:16:56.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c101990 0x7fbb1c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a0490 con 0x7fbb1c102330 2023-10-18T09:16:56.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a05d0 con 0x7fbb1c1007a0 2023-10-18T09:16:56.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.143+0000 7fbb211ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1a0760 con 0x7fbb1c101990 2023-10-18T09:16:56.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53600/0 (socket says 172.21.15.61:53600) 2023-10-18T09:16:56.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 -- 172.21.15.61:0/1779001277 learned_addr learned my addr 172.21.15.61:0/1779001277 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:56.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1ad9d700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1a59c700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c101990 0x7fbb1c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 -- 172.21.15.61:0/1779001277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c101990 msgr2=0x7fbb1c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c101990 0x7fbb1c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 -- 172.21.15.61:0/1779001277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c1007a0 msgr2=0x7fbb1c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 -- 172.21.15.61:0/1779001277 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c0095f0 con 0x7fbb1c102330 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.144+0000 7fbb1b59e700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400ef30 tx=0x7fbb1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.145+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14018070 con 0x7fbb1c102330 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.145+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb1400f040 con 0x7fbb1c102330 2023-10-18T09:16:56.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.145+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1a09e0 con 0x7fbb1c102330 2023-10-18T09:16:56.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.145+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400cd00 con 0x7fbb1c102330 2023-10-18T09:16:56.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.145+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1a0ea0 con 0x7fbb1c102330 2023-10-18T09:16:56.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.146+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c0675a0 con 0x7fbb1c102330 2023-10-18T09:16:56.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.148+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb14003a00 con 0x7fbb1c102330 2023-10-18T09:16:56.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.149+0000 7fbb03fff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 0x7fbb040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.149+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb140ccac0 con 0x7fbb1c102330 2023-10-18T09:16:56.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.149+0000 7fbb1ad9d700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 0x7fbb040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.150+0000 7fbb1ad9d700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 0x7fbb040a3940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009990 tx=0x7fbb0c0036f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:56.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.153+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb14095f60 con 0x7fbb1c102330 2023-10-18T09:16:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:55 smithi110 ceph-mon[134818]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:55 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T09:16:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:55 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:55 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-18T09:16:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:55 smithi061 ceph-mon[98573]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:55 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T09:16:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:55 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:55 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-18T09:16:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:55 smithi170 ceph-mon[135150]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:55 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-18T09:16:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:55 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:56.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:55 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-10-18T09:16:56.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.415+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fbb1c10c000 con 0x7fbb1c102330 2023-10-18T09:16:56.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.501+0000 7fbb03fff700 1 -- 172.21.15.61:0/1779001277 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbb14099580 con 0x7fbb1c102330 2023-10-18T09:16:56.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 msgr2=0x7fbb040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 0x7fbb040a3940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009990 tx=0x7fbb0c0036f0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 msgr2=0x7fbb1c19fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400ef30 tx=0x7fbb1400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 shutdown_connections 2023-10-18T09:16:56.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb040a1480 0x7fbb040a3940 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb1c102330 0x7fbb1c19fec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb1c1007a0 0x7fbb1c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 --2- 172.21.15.61:0/1779001277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c101990 0x7fbb1c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 >> 172.21.15.61:0/1779001277 conn(0x7fbb1c0fbcf0 msgr2=0x7fbb1c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:56.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 shutdown_connections 2023-10-18T09:16:56.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.504+0000 7fbb211ff700 1 -- 172.21.15.61:0/1779001277 wait complete. 2023-10-18T09:16:56.533 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-18T09:16:56.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3900998414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 msgr2=0x7f1ca8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3900998414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca8100bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40099c0 tx=0x7f1ca4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3900998414 shutdown_connections 2023-10-18T09:16:56.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3900998414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8102330 0x7f1ca8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3900998414 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca8100bc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3900998414 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8101990 0x7f1ca8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.737+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3900998414 >> 172.21.15.61:0/3900998414 conn(0x7f1ca80fbcf0 msgr2=0x7f1ca80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:56.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.738+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3900998414 shutdown_connections 2023-10-18T09:16:56.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.738+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3900998414 wait complete. 2023-10-18T09:16:56.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 Processor -- start 2023-10-18T09:16:56.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 -- start start 2023-10-18T09:16:56.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8101990 0x7f1ca8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8102330 0x7f1ca819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a04b0 con 0x7f1ca81007a0 2023-10-18T09:16:56.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a05f0 con 0x7f1ca8101990 2023-10-18T09:16:56.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1caf3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca81a0780 con 0x7f1ca8102330 2023-10-18T09:16:56.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1cad19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1cad19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53624/0 (socket says 172.21.15.61:53624) 2023-10-18T09:16:56.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1cad19b700 1 -- 172.21.15.61:0/3013131008 learned_addr learned my addr 172.21.15.61:0/3013131008 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:56.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1cac99a700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8101990 0x7f1ca8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.739+0000 7f1cad99c700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8102330 0x7f1ca819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 -- 172.21.15.61:0/3013131008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8102330 msgr2=0x7f1ca819fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8102330 0x7f1ca819fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 -- 172.21.15.61:0/3013131008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8101990 msgr2=0x7f1ca8198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:56.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8101990 0x7f1ca8198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:56.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 -- 172.21.15.61:0/3013131008 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40095f0 con 0x7f1ca81007a0 2023-10-18T09:16:56.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1cad19b700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4003730 tx=0x7f1ca400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca401d070 con 0x7f1ca81007a0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca81a0a00 con 0x7f1ca81007a0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca400bda0 con 0x7f1ca81007a0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4004b30 con 0x7f1ca81007a0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.740+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca81a0e20 con 0x7f1ca81007a0 2023-10-18T09:16:56.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.741+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca80675a0 con 0x7f1ca81007a0 2023-10-18T09:16:56.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.743+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1ca4004c90 con 0x7f1ca81007a0 2023-10-18T09:16:56.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.745+0000 7f1c9e7fc700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 0x7f1c940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:56.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.745+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1ca40ce1b0 con 0x7f1ca81007a0 2023-10-18T09:16:56.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.745+0000 7f1cac99a700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 0x7f1c940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:56.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.746+0000 7f1cac99a700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 0x7f1c940a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007c60 tx=0x7f1c980073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:56.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:56.747+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ca4097650 con 0x7f1ca81007a0 2023-10-18T09:16:57.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.010+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1ca810c000 con 0x7f1ca81007a0 2023-10-18T09:16:57.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-18T09:16:57.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-18T09:16:57.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-18T09:16:57.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:57.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-18T09:16:57.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-18T09:16:57.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T09:16:57.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T09:16:57.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:active 2023-10-18T09:16:57.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-18T09:16:57.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-18T09:16:57.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:57.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:56 smithi110 ceph-mon[134818]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-18T09:16:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-18T09:16:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-18T09:16:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-18T09:16:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-18T09:16:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-18T09:16:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T09:16:57.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T09:16:57.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:active 2023-10-18T09:16:57.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-18T09:16:57.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-18T09:16:57.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:57.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:56 smithi061 ceph-mon[98573]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-18T09:16:57.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-18T09:16:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/67559401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-10-18T09:16:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-18T09:16:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-18T09:16:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-18T09:16:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-18T09:16:57.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-18T09:16:57.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1779001277' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-18T09:16:57.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] up:active 2023-10-18T09:16:57.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-18T09:16:57.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-10-18T09:16:57.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:57.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:56 smithi170 ceph-mon[135150]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-18T09:16:57.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.508+0000 7f1c9e7fc700 1 -- 172.21.15.61:0/3013131008 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1ca409ac70 con 0x7f1ca81007a0 2023-10-18T09:16:57.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 msgr2=0x7f1c940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:57.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 0x7f1c940a3940 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c98007c60 tx=0x7f1c980073d0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 msgr2=0x7f1ca81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:57.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4003730 tx=0x7f1ca400fa00 comp rx=0 tx=0).stop 2023-10-18T09:16:57.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 shutdown_connections 2023-10-18T09:16:57.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c940a1480 0x7f1c940a3940 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ca8102330 0x7f1ca819fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ca81007a0 0x7f1ca81989d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 --2- 172.21.15.61:0/3013131008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ca8101990 0x7f1ca8198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 >> 172.21.15.61:0/3013131008 conn(0x7f1ca80fbcf0 msgr2=0x7f1ca8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:57.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 shutdown_connections 2023-10-18T09:16:57.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.512+0000 7f1caf3ff700 1 -- 172.21.15.61:0/3013131008 wait complete. 2023-10-18T09:16:57.542 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-18T09:16:57.542 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-18T09:16:57.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.752+0000 7f1d38f9d700 1 -- 172.21.15.61:0/968355721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 msgr2=0x7f1d341034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:57.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.752+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/968355721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341034a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009990 tx=0x7f1d24009ca0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 -- 172.21.15.61:0/968355721 shutdown_connections 2023-10-18T09:16:57.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/968355721 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d34106310 0x7f1d34108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/968355721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341034a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/968355721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d341039e0 0x7f1d34105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 -- 172.21.15.61:0/968355721 >> 172.21.15.61:0/968355721 conn(0x7f1d340faa80 msgr2=0x7f1d340fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:57.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 -- 172.21.15.61:0/968355721 shutdown_connections 2023-10-18T09:16:57.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.753+0000 7f1d38f9d700 1 -- 172.21.15.61:0/968355721 wait complete. 2023-10-18T09:16:57.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 Processor -- start 2023-10-18T09:16:57.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 -- start start 2023-10-18T09:16:57.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:57.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d341039e0 0x7f1d34198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:57.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d34106310 0x7f1d3419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:57.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341a02d0 con 0x7f1d341039e0 2023-10-18T09:16:57.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341a0410 con 0x7f1d341010b0 2023-10-18T09:16:57.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d38f9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341a05a0 con 0x7f1d34106310 2023-10-18T09:16:57.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d337fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:57.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d337fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36262/0 (socket says 172.21.15.61:36262) 2023-10-18T09:16:57.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.754+0000 7f1d337fe700 1 -- 172.21.15.61:0/3674640002 learned_addr learned my addr 172.21.15.61:0/3674640002 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:57.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d32ffd700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d341039e0 0x7f1d34198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:57.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d33fff700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d34106310 0x7f1d3419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:57.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 -- 172.21.15.61:0/3674640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d34106310 msgr2=0x7f1d3419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:57.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d34106310 0x7f1d3419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 -- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d341039e0 msgr2=0x7f1d34198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:57.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d341039e0 0x7f1d34198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:57.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 -- 172.21.15.61:0/3674640002 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240095f0 con 0x7f1d341010b0 2023-10-18T09:16:57.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d337fe700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000c00 tx=0x7f1d24005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:57.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2401d070 con 0x7f1d341010b0 2023-10-18T09:16:57.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d24021410 con 0x7f1d341010b0 2023-10-18T09:16:57.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d341a0820 con 0x7f1d341010b0 2023-10-18T09:16:57.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.755+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24017650 con 0x7f1d341010b0 2023-10-18T09:16:57.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.756+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d341a0ca0 con 0x7f1d341010b0 2023-10-18T09:16:57.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.756+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34107f10 con 0x7f1d341010b0 2023-10-18T09:16:57.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.759+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1d24021810 con 0x7f1d341010b0 2023-10-18T09:16:57.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.760+0000 7f1d30ff9700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 0x7f1d1c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:57.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.760+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d240cead0 con 0x7f1d341010b0 2023-10-18T09:16:57.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.760+0000 7f1d32ffd700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 0x7f1d1c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:57.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.761+0000 7f1d32ffd700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 0x7f1d1c0a39c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007c60 tx=0x7f1d280073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:57.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:57.764+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d24097f70 con 0x7f1d341010b0 2023-10-18T09:16:58.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.011+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d34060c30 con 0x7f1d1c0a1500 2023-10-18T09:16:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T09:16:58.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T09:16:58.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:active 2023-10-18T09:16:58.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-18T09:16:58.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:58.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-10-18T09:16:58.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:57 smithi110 ceph-mon[134818]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-18T09:16:58.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:58.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T09:16:58.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T09:16:58.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:active 2023-10-18T09:16:58.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-18T09:16:58.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:58.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-10-18T09:16:58.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:57 smithi061 ceph-mon[98573]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-18T09:16:58.257 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:16:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:16:58.017+0000 7fb0fee46700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T09:16:58.257 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:16:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:16:58.017+0000 7fb0fee46700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T09:16:58.257 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:16:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:16:58.017+0000 7fb0fee46700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T09:16:58.257 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:16:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:16:58.017+0000 7fb0fee46700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T09:16:58.257 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:16:58 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:16:58.017+0000 7fb0fee46700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-18T09:16:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-18T09:16:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-18T09:16:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3013131008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-18T09:16:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] up:active 2023-10-18T09:16:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-10-18T09:16:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2023-10-18T09:16:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:creating} 2 up:standby-replay 6 up:standby 2023-10-18T09:16:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:57 smithi170 ceph-mon[135150]: daemon mds.f is now active in filesystem cephfs as rank 3 2023-10-18T09:16:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.558+0000 7f1d30ff9700 1 -- 172.21.15.61:0/3674640002 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d34060c30 con 0x7f1d1c0a1500 2023-10-18T09:16:58.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 msgr2=0x7f1d1c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 0x7f1d1c0a39c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007c60 tx=0x7f1d280073d0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 msgr2=0x7f1d341987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000c00 tx=0x7f1d24005740 comp rx=0 tx=0).stop 2023-10-18T09:16:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 shutdown_connections 2023-10-18T09:16:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d1c0a1500 0x7f1d1c0a39c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d34106310 0x7f1d3419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d341010b0 0x7f1d341987a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 --2- 172.21.15.61:0/3674640002 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d341039e0 0x7f1d34198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 >> 172.21.15.61:0/3674640002 conn(0x7f1d340faa80 msgr2=0x7f1d34100950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 shutdown_connections 2023-10-18T09:16:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.562+0000 7f1d38f9d700 1 -- 172.21.15.61:0/3674640002 wait complete. 2023-10-18T09:16:58.584 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-18T09:16:58.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3072458492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 msgr2=0x7f8ef4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:58.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3072458492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4101df0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80099c0 tx=0x7f8ee8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3072458492 shutdown_connections 2023-10-18T09:16:58.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3072458492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef4102330 0x7f8ef4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3072458492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef41007a0 0x7f8ef4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3072458492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4101df0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3072458492 >> 172.21.15.61:0/3072458492 conn(0x7f8ef40fbcf0 msgr2=0x7f8ef40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:58.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.791+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3072458492 shutdown_connections 2023-10-18T09:16:58.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.792+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3072458492 wait complete. 2023-10-18T09:16:58.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.792+0000 7f8ef9bff700 1 Processor -- start 2023-10-18T09:16:58.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 -- start start 2023-10-18T09:16:58.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef41007a0 0x7f8ef4074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:58.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:58.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef4102330 0x7f8ef4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:58.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40738c0 con 0x7f8ef4101990 2023-10-18T09:16:58.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4073a30 con 0x7f8ef41007a0 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef9bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41a0760 con 0x7f8ef4102330 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef2ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53662/0 (socket says 172.21.15.61:53662) 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef2ffd700 1 -- 172.21.15.61:0/3101924365 learned_addr learned my addr 172.21.15.61:0/3101924365 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef37fe700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef41007a0 0x7f8ef4074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:58.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.793+0000 7f8ef3fff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef4102330 0x7f8ef4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:58.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 -- 172.21.15.61:0/3101924365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef4102330 msgr2=0x7f8ef4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:58.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef4102330 0x7f8ef4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 -- 172.21.15.61:0/3101924365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef41007a0 msgr2=0x7f8ef4074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:58.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef41007a0 0x7f8ef4074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:58.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 -- 172.21.15.61:0/3101924365 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8ef4101990 2023-10-18T09:16:58.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef2ffd700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80036f0 tx=0x7f8ee8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:58.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee801d070 con 0x7f8ef4101990 2023-10-18T09:16:58.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee8004050 con 0x7f8ef4101990 2023-10-18T09:16:58.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef41a0990 con 0x7f8ef4101990 2023-10-18T09:16:58.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8022cf0 con 0x7f8ef4101990 2023-10-18T09:16:58.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.794+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef41a0d00 con 0x7f8ef4101990 2023-10-18T09:16:58.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.795+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4107c40 con 0x7f8ef4101990 2023-10-18T09:16:58.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.797+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8ee8022470 con 0x7f8ef4101990 2023-10-18T09:16:58.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.799+0000 7f8ef0ff9700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 0x7f8edc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:58.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.799+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8ee80ce1c0 con 0x7f8ef4101990 2023-10-18T09:16:58.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.799+0000 7f8ef37fe700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 0x7f8edc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:58.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.800+0000 7f8ef37fe700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 0x7f8edc0a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41017f0 tx=0x7f8ee400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:58.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:58.803+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8ee805d100 con 0x7f8ef4101990 2023-10-18T09:16:59.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.051+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ef4060c30 con 0x7f8edc0a1480 2023-10-18T09:16:59.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.055+0000 7f8ef0ff9700 1 -- 172.21.15.61:0/3101924365 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f8ef4060c30 con 0x7f8edc0a1480 2023-10-18T09:16:59.058 INFO:teuthology.orchestra.run.smithi061.stdout:/volumes/_nogroup/sv_0/af313209-340d-40cb-8808-b88fe1905393 2023-10-18T09:16:59.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 msgr2=0x7f8edc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 0x7f8edc0a3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ef41017f0 tx=0x7f8ee400b410 comp rx=0 tx=0).stop 2023-10-18T09:16:59.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 msgr2=0x7f8ef4072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80036f0 tx=0x7f8ee8009580 comp rx=0 tx=0).stop 2023-10-18T09:16:59.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 shutdown_connections 2023-10-18T09:16:59.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8edc0a1480 0x7f8edc0a3940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef4102330 0x7f8ef4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ef41007a0 0x7f8ef4074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 --2- 172.21.15.61:0/3101924365 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ef4101990 0x7f8ef4072be0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 >> 172.21.15.61:0/3101924365 conn(0x7f8ef40fbcf0 msgr2=0x7f8ef4104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:59.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 shutdown_connections 2023-10-18T09:16:59.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.058+0000 7f8ef9bff700 1 -- 172.21.15.61:0/3101924365 wait complete. 2023-10-18T09:16:59.079 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-18T09:16:59.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 -- 172.21.15.61:0/814174961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15241007a0 msgr2=0x7f1524100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 --2- 172.21.15.61:0/814174961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15241007a0 0x7f1524100bc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1514009920 tx=0x7f1514009c30 comp rx=0 tx=0).stop 2023-10-18T09:16:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 -- 172.21.15.61:0/814174961 shutdown_connections 2023-10-18T09:16:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 --2- 172.21.15.61:0/814174961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524102330 0x7f1524108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 --2- 172.21.15.61:0/814174961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15241007a0 0x7f1524100bc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 --2- 172.21.15.61:0/814174961 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1524101990 0x7f1524101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 -- 172.21.15.61:0/814174961 >> 172.21.15.61:0/814174961 conn(0x7f15240fbcf0 msgr2=0x7f15240fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 -- 172.21.15.61:0/814174961 shutdown_connections 2023-10-18T09:16:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.286+0000 7f152a3ff700 1 -- 172.21.15.61:0/814174961 wait complete. 2023-10-18T09:16:59.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 Processor -- start 2023-10-18T09:16:59.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 -- start start 2023-10-18T09:16:59.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524101990 0x7f1524196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1524102330 0x7f152419dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419e2f0 con 0x7f15241007a0 2023-10-18T09:16:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419e430 con 0x7f1524102330 2023-10-18T09:16:59.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f152a3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152419e5c0 con 0x7f1524101990 2023-10-18T09:16:59.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.287+0000 7f1523fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41010/0 (socket says 172.21.15.61:41010) 2023-10-18T09:16:59.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f15237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524101990 0x7f1524196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 -- 172.21.15.61:0/3093012884 learned_addr learned my addr 172.21.15.61:0/3093012884 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f152899c700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1524102330 0x7f152419dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 -- 172.21.15.61:0/3093012884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524101990 msgr2=0x7f1524196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524101990 0x7f1524196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 -- 172.21.15.61:0/3093012884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1524102330 msgr2=0x7f152419dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1524102330 0x7f152419dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 -- 172.21.15.61:0/3093012884 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009520 con 0x7f15241007a0 2023-10-18T09:16:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f1523fff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1514009230 tx=0x7f151400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151401c070 con 0x7f15241007a0 2023-10-18T09:16:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.288+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151400bba0 con 0x7f15241007a0 2023-10-18T09:16:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.289+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152419e840 con 0x7f15241007a0 2023-10-18T09:16:59.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.289+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514004930 con 0x7f15241007a0 2023-10-18T09:16:59.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.289+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152419ecc0 con 0x7f15241007a0 2023-10-18T09:16:59.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.290+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1524107c70 con 0x7f15241007a0 2023-10-18T09:16:59.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.292+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 16) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1514020070 con 0x7f15241007a0 2023-10-18T09:16:59.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.293+0000 7f15217fa700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 0x7f150c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.293+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f15140cdab0 con 0x7f15241007a0 2023-10-18T09:16:59.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.293+0000 7f15237fe700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 0x7f150c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.294+0000 7f15237fe700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 0x7f150c0a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1518005fd0 tx=0x7f1518009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:59.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.297+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1514096f50 con 0x7f15241007a0 2023-10-18T09:16:59.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.540+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1514097320 con 0x7f15241007a0 2023-10-18T09:16:59.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.551+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f1524060c30 con 0x7f150c0a1500 2023-10-18T09:16:59.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.575+0000 7f15217fa700 1 -- 172.21.15.61:0/3093012884 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1524060c30 con 0x7f150c0a1500 2023-10-18T09:16:59.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 msgr2=0x7f150c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 0x7f150c0a39c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1518005fd0 tx=0x7f1518009310 comp rx=0 tx=0).stop 2023-10-18T09:16:59.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 msgr2=0x7f15241967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1514009230 tx=0x7f151400bef0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 shutdown_connections 2023-10-18T09:16:59.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f150c0a1500 0x7f150c0a39c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1524102330 0x7f152419dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f15241007a0 0x7f15241967c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 --2- 172.21.15.61:0/3093012884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1524101990 0x7f1524196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.577+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 >> 172.21.15.61:0/3093012884 conn(0x7f15240fbcf0 msgr2=0x7f1524104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:59.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.578+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 shutdown_connections 2023-10-18T09:16:59.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.578+0000 7f152a3ff700 1 -- 172.21.15.61:0/3093012884 wait complete. 2023-10-18T09:16:59.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-18T09:16:59.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.805+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/3902622971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 msgr2=0x7fe7e8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.805+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/3902622971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8101df0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80099c0 tx=0x7fe7d8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/3902622971 shutdown_connections 2023-10-18T09:16:59.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/3902622971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e8102330 0x7fe7e8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/3902622971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e81007a0 0x7fe7e8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/3902622971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8101df0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/3902622971 >> 172.21.15.61:0/3902622971 conn(0x7fe7e80fbcf0 msgr2=0x7fe7e80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:16:59.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/3902622971 shutdown_connections 2023-10-18T09:16:59.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.806+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/3902622971 wait complete. 2023-10-18T09:16:59.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 Processor -- start 2023-10-18T09:16:59.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 -- start start 2023-10-18T09:16:59.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 0x7fe7e8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e819a5a0 con 0x7fe7e8101990 2023-10-18T09:16:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8073660 con 0x7fe7e8102330 2023-10-18T09:16:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ef3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8073820 con 0x7fe7e81007a0 2023-10-18T09:16:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ed19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ec99a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ed19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39988/0 (socket says 172.21.15.61:39988) 2023-10-18T09:16:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ed19b700 1 -- 172.21.15.61:0/2277411959 learned_addr learned my addr 172.21.15.61:0/2277411959 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:16:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.807+0000 7fe7ed99c700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 0x7fe7e8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 -- 172.21.15.61:0/2277411959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 msgr2=0x7fe7e8073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 0x7fe7e8073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 -- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 msgr2=0x7fe7e8072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:16:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:16:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 -- 172.21.15.61:0/2277411959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80095f0 con 0x7fe7e81007a0 2023-10-18T09:16:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed99c700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 0x7fe7e8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ec99a700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:16:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ed19b700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400e970 tx=0x7fe7e400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4009790 con 0x7fe7e81007a0 2023-10-18T09:16:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e81a4f50 con 0x7fe7e81007a0 2023-10-18T09:16:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7e400c7e0 con 0x7fe7e81007a0 2023-10-18T09:16:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.808+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4003c70 con 0x7fe7e81007a0 2023-10-18T09:16:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.809+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e81a52c0 con 0x7fe7e81007a0 2023-10-18T09:16:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.810+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e8107c40 con 0x7fe7e81007a0 2023-10-18T09:16:59.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.812+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe7e400cc50 con 0x7fe7e81007a0 2023-10-18T09:16:59.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.813+0000 7fe7de7fc700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 0x7fe7d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:16:59.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.813+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe7e4014070 con 0x7fe7e81007a0 2023-10-18T09:16:59.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.813+0000 7fe7ec99a700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 0x7fe7d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:16:59.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.814+0000 7fe7ec99a700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 0x7fe7d40a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8003730 tx=0x7fe7d801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:16:59.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:16:59.817+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7e4096530 con 0x7fe7e81007a0 2023-10-18T09:16:59.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:16:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:active 2023-10-18T09:16:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-18T09:16:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-18T09:16:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: Cluster is now healthy 2023-10-18T09:16:59.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-10-18T09:16:59.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-18T09:16:59.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-10-18T09:16:59.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:16:59 smithi170 ceph-mon[135150]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:16:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:16:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:active 2023-10-18T09:16:59.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-18T09:16:59.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-18T09:16:59.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: Cluster is now healthy 2023-10-18T09:16:59.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-10-18T09:16:59.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-18T09:16:59.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-10-18T09:16:59.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:16:59 smithi110 ceph-mon[134818]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:17:00.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: from='client.24929 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:00.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] up:active 2023-10-18T09:17:00.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-10-18T09:17:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-18T09:17:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: Cluster is now healthy 2023-10-18T09:17:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=f=up:active} 2 up:standby-replay 6 up:standby 2023-10-18T09:17:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2023-10-18T09:17:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: daemon mds.e is now active in filesystem cephfs as rank 4 2023-10-18T09:17:00.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:16:59 smithi061 ceph-mon[98573]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-18T09:17:00.071 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.068+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7e8060c30 con 0x7fe7d40a1510 2023-10-18T09:17:00.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.072+0000 7fe7de7fc700 1 -- 172.21.15.61:0/2277411959 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fe7e8060c30 con 0x7fe7d40a1510 2023-10-18T09:17:00.075 INFO:teuthology.orchestra.run.smithi061.stdout:/volumes/_nogroup/sv_1/916b255b-5c6f-497a-bbc4-d85b7266a4d4 2023-10-18T09:17:00.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 msgr2=0x7fe7d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 0x7fe7d40a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8003730 tx=0x7fe7d801a040 comp rx=0 tx=0).stop 2023-10-18T09:17:00.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 msgr2=0x7fe7e8074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400e970 tx=0x7fe7e400ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:00.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 shutdown_connections 2023-10-18T09:17:00.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe7d40a1510 0x7fe7d40a39d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe7e8102330 0x7fe7e8073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e81007a0 0x7fe7e8074550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 --2- 172.21.15.61:0/2277411959 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe7e8101990 0x7fe7e8072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 >> 172.21.15.61:0/2277411959 conn(0x7fe7e80fbcf0 msgr2=0x7fe7e8104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:00.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 shutdown_connections 2023-10-18T09:17:00.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.075+0000 7fe7ef3ff700 1 -- 172.21.15.61:0/2277411959 wait complete. 2023-10-18T09:17:00.096 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:17:00.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 -- 172.21.15.61:0/1383224359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 msgr2=0x7f626c108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 --2- 172.21.15.61:0/1383224359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c108ab0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f626400b0d0 tx=0x7f626400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 -- 172.21.15.61:0/1383224359 shutdown_connections 2023-10-18T09:17:00.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 --2- 172.21.15.61:0/1383224359 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c108ab0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 --2- 172.21.15.61:0/1383224359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c100670 0x7f626c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 --2- 172.21.15.61:0/1383224359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c101860 0x7f626c101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 -- 172.21.15.61:0/1383224359 >> 172.21.15.61:0/1383224359 conn(0x7f626c0fbba0 msgr2=0x7f626c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:00.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 -- 172.21.15.61:0/1383224359 shutdown_connections 2023-10-18T09:17:00.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.299+0000 7f62731ff700 1 -- 172.21.15.61:0/1383224359 wait complete. 2023-10-18T09:17:00.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.300+0000 7f62731ff700 1 Processor -- start 2023-10-18T09:17:00.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.300+0000 7f62731ff700 1 -- start start 2023-10-18T09:17:00.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c100670 0x7f626c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c101860 0x7f626c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a0440 con 0x7f626c102200 2023-10-18T09:17:00.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a0580 con 0x7f626c101860 2023-10-18T09:17:00.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f62731ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626c1a0710 con 0x7f626c100670 2023-10-18T09:17:00.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41044/0 (socket says 172.21.15.61:41044) 2023-10-18T09:17:00.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 -- 172.21.15.61:0/2016492869 learned_addr learned my addr 172.21.15.61:0/2016492869 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:00.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f6270f9b700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c100670 0x7f626c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f626bfff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c101860 0x7f626c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 -- 172.21.15.61:0/2016492869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c100670 msgr2=0x7f626c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c100670 0x7f626c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 -- 172.21.15.61:0/2016492869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c101860 msgr2=0x7f626c198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.301+0000 7f627179c700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c101860 0x7f626c198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f627179c700 1 -- 172.21.15.61:0/2016492869 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264009d00 con 0x7f626c102200 2023-10-18T09:17:00.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f627179c700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6264007b60 tx=0x7f6264008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:00.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626400e040 con 0x7f626c102200 2023-10-18T09:17:00.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c1a0990 con 0x7f626c102200 2023-10-18T09:17:00.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f626401d930 con 0x7f626c102200 2023-10-18T09:17:00.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626401b9f0 con 0x7f626c102200 2023-10-18T09:17:00.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.302+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626c1a0e10 con 0x7f626c102200 2023-10-18T09:17:00.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.303+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626c0673e0 con 0x7f626c102200 2023-10-18T09:17:00.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.305+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f626401daa0 con 0x7f626c102200 2023-10-18T09:17:00.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.308+0000 7f6269ffb700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 0x7f625c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.308+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f62640ce4c0 con 0x7f626c102200 2023-10-18T09:17:00.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.308+0000 7f6270f9b700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 0x7f625c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.308+0000 7f6270f9b700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 0x7f625c0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f626c1016c0 tx=0x7f6260009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:00.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.312+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6264097800 con 0x7f626c102200 2023-10-18T09:17:00.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.573+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f626c10bee0 con 0x7f626c102200 2023-10-18T09:17:00.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.574+0000 7f6269ffb700 1 -- 172.21.15.61:0/2016492869 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f626409ae20 con 0x7f626c102200 2023-10-18T09:17:00.577 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:00.577 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:17:00.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 msgr2=0x7f625c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 0x7f625c0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f626c1016c0 tx=0x7f6260009260 comp rx=0 tx=0).stop 2023-10-18T09:17:00.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 msgr2=0x7f626c19fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6264007b60 tx=0x7f6264008e40 comp rx=0 tx=0).stop 2023-10-18T09:17:00.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 shutdown_connections 2023-10-18T09:17:00.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f625c0a1510 0x7f625c0a39d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f626c102200 0x7f626c19fe70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f626c100670 0x7f626c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 --2- 172.21.15.61:0/2016492869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f626c101860 0x7f626c198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.576+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 >> 172.21.15.61:0/2016492869 conn(0x7f626c0fbba0 msgr2=0x7f626c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:00.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.577+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 shutdown_connections 2023-10-18T09:17:00.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.577+0000 7f62731ff700 1 -- 172.21.15.61:0/2016492869 wait complete. 2023-10-18T09:17:00.581 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:17:00.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:17:00.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3866517388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 msgr2=0x7f1ac810aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3866517388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac810aa80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000b0a0 tx=0x7f1ac000b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3866517388 shutdown_connections 2023-10-18T09:17:00.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3866517388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac810aa80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3866517388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3866517388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 0x7f1ac8073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3866517388 >> 172.21.15.61:0/3866517388 conn(0x7f1ac80fbc90 msgr2=0x7f1ac80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:00.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3866517388 shutdown_connections 2023-10-18T09:17:00.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.800+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3866517388 wait complete. 2023-10-18T09:17:00.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.801+0000 7f1acd9ff700 1 Processor -- start 2023-10-18T09:17:00.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 -- start start 2023-10-18T09:17:00.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 0x7f1ac8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81a04a0 con 0x7f1ac8108610 2023-10-18T09:17:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81a05e0 con 0x7f1ac80731f0 2023-10-18T09:17:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1acd9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81a0770 con 0x7f1ac80747d0 2023-10-18T09:17:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1ac77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1ac77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41054/0 (socket says 172.21.15.61:41054) 2023-10-18T09:17:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1ac67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1ac77fe700 1 -- 172.21.15.61:0/3051101303 learned_addr learned my addr 172.21.15.61:0/3051101303 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:00.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.802+0000 7f1ac6ffd700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 0x7f1ac8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 -- 172.21.15.61:0/3051101303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 msgr2=0x7f1ac8198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 -- 172.21.15.61:0/3051101303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 msgr2=0x7f1ac8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:00.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 0x7f1ac8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:00.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 -- 172.21.15.61:0/3051101303 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0009d00 con 0x7f1ac8108610 2023-10-18T09:17:00.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac67fc700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:00.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1ac77fe700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0007a60 tx=0x7f1ac0007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:00.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000e040 con 0x7f1ac8108610 2023-10-18T09:17:00.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac0004c50 con 0x7f1ac8108610 2023-10-18T09:17:00.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac81a09f0 con 0x7f1ac8108610 2023-10-18T09:17:00.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001b6c0 con 0x7f1ac8108610 2023-10-18T09:17:00.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.803+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81a0e10 con 0x7f1ac8108610 2023-10-18T09:17:00.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.804+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8107420 con 0x7f1ac8108610 2023-10-18T09:17:00.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.806+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1ac001d470 con 0x7f1ac8108610 2023-10-18T09:17:00.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.808+0000 7f1acc9fd700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 0x7f1ab00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:00.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.808+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1ac00ce100 con 0x7f1ac8108610 2023-10-18T09:17:00.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.808+0000 7f1ac6ffd700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 0x7f1ab00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:00.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.809+0000 7f1ac6ffd700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 0x7f1ab00a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009c60 tx=0x7f1ab8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:00.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:00.812+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ac0097440 con 0x7f1ac8108610 2023-10-18T09:17:00.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:00.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:active 2023-10-18T09:17:00.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-18T09:17:00.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-18T09:17:00.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:17:00.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:00 smithi170 ceph-mon[135150]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:00.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:00.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:active 2023-10-18T09:17:00.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-18T09:17:00.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-18T09:17:00.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:17:00.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:00 smithi110 ceph-mon[134818]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:01.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:01.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] up:active 2023-10-18T09:17:01.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2023-10-18T09:17:01.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-18T09:17:01.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:17:01.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:00 smithi061 ceph-mon[98573]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:01.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.029+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ac810e170 con 0x7f1ac8108610 2023-10-18T09:17:01.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.031+0000 7f1acc9fd700 1 -- 172.21.15.61:0/3051101303 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f1ac009aa60 con 0x7f1ac8108610 2023-10-18T09:17:01.033 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:01.034 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:17:01.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 msgr2=0x7f1ab00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 0x7f1ab00a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009c60 tx=0x7f1ab8009260 comp rx=0 tx=0).stop 2023-10-18T09:17:01.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 msgr2=0x7f1ac819fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0007a60 tx=0x7f1ac0007930 comp rx=0 tx=0).stop 2023-10-18T09:17:01.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 shutdown_connections 2023-10-18T09:17:01.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1ab00a1510 0x7f1ab00a39d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1ac8108610 0x7f1ac819fed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ac80731f0 0x7f1ac8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 --2- 172.21.15.61:0/3051101303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac80747d0 0x7f1ac8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 >> 172.21.15.61:0/3051101303 conn(0x7f1ac80fbc90 msgr2=0x7f1ac8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:01.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 shutdown_connections 2023-10-18T09:17:01.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.033+0000 7f1acd9ff700 1 -- 172.21.15.61:0/3051101303 wait complete. 2023-10-18T09:17:01.055 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-18T09:17:01.055 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:17:01.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/400211533 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 msgr2=0x7ff9b8073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/400211533 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8073670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009990 tx=0x7ff9a0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/400211533 shutdown_connections 2023-10-18T09:17:01.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/400211533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b8108610 0x7ff9b810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/400211533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b80747d0 0x7ff9b8072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/400211533 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8073670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/400211533 >> 172.21.15.61:0/400211533 conn(0x7ff9b80fbc90 msgr2=0x7ff9b80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:01.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/400211533 shutdown_connections 2023-10-18T09:17:01.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.258+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/400211533 wait complete. 2023-10-18T09:17:01.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.259+0000 7ff9bd9ff700 1 Processor -- start 2023-10-18T09:17:01.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 -- start start 2023-10-18T09:17:01.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 0x7ff9b8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b8108610 0x7ff9b819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a0440 con 0x7ff9b80731f0 2023-10-18T09:17:01.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a0580 con 0x7ff9b8108610 2023-10-18T09:17:01.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9bd9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81a0710 con 0x7ff9b80747d0 2023-10-18T09:17:01.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9b6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9affff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 0x7ff9b8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9b6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41058/0 (socket says 172.21.15.61:41058) 2023-10-18T09:17:01.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.260+0000 7ff9b6ffd700 1 -- 172.21.15.61:0/1728727013 learned_addr learned my addr 172.21.15.61:0/1728727013 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:01.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9b77fe700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b8108610 0x7ff9b819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 -- 172.21.15.61:0/1728727013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b8108610 msgr2=0x7ff9b819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b8108610 0x7ff9b819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 -- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 msgr2=0x7ff9b8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 -- 172.21.15.61:0/1728727013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00095f0 con 0x7ff9b80747d0 2023-10-18T09:17:01.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9affff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 0x7ff9b8198ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c00 tx=0x7ff9a0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:01.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a001d070 con 0x7ff9b80747d0 2023-10-18T09:17:01.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b81a0990 con 0x7ff9b80747d0 2023-10-18T09:17:01.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9a0021410 con 0x7ff9b80747d0 2023-10-18T09:17:01.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.261+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0021ea0 con 0x7ff9b80747d0 2023-10-18T09:17:01.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.262+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b81a0db0 con 0x7ff9b80747d0 2023-10-18T09:17:01.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.263+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b8107420 con 0x7ff9b80747d0 2023-10-18T09:17:01.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.264+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff9a0021580 con 0x7ff9b80747d0 2023-10-18T09:17:01.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.267+0000 7ff9bc9fd700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 0x7ff9a40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.267+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff9a00ce290 con 0x7ff9b80747d0 2023-10-18T09:17:01.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.267+0000 7ff9b6ffd700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 0x7ff9a40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.268+0000 7ff9b6ffd700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 0x7ff9a40a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009d20 tx=0x7ff9a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:01.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.270+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff9a0097630 con 0x7ff9b80747d0 2023-10-18T09:17:01.462 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: 2023-10-18T09:17:01.155+0000 7f8e6fe88700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-10-18T09:17:01.462 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: 2023-10-18T09:17:01.155+0000 7f8e6fe88700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-10-18T09:17:01.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.480+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9b8060c30 con 0x7ff9a40a1510 2023-10-18T09:17:01.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.496+0000 7ff9bc9fd700 1 -- 172.21.15.61:0/1728727013 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422552 (secure 0 0 0) 0x7ff9b8060c30 con 0x7ff9a40a1510 2023-10-18T09:17:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:01.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 msgr2=0x7ff9a40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 0x7ff9a40a39d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009d20 tx=0x7ff9a8009260 comp rx=0 tx=0).stop 2023-10-18T09:17:01.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 msgr2=0x7ff9b8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 0x7ff9b8198ea0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c00 tx=0x7ff9a0005740 comp rx=0 tx=0).stop 2023-10-18T09:17:01.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 shutdown_connections 2023-10-18T09:17:01.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff9a40a1510 0x7ff9a40a39d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9b8108610 0x7ff9b819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff9b80731f0 0x7ff9b8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 --2- 172.21.15.61:0/1728727013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9b80747d0 0x7ff9b8198ea0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 >> 172.21.15.61:0/1728727013 conn(0x7ff9b80fbc90 msgr2=0x7ff9b8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:01.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 shutdown_connections 2023-10-18T09:17:01.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.501+0000 7ff9bd9ff700 1 -- 172.21.15.61:0/1728727013 wait complete. 2023-10-18T09:17:01.511 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:17:01.523 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-18T09:17:00.839915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600532,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":147,"num_read_kb":126,"num_write":344,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337540,"kb_used_data":13344,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604284,"statfs":{"total":1151940427776,"available":1151594786816,"internally_reserved":0,"allocated":13664256,"data_stored":7462521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5665,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":116,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003790"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896939+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:16:30.896939+0000","last_peered":"2023-10-18T09:16:30.896939+0000","last_clean":"2023-10-18T09:16:30.896939+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:16:30.896939+0000","last_undegraded":"2023-10-18T09:16:30.896939+0000","last_fullsized":"2023-10-18T09:16:30.896939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899150+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:16:30.899150+0000","last_peered":"2023-10-18T09:16:30.899150+0000","last_clean":"2023-10-18T09:16:30.899150+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:16:30.899150+0000","last_undegraded":"2023-10-18T09:16:30.899150+0000","last_fullsized":"2023-10-18T09:16:30.899150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895178+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:16:30.895178+0000","last_peered":"2023-10-18T09:16:30.895178+0000","last_clean":"2023-10-18T09:16:30.895178+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:16:30.895178+0000","last_undegraded":"2023-10-18T09:16:30.895178+0000","last_fullsized":"2023-10-18T09:16:30.895178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894474+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:16:30.894474+0000","last_peered":"2023-10-18T09:16:30.894474+0000","last_clean":"2023-10-18T09:16:30.894474+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:16:30.894474+0000","last_undegraded":"2023-10-18T09:16:30.894474+0000","last_fullsized":"2023-10-18T09:16:30.894474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.890834+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:16:30.890834+0000","last_peered":"2023-10-18T09:16:30.890834+0000","last_clean":"2023-10-18T09:16:30.890834+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:16:30.890834+0000","last_undegraded":"2023-10-18T09:16:30.890834+0000","last_fullsized":"2023-10-18T09:16:30.890834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913481+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:16:30.913481+0000","last_peered":"2023-10-18T09:16:30.913481+0000","last_clean":"2023-10-18T09:16:30.913481+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:16:30.913481+0000","last_undegraded":"2023-10-18T09:16:30.913481+0000","last_fullsized":"2023-10-18T09:16:30.913481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881754+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:16:30.881754+0000","last_peered":"2023-10-18T09:16:30.881754+0000","last_clean":"2023-10-18T09:16:30.881754+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:16:30.881754+0000","last_undegraded":"2023-10-18T09:16:30.881754+0000","last_fullsized":"2023-10-18T09:16:30.881754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.885710+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:16:30.885710+0000","last_peered":"2023-10-18T09:16:30.885710+0000","last_clean":"2023-10-18T09:16:30.885710+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:16:30.885710+0000","last_undegraded":"2023-10-18T09:16:30.885710+0000","last_fullsized":"2023-10-18T09:16:30.885710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892514+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:16:30.892514+0000","last_peered":"2023-10-18T09:16:30.892514+0000","last_clean":"2023-10-18T09:16:30.892514+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:16:30.892514+0000","last_undegraded":"2023-10-18T09:16:30.892514+0000","last_fullsized":"2023-10-18T09:16:30.892514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913945+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:16:30.913945+0000","last_peered":"2023-10-18T09:16:30.913945+0000","last_clean":"2023-10-18T09:16:30.913945+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:16:30.913945+0000","last_undegraded":"2023-10-18T09:16:30.913945+0000","last_fullsized":"2023-10-18T09:16:30.913945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882374+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:16:30.882374+0000","last_peered":"2023-10-18T09:16:30.882374+0000","last_clean":"2023-10-18T09:16:30.882374+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:16:30.882374+0000","last_undegraded":"2023-10-18T09:16:30.882374+0000","last_fullsized":"2023-10-18T09:16:30.882374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894398+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:16:30.894398+0000","last_peered":"2023-10-18T09:16:30.894398+0000","last_clean":"2023-10-18T09:16:30.894398+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:16:30.894398+0000","last_undegraded":"2023-10-18T09:16:30.894398+0000","last_fullsized":"2023-10-18T09:16:30.894398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894242+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:16:30.894242+0000","last_peered":"2023-10-18T09:16:30.894242+0000","last_clean":"2023-10-18T09:16:30.894242+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:16:30.894242+0000","last_undegraded":"2023-10-18T09:16:30.894242+0000","last_fullsized":"2023-10-18T09:16:30.894242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894421+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:16:30.894421+0000","last_peered":"2023-10-18T09:16:30.894421+0000","last_clean":"2023-10-18T09:16:30.894421+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:16:30.894421+0000","last_undegraded":"2023-10-18T09:16:30.894421+0000","last_fullsized":"2023-10-18T09:16:30.894421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894348+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:16:30.894348+0000","last_peered":"2023-10-18T09:16:30.894348+0000","last_clean":"2023-10-18T09:16:30.894348+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:16:30.894348+0000","last_undegraded":"2023-10-18T09:16:30.894348+0000","last_fullsized":"2023-10-18T09:16:30.894348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891879+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:16:30.891879+0000","last_peered":"2023-10-18T09:16:30.891879+0000","last_clean":"2023-10-18T09:16:30.891879+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:16:30.891879+0000","last_undegraded":"2023-10-18T09:16:30.891879+0000","last_fullsized":"2023-10-18T09:16:30.891879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896852+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:16:30.896852+0000","last_peered":"2023-10-18T09:16:30.896852+0000","last_clean":"2023-10-18T09:16:30.896852+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:16:30.896852+0000","last_undegraded":"2023-10-18T09:16:30.896852+0000","last_fullsized":"2023-10-18T09:16:30.896852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.904116+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:16:30.904116+0000","last_peered":"2023-10-18T09:16:30.904116+0000","last_clean":"2023-10-18T09:16:30.904116+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:16:30.904116+0000","last_undegraded":"2023-10-18T09:16:30.904116+0000","last_fullsized":"2023-10-18T09:16:30.904116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882826+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:16:30.882826+0000","last_peered":"2023-10-18T09:16:30.882826+0000","last_clean":"2023-10-18T09:16:30.882826+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:16:30.882826+0000","last_undegraded":"2023-10-18T09:16:30.882826+0000","last_fullsized":"2023-10-18T09:16:30.882826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499565+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:16:31.499565+0000","last_peered":"2023-10-18T09:16:31.499565+0000","last_clean":"2023-10-18T09:16:31.499565+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:16:31.499565+0000","last_undegraded":"2023-10-18T09:16:31.499565+0000","last_fullsized":"2023-10-18T09:16:31.499565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535374+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:16:58.535374+0000","last_peered":"2023-10-18T09:16:58.535374+0000","last_clean":"2023-10-18T09:16:58.535374+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:16:58.535374+0000","last_undegraded":"2023-10-18T09:16:58.535374+0000","last_fullsized":"2023-10-18T09:16:58.535374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886035+0000","last_change":"2023-10-18T09:16:26.510858+0000","last_active":"2023-10-18T09:16:30.886035+0000","last_peered":"2023-10-18T09:16:30.886035+0000","last_clean":"2023-10-18T09:16:30.886035+0000","last_became_active":"2023-10-18T09:16:26.509795+0000","last_became_peered":"2023-10-18T09:16:26.509795+0000","last_unstale":"2023-10-18T09:16:30.886035+0000","last_undegraded":"2023-10-18T09:16:30.886035+0000","last_fullsized":"2023-10-18T09:16:30.886035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:01.458412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978286+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:16:31.978286+0000","last_peered":"2023-10-18T09:16:31.978286+0000","last_clean":"2023-10-18T09:16:31.978286+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:16:31.978286+0000","last_undegraded":"2023-10-18T09:16:31.978286+0000","last_fullsized":"2023-10-18T09:16:31.978286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.994315+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:16:31.994315+0000","last_peered":"2023-10-18T09:16:31.994315+0000","last_clean":"2023-10-18T09:16:31.994315+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:16:31.994315+0000","last_undegraded":"2023-10-18T09:16:31.994315+0000","last_fullsized":"2023-10-18T09:16:31.994315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899318+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:16:30.899318+0000","last_peered":"2023-10-18T09:16:30.899318+0000","last_clean":"2023-10-18T09:16:30.899318+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:16:30.899318+0000","last_undegraded":"2023-10-18T09:16:30.899318+0000","last_fullsized":"2023-10-18T09:16:30.899318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979121+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:16:31.979121+0000","last_peered":"2023-10-18T09:16:31.979121+0000","last_clean":"2023-10-18T09:16:31.979121+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:16:31.979121+0000","last_undegraded":"2023-10-18T09:16:31.979121+0000","last_fullsized":"2023-10-18T09:16:31.979121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535432+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:16:58.535432+0000","last_peered":"2023-10-18T09:16:58.535432+0000","last_clean":"2023-10-18T09:16:58.535432+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:16:58.535432+0000","last_undegraded":"2023-10-18T09:16:58.535432+0000","last_fullsized":"2023-10-18T09:16:58.535432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892543+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:16:30.892543+0000","last_peered":"2023-10-18T09:16:30.892543+0000","last_clean":"2023-10-18T09:16:30.892543+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:16:30.892543+0000","last_undegraded":"2023-10-18T09:16:30.892543+0000","last_fullsized":"2023-10-18T09:16:30.892543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978412+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:16:31.978412+0000","last_peered":"2023-10-18T09:16:31.978412+0000","last_clean":"2023-10-18T09:16:31.978412+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:16:31.978412+0000","last_undegraded":"2023-10-18T09:16:31.978412+0000","last_fullsized":"2023-10-18T09:16:31.978412+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535988+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:16:58.535988+0000","last_peered":"2023-10-18T09:16:58.535988+0000","last_clean":"2023-10-18T09:16:58.535988+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:16:58.535988+0000","last_undegraded":"2023-10-18T09:16:58.535988+0000","last_fullsized":"2023-10-18T09:16:58.535988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894243+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:16:30.894243+0000","last_peered":"2023-10-18T09:16:30.894243+0000","last_clean":"2023-10-18T09:16:30.894243+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:16:30.894243+0000","last_undegraded":"2023-10-18T09:16:30.894243+0000","last_fullsized":"2023-10-18T09:16:30.894243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499907+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:16:31.499907+0000","last_peered":"2023-10-18T09:16:31.499907+0000","last_clean":"2023-10-18T09:16:31.499907+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:16:31.499907+0000","last_undegraded":"2023-10-18T09:16:31.499907+0000","last_fullsized":"2023-10-18T09:16:31.499907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.520136+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:16:58.520136+0000","last_peered":"2023-10-18T09:16:58.520136+0000","last_clean":"2023-10-18T09:16:58.520136+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:16:58.520136+0000","last_undegraded":"2023-10-18T09:16:58.520136+0000","last_fullsized":"2023-10-18T09:16:58.520136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891722+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:16:30.891722+0000","last_peered":"2023-10-18T09:16:30.891722+0000","last_clean":"2023-10-18T09:16:30.891722+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:16:30.891722+0000","last_undegraded":"2023-10-18T09:16:30.891722+0000","last_fullsized":"2023-10-18T09:16:30.891722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.500073+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:16:31.500073+0000","last_peered":"2023-10-18T09:16:31.500073+0000","last_clean":"2023-10-18T09:16:31.500073+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:16:31.500073+0000","last_undegraded":"2023-10-18T09:16:31.500073+0000","last_fullsized":"2023-10-18T09:16:31.500073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913415+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:16:30.913415+0000","last_peered":"2023-10-18T09:16:30.913415+0000","last_clean":"2023-10-18T09:16:30.913415+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:16:30.913415+0000","last_undegraded":"2023-10-18T09:16:30.913415+0000","last_fullsized":"2023-10-18T09:16:30.913415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891407+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:16:30.891407+0000","last_peered":"2023-10-18T09:16:30.891407+0000","last_clean":"2023-10-18T09:16:30.891407+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:16:30.891407+0000","last_undegraded":"2023-10-18T09:16:30.891407+0000","last_fullsized":"2023-10-18T09:16:30.891407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499032+0000","last_change":"2023-10-18T09:16:30.910525+0000","last_active":"2023-10-18T09:16:31.499032+0000","last_peered":"2023-10-18T09:16:31.499032+0000","last_clean":"2023-10-18T09:16:31.499032+0000","last_became_active":"2023-10-18T09:16:30.909960+0000","last_became_peered":"2023-10-18T09:16:30.909960+0000","last_unstale":"2023-10-18T09:16:31.499032+0000","last_undegraded":"2023-10-18T09:16:31.499032+0000","last_fullsized":"2023-10-18T09:16:31.499032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:21.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894017+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:16:30.894017+0000","last_peered":"2023-10-18T09:16:30.894017+0000","last_clean":"2023-10-18T09:16:30.894017+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:16:30.894017+0000","last_undegraded":"2023-10-18T09:16:30.894017+0000","last_fullsized":"2023-10-18T09:16:30.894017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.885819+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:16:30.885819+0000","last_peered":"2023-10-18T09:16:30.885819+0000","last_clean":"2023-10-18T09:16:30.885819+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:16:30.885819+0000","last_undegraded":"2023-10-18T09:16:30.885819+0000","last_fullsized":"2023-10-18T09:16:30.885819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979112+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:16:31.979112+0000","last_peered":"2023-10-18T09:16:31.979112+0000","last_clean":"2023-10-18T09:16:31.979112+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:16:31.979112+0000","last_undegraded":"2023-10-18T09:16:31.979112+0000","last_fullsized":"2023-10-18T09:16:31.979112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.524545+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:16:57.524545+0000","last_peered":"2023-10-18T09:16:57.524545+0000","last_clean":"2023-10-18T09:16:57.524545+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:16:57.524545+0000","last_undegraded":"2023-10-18T09:16:57.524545+0000","last_fullsized":"2023-10-18T09:16:57.524545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.901444+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:16:30.901444+0000","last_peered":"2023-10-18T09:16:30.901444+0000","last_clean":"2023-10-18T09:16:30.901444+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:16:30.901444+0000","last_undegraded":"2023-10-18T09:16:30.901444+0000","last_fullsized":"2023-10-18T09:16:30.901444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.993802+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:16:31.993802+0000","last_peered":"2023-10-18T09:16:31.993802+0000","last_clean":"2023-10-18T09:16:31.993802+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:16:31.993802+0000","last_undegraded":"2023-10-18T09:16:31.993802+0000","last_fullsized":"2023-10-18T09:16:31.993802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.914017+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:16:30.914017+0000","last_peered":"2023-10-18T09:16:30.914017+0000","last_clean":"2023-10-18T09:16:30.914017+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:16:30.914017+0000","last_undegraded":"2023-10-18T09:16:30.914017+0000","last_fullsized":"2023-10-18T09:16:30.914017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893229+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:16:30.893229+0000","last_peered":"2023-10-18T09:16:30.893229+0000","last_clean":"2023-10-18T09:16:30.893229+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:16:30.893229+0000","last_undegraded":"2023-10-18T09:16:30.893229+0000","last_fullsized":"2023-10-18T09:16:30.893229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499129+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:16:31.499129+0000","last_peered":"2023-10-18T09:16:31.499129+0000","last_clean":"2023-10-18T09:16:31.499129+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:16:31.499129+0000","last_undegraded":"2023-10-18T09:16:31.499129+0000","last_fullsized":"2023-10-18T09:16:31.499129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.921053+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:16:55.921053+0000","last_peered":"2023-10-18T09:16:55.921053+0000","last_clean":"2023-10-18T09:16:55.921053+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:16:55.921053+0000","last_undegraded":"2023-10-18T09:16:55.921053+0000","last_fullsized":"2023-10-18T09:16:55.921053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899414+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:16:30.899414+0000","last_peered":"2023-10-18T09:16:30.899414+0000","last_clean":"2023-10-18T09:16:30.899414+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:16:30.899414+0000","last_undegraded":"2023-10-18T09:16:30.899414+0000","last_fullsized":"2023-10-18T09:16:30.899414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979419+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:16:31.979419+0000","last_peered":"2023-10-18T09:16:31.979419+0000","last_clean":"2023-10-18T09:16:31.979419+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:16:31.979419+0000","last_undegraded":"2023-10-18T09:16:31.979419+0000","last_fullsized":"2023-10-18T09:16:31.979419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:56.513262+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:16:56.513262+0000","last_peered":"2023-10-18T09:16:56.513262+0000","last_clean":"2023-10-18T09:16:56.513262+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:16:56.513262+0000","last_undegraded":"2023-10-18T09:16:56.513262+0000","last_fullsized":"2023-10-18T09:16:56.513262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892709+0000","last_change":"2023-10-18T09:16:26.520057+0000","last_active":"2023-10-18T09:16:30.892709+0000","last_peered":"2023-10-18T09:16:30.892709+0000","last_clean":"2023-10-18T09:16:30.892709+0000","last_became_active":"2023-10-18T09:16:26.518997+0000","last_became_peered":"2023-10-18T09:16:26.518997+0000","last_unstale":"2023-10-18T09:16:30.892709+0000","last_undegraded":"2023-10-18T09:16:30.892709+0000","last_fullsized":"2023-10-18T09:16:30.892709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:54.779476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499191+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:16:31.499191+0000","last_peered":"2023-10-18T09:16:31.499191+0000","last_clean":"2023-10-18T09:16:31.499191+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:16:31.499191+0000","last_undegraded":"2023-10-18T09:16:31.499191+0000","last_fullsized":"2023-10-18T09:16:31.499191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:46.614761+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:16:46.614761+0000","last_peered":"2023-10-18T09:16:46.614761+0000","last_clean":"2023-10-18T09:16:46.614761+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:16:46.614761+0000","last_undegraded":"2023-10-18T09:16:46.614761+0000","last_fullsized":"2023-10-18T09:16:46.614761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891040+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:16:30.891040+0000","last_peered":"2023-10-18T09:16:30.891040+0000","last_clean":"2023-10-18T09:16:30.891040+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:16:30.891040+0000","last_undegraded":"2023-10-18T09:16:30.891040+0000","last_fullsized":"2023-10-18T09:16:30.891040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979353+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:16:31.979353+0000","last_peered":"2023-10-18T09:16:31.979353+0000","last_clean":"2023-10-18T09:16:31.979353+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:16:31.979353+0000","last_undegraded":"2023-10-18T09:16:31.979353+0000","last_fullsized":"2023-10-18T09:16:31.979353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535609+0000","last_change":"2023-10-18T09:16:25.864359+0000","last_active":"2023-10-18T09:16:58.535609+0000","last_peered":"2023-10-18T09:16:58.535609+0000","last_clean":"2023-10-18T09:16:58.535609+0000","last_became_active":"2023-10-18T09:16:25.864019+0000","last_became_peered":"2023-10-18T09:16:25.864019+0000","last_unstale":"2023-10-18T09:16:58.535609+0000","last_undegraded":"2023-10-18T09:16:58.535609+0000","last_fullsized":"2023-10-18T09:16:58.535609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:36.407133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.897598+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:16:30.897598+0000","last_peered":"2023-10-18T09:16:30.897598+0000","last_clean":"2023-10-18T09:16:30.897598+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:16:30.897598+0000","last_undegraded":"2023-10-18T09:16:30.897598+0000","last_fullsized":"2023-10-18T09:16:30.897598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.498769+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:16:31.498769+0000","last_peered":"2023-10-18T09:16:31.498769+0000","last_clean":"2023-10-18T09:16:31.498769+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:16:31.498769+0000","last_undegraded":"2023-10-18T09:16:31.498769+0000","last_fullsized":"2023-10-18T09:16:31.498769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526285+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:16:57.526285+0000","last_peered":"2023-10-18T09:16:57.526285+0000","last_clean":"2023-10-18T09:16:57.526285+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:16:57.526285+0000","last_undegraded":"2023-10-18T09:16:57.526285+0000","last_fullsized":"2023-10-18T09:16:57.526285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882100+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:16:30.882100+0000","last_peered":"2023-10-18T09:16:30.882100+0000","last_clean":"2023-10-18T09:16:30.882100+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:16:30.882100+0000","last_undegraded":"2023-10-18T09:16:30.882100+0000","last_fullsized":"2023-10-18T09:16:30.882100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499442+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:16:31.499442+0000","last_peered":"2023-10-18T09:16:31.499442+0000","last_clean":"2023-10-18T09:16:31.499442+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:16:31.499442+0000","last_undegraded":"2023-10-18T09:16:31.499442+0000","last_fullsized":"2023-10-18T09:16:31.499442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.524500+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:16:57.524500+0000","last_peered":"2023-10-18T09:16:57.524500+0000","last_clean":"2023-10-18T09:16:57.524500+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:16:57.524500+0000","last_undegraded":"2023-10-18T09:16:57.524500+0000","last_fullsized":"2023-10-18T09:16:57.524500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881818+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:16:30.881818+0000","last_peered":"2023-10-18T09:16:30.881818+0000","last_clean":"2023-10-18T09:16:30.881818+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:16:30.881818+0000","last_undegraded":"2023-10-18T09:16:30.881818+0000","last_fullsized":"2023-10-18T09:16:30.881818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978777+0000","last_change":"2023-10-18T09:16:30.910478+0000","last_active":"2023-10-18T09:16:31.978777+0000","last_peered":"2023-10-18T09:16:31.978777+0000","last_clean":"2023-10-18T09:16:31.978777+0000","last_became_active":"2023-10-18T09:16:30.910036+0000","last_became_peered":"2023-10-18T09:16:30.910036+0000","last_unstale":"2023-10-18T09:16:31.978777+0000","last_undegraded":"2023-10-18T09:16:31.978777+0000","last_fullsized":"2023-10-18T09:16:31.978777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:12:55.671867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.525217+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:16:57.525217+0000","last_peered":"2023-10-18T09:16:57.525217+0000","last_clean":"2023-10-18T09:16:57.525217+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:16:57.525217+0000","last_undegraded":"2023-10-18T09:16:57.525217+0000","last_fullsized":"2023-10-18T09:16:57.525217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893351+0000","last_change":"2023-10-18T09:16:26.516754+0000","last_active":"2023-10-18T09:16:30.893351+0000","last_peered":"2023-10-18T09:16:30.893351+0000","last_clean":"2023-10-18T09:16:30.893351+0000","last_became_active":"2023-10-18T09:16:26.516350+0000","last_became_peered":"2023-10-18T09:16:26.516350+0000","last_unstale":"2023-10-18T09:16:30.893351+0000","last_undegraded":"2023-10-18T09:16:30.893351+0000","last_fullsized":"2023-10-18T09:16:30.893351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:37.396217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979040+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:16:31.979040+0000","last_peered":"2023-10-18T09:16:31.979040+0000","last_clean":"2023-10-18T09:16:31.979040+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:16:31.979040+0000","last_undegraded":"2023-10-18T09:16:31.979040+0000","last_fullsized":"2023-10-18T09:16:31.979040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978360+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:16:31.978360+0000","last_peered":"2023-10-18T09:16:31.978360+0000","last_clean":"2023-10-18T09:16:31.978360+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:16:31.978360+0000","last_undegraded":"2023-10-18T09:16:31.978360+0000","last_fullsized":"2023-10-18T09:16:31.978360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895086+0000","last_change":"2023-10-18T09:16:26.517973+0000","last_active":"2023-10-18T09:16:30.895086+0000","last_peered":"2023-10-18T09:16:30.895086+0000","last_clean":"2023-10-18T09:16:30.895086+0000","last_became_active":"2023-10-18T09:16:26.517708+0000","last_became_peered":"2023-10-18T09:16:26.517708+0000","last_unstale":"2023-10-18T09:16:30.895086+0000","last_undegraded":"2023-10-18T09:16:30.895086+0000","last_fullsized":"2023-10-18T09:16:30.895086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:07.086081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978393+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:16:31.978393+0000","last_peered":"2023-10-18T09:16:31.978393+0000","last_clean":"2023-10-18T09:16:31.978393+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:16:31.978393+0000","last_undegraded":"2023-10-18T09:16:31.978393+0000","last_fullsized":"2023-10-18T09:16:31.978393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499981+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:16:31.499981+0000","last_peered":"2023-10-18T09:16:31.499981+0000","last_clean":"2023-10-18T09:16:31.499981+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:16:31.499981+0000","last_undegraded":"2023-10-18T09:16:31.499981+0000","last_fullsized":"2023-10-18T09:16:31.499981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913739+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:16:30.913739+0000","last_peered":"2023-10-18T09:16:30.913739+0000","last_clean":"2023-10-18T09:16:30.913739+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:16:30.913739+0000","last_undegraded":"2023-10-18T09:16:30.913739+0000","last_fullsized":"2023-10-18T09:16:30.913739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978082+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:16:31.978082+0000","last_peered":"2023-10-18T09:16:31.978082+0000","last_clean":"2023-10-18T09:16:31.978082+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:16:31.978082+0000","last_undegraded":"2023-10-18T09:16:31.978082+0000","last_fullsized":"2023-10-18T09:16:31.978082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978370+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:16:31.978370+0000","last_peered":"2023-10-18T09:16:31.978370+0000","last_clean":"2023-10-18T09:16:31.978370+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:16:31.978370+0000","last_undegraded":"2023-10-18T09:16:31.978370+0000","last_fullsized":"2023-10-18T09:16:31.978370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913574+0000","last_change":"2023-10-18T09:16:26.518651+0000","last_active":"2023-10-18T09:16:30.913574+0000","last_peered":"2023-10-18T09:16:30.913574+0000","last_clean":"2023-10-18T09:16:30.913574+0000","last_became_active":"2023-10-18T09:16:26.518365+0000","last_became_peered":"2023-10-18T09:16:26.518365+0000","last_unstale":"2023-10-18T09:16:30.913574+0000","last_undegraded":"2023-10-18T09:16:30.913574+0000","last_fullsized":"2023-10-18T09:16:30.913574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:02.868372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979109+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:16:31.979109+0000","last_peered":"2023-10-18T09:16:31.979109+0000","last_clean":"2023-10-18T09:16:31.979109+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:16:31.979109+0000","last_undegraded":"2023-10-18T09:16:31.979109+0000","last_fullsized":"2023-10-18T09:16:31.979109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891585+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:16:30.891585+0000","last_peered":"2023-10-18T09:16:30.891585+0000","last_clean":"2023-10-18T09:16:30.891585+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:16:30.891585+0000","last_undegraded":"2023-10-18T09:16:30.891585+0000","last_fullsized":"2023-10-18T09:16:30.891585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.977610+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:16:31.977610+0000","last_peered":"2023-10-18T09:16:31.977610+0000","last_clean":"2023-10-18T09:16:31.977610+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:16:31.977610+0000","last_undegraded":"2023-10-18T09:16:31.977610+0000","last_fullsized":"2023-10-18T09:16:31.977610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979926+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:16:31.979926+0000","last_peered":"2023-10-18T09:16:31.979926+0000","last_clean":"2023-10-18T09:16:31.979926+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:16:31.979926+0000","last_undegraded":"2023-10-18T09:16:31.979926+0000","last_fullsized":"2023-10-18T09:16:31.979926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886038+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:16:30.886038+0000","last_peered":"2023-10-18T09:16:30.886038+0000","last_clean":"2023-10-18T09:16:30.886038+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:16:30.886038+0000","last_undegraded":"2023-10-18T09:16:30.886038+0000","last_fullsized":"2023-10-18T09:16:30.886038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.527172+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:16:57.527172+0000","last_peered":"2023-10-18T09:16:57.527172+0000","last_clean":"2023-10-18T09:16:57.527172+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:16:57.527172+0000","last_undegraded":"2023-10-18T09:16:57.527172+0000","last_fullsized":"2023-10-18T09:16:57.527172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":311,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881635+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:16:30.881635+0000","last_peered":"2023-10-18T09:16:30.881635+0000","last_clean":"2023-10-18T09:16:30.881635+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:16:30.881635+0000","last_undegraded":"2023-10-18T09:16:30.881635+0000","last_fullsized":"2023-10-18T09:16:30.881635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979156+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:16:31.979156+0000","last_peered":"2023-10-18T09:16:31.979156+0000","last_clean":"2023-10-18T09:16:31.979156+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:16:31.979156+0000","last_undegraded":"2023-10-18T09:16:31.979156+0000","last_fullsized":"2023-10-18T09:16:31.979156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893884+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:16:30.893884+0000","last_peered":"2023-10-18T09:16:30.893884+0000","last_clean":"2023-10-18T09:16:30.893884+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:16:30.893884+0000","last_undegraded":"2023-10-18T09:16:30.893884+0000","last_fullsized":"2023-10-18T09:16:30.893884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:56.517486+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:16:56.517486+0000","last_peered":"2023-10-18T09:16:56.517486+0000","last_clean":"2023-10-18T09:16:56.517486+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:16:56.517486+0000","last_undegraded":"2023-10-18T09:16:56.517486+0000","last_fullsized":"2023-10-18T09:16:56.517486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.498729+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:16:31.498729+0000","last_peered":"2023-10-18T09:16:31.498729+0000","last_clean":"2023-10-18T09:16:31.498729+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:16:31.498729+0000","last_undegraded":"2023-10-18T09:16:31.498729+0000","last_fullsized":"2023-10-18T09:16:31.498729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.527439+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:16:57.527439+0000","last_peered":"2023-10-18T09:16:57.527439+0000","last_clean":"2023-10-18T09:16:57.527439+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:16:57.527439+0000","last_undegraded":"2023-10-18T09:16:57.527439+0000","last_fullsized":"2023-10-18T09:16:57.527439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894211+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:16:30.894211+0000","last_peered":"2023-10-18T09:16:30.894211+0000","last_clean":"2023-10-18T09:16:30.894211+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:16:30.894211+0000","last_undegraded":"2023-10-18T09:16:30.894211+0000","last_fullsized":"2023-10-18T09:16:30.894211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.993743+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:16:31.993743+0000","last_peered":"2023-10-18T09:16:31.993743+0000","last_clean":"2023-10-18T09:16:31.993743+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:16:31.993743+0000","last_undegraded":"2023-10-18T09:16:31.993743+0000","last_fullsized":"2023-10-18T09:16:31.993743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.528421+0000","last_change":"2023-10-18T09:16:25.866900+0000","last_active":"2023-10-18T09:16:57.528421+0000","last_peered":"2023-10-18T09:16:57.528421+0000","last_clean":"2023-10-18T09:16:57.528421+0000","last_became_active":"2023-10-18T09:16:25.866545+0000","last_became_peered":"2023-10-18T09:16:25.866545+0000","last_unstale":"2023-10-18T09:16:57.528421+0000","last_undegraded":"2023-10-18T09:16:57.528421+0000","last_fullsized":"2023-10-18T09:16:57.528421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:01.437689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886952+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:16:30.886952+0000","last_peered":"2023-10-18T09:16:30.886952+0000","last_clean":"2023-10-18T09:16:30.886952+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:16:30.886952+0000","last_undegraded":"2023-10-18T09:16:30.886952+0000","last_fullsized":"2023-10-18T09:16:30.886952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499444+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:16:31.499444+0000","last_peered":"2023-10-18T09:16:31.499444+0000","last_clean":"2023-10-18T09:16:31.499444+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:16:31.499444+0000","last_undegraded":"2023-10-18T09:16:31.499444+0000","last_fullsized":"2023-10-18T09:16:31.499444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.536235+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:16:58.536235+0000","last_peered":"2023-10-18T09:16:58.536235+0000","last_clean":"2023-10-18T09:16:58.536235+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:16:58.536235+0000","last_undegraded":"2023-10-18T09:16:58.536235+0000","last_fullsized":"2023-10-18T09:16:58.536235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899243+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:16:30.899243+0000","last_peered":"2023-10-18T09:16:30.899243+0000","last_clean":"2023-10-18T09:16:30.899243+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:16:30.899243+0000","last_undegraded":"2023-10-18T09:16:30.899243+0000","last_fullsized":"2023-10-18T09:16:30.899243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979193+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:16:31.979193+0000","last_peered":"2023-10-18T09:16:31.979193+0000","last_clean":"2023-10-18T09:16:31.979193+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:16:31.979193+0000","last_undegraded":"2023-10-18T09:16:31.979193+0000","last_fullsized":"2023-10-18T09:16:31.979193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.922535+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:16:55.922535+0000","last_peered":"2023-10-18T09:16:55.922535+0000","last_clean":"2023-10-18T09:16:55.922535+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:16:55.922535+0000","last_undegraded":"2023-10-18T09:16:55.922535+0000","last_fullsized":"2023-10-18T09:16:55.922535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.890715+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:16:30.890715+0000","last_peered":"2023-10-18T09:16:30.890715+0000","last_clean":"2023-10-18T09:16:30.890715+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:16:30.890715+0000","last_undegraded":"2023-10-18T09:16:30.890715+0000","last_fullsized":"2023-10-18T09:16:30.890715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499562+0000","last_change":"2023-10-18T09:16:30.908660+0000","last_active":"2023-10-18T09:16:31.499562+0000","last_peered":"2023-10-18T09:16:31.499562+0000","last_clean":"2023-10-18T09:16:31.499562+0000","last_became_active":"2023-10-18T09:16:30.906449+0000","last_became_peered":"2023-10-18T09:16:30.906449+0000","last_unstale":"2023-10-18T09:16:31.499562+0000","last_undegraded":"2023-10-18T09:16:31.499562+0000","last_fullsized":"2023-10-18T09:16:31.499562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:03:22.596758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.923681+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:16:55.923681+0000","last_peered":"2023-10-18T09:16:55.923681+0000","last_clean":"2023-10-18T09:16:55.923681+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:16:55.923681+0000","last_undegraded":"2023-10-18T09:16:55.923681+0000","last_fullsized":"2023-10-18T09:16:55.923681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894142+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:16:30.894142+0000","last_peered":"2023-10-18T09:16:30.894142+0000","last_clean":"2023-10-18T09:16:30.894142+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:16:30.894142+0000","last_undegraded":"2023-10-18T09:16:30.894142+0000","last_fullsized":"2023-10-18T09:16:30.894142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979153+0000","last_change":"2023-10-18T09:16:30.909709+0000","last_active":"2023-10-18T09:16:31.979153+0000","last_peered":"2023-10-18T09:16:31.979153+0000","last_clean":"2023-10-18T09:16:31.979153+0000","last_became_active":"2023-10-18T09:16:30.909187+0000","last_became_peered":"2023-10-18T09:16:30.909187+0000","last_unstale":"2023-10-18T09:16:31.979153+0000","last_undegraded":"2023-10-18T09:16:31.979153+0000","last_fullsized":"2023-10-18T09:16:31.979153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:14:01.975160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526205+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:16:57.526205+0000","last_peered":"2023-10-18T09:16:57.526205+0000","last_clean":"2023-10-18T09:16:57.526205+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:16:57.526205+0000","last_undegraded":"2023-10-18T09:16:57.526205+0000","last_fullsized":"2023-10-18T09:16:57.526205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886513+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:16:30.886513+0000","last_peered":"2023-10-18T09:16:30.886513+0000","last_clean":"2023-10-18T09:16:30.886513+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:16:30.886513+0000","last_undegraded":"2023-10-18T09:16:30.886513+0000","last_fullsized":"2023-10-18T09:16:30.886513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978879+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:16:31.978879+0000","last_peered":"2023-10-18T09:16:31.978879+0000","last_clean":"2023-10-18T09:16:31.978879+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:16:31.978879+0000","last_undegraded":"2023-10-18T09:16:31.978879+0000","last_fullsized":"2023-10-18T09:16:31.978879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.924188+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:16:55.924188+0000","last_peered":"2023-10-18T09:16:55.924188+0000","last_clean":"2023-10-18T09:16:55.924188+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:16:55.924188+0000","last_undegraded":"2023-10-18T09:16:55.924188+0000","last_fullsized":"2023-10-18T09:16:55.924188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881949+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:16:30.881949+0000","last_peered":"2023-10-18T09:16:30.881949+0000","last_clean":"2023-10-18T09:16:30.881949+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:16:30.881949+0000","last_undegraded":"2023-10-18T09:16:30.881949+0000","last_fullsized":"2023-10-18T09:16:30.881949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.680106+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:16:31.680106+0000","last_peered":"2023-10-18T09:16:31.680106+0000","last_clean":"2023-10-18T09:16:31.680106+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:16:31.680106+0000","last_undegraded":"2023-10-18T09:16:31.680106+0000","last_fullsized":"2023-10-18T09:16:31.680106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:59.551894+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:16:59.551894+0000","last_peered":"2023-10-18T09:16:59.551894+0000","last_clean":"2023-10-18T09:16:59.551894+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:16:59.551894+0000","last_undegraded":"2023-10-18T09:16:59.551894+0000","last_fullsized":"2023-10-18T09:16:59.551894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882434+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:16:30.882434+0000","last_peered":"2023-10-18T09:16:30.882434+0000","last_clean":"2023-10-18T09:16:30.882434+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:16:30.882434+0000","last_undegraded":"2023-10-18T09:16:30.882434+0000","last_fullsized":"2023-10-18T09:16:30.882434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978869+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:16:31.978869+0000","last_peered":"2023-10-18T09:16:31.978869+0000","last_clean":"2023-10-18T09:16:31.978869+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:16:31.978869+0000","last_undegraded":"2023-10-18T09:16:31.978869+0000","last_fullsized":"2023-10-18T09:16:31.978869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526493+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:16:57.526493+0000","last_peered":"2023-10-18T09:16:57.526493+0000","last_clean":"2023-10-18T09:16:57.526493+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:16:57.526493+0000","last_undegraded":"2023-10-18T09:16:57.526493+0000","last_fullsized":"2023-10-18T09:16:57.526493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882212+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:16:30.882212+0000","last_peered":"2023-10-18T09:16:30.882212+0000","last_clean":"2023-10-18T09:16:30.882212+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:16:30.882212+0000","last_undegraded":"2023-10-18T09:16:30.882212+0000","last_fullsized":"2023-10-18T09:16:30.882212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978052+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:16:31.978052+0000","last_peered":"2023-10-18T09:16:31.978052+0000","last_clean":"2023-10-18T09:16:31.978052+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:16:31.978052+0000","last_undegraded":"2023-10-18T09:16:31.978052+0000","last_fullsized":"2023-10-18T09:16:31.978052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896459+0000","last_change":"2023-10-18T09:16:25.867090+0000","last_active":"2023-10-18T09:16:30.896459+0000","last_peered":"2023-10-18T09:16:30.896459+0000","last_clean":"2023-10-18T09:16:30.896459+0000","last_became_active":"2023-10-18T09:16:25.864322+0000","last_became_peered":"2023-10-18T09:16:25.864322+0000","last_unstale":"2023-10-18T09:16:30.896459+0000","last_undegraded":"2023-10-18T09:16:30.896459+0000","last_fullsized":"2023-10-18T09:16:30.896459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:23:31.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882392+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:16:30.882392+0000","last_peered":"2023-10-18T09:16:30.882392+0000","last_clean":"2023-10-18T09:16:30.882392+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:16:30.882392+0000","last_undegraded":"2023-10-18T09:16:30.882392+0000","last_fullsized":"2023-10-18T09:16:30.882392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894259+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:16:30.894259+0000","last_peered":"2023-10-18T09:16:30.894259+0000","last_clean":"2023-10-18T09:16:30.894259+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:16:30.894259+0000","last_undegraded":"2023-10-18T09:16:30.894259+0000","last_fullsized":"2023-10-18T09:16:30.894259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.883101+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:16:30.883101+0000","last_peered":"2023-10-18T09:16:30.883101+0000","last_clean":"2023-10-18T09:16:30.883101+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:16:30.883101+0000","last_undegraded":"2023-10-18T09:16:30.883101+0000","last_fullsized":"2023-10-18T09:16:30.883101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881853+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:16:30.881853+0000","last_peered":"2023-10-18T09:16:30.881853+0000","last_clean":"2023-10-18T09:16:30.881853+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:16:30.881853+0000","last_undegraded":"2023-10-18T09:16:30.881853+0000","last_fullsized":"2023-10-18T09:16:30.881853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896602+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:16:30.896602+0000","last_peered":"2023-10-18T09:16:30.896602+0000","last_clean":"2023-10-18T09:16:30.896602+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:16:30.896602+0000","last_undegraded":"2023-10-18T09:16:30.896602+0000","last_fullsized":"2023-10-18T09:16:30.896602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894042+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:16:30.894042+0000","last_peered":"2023-10-18T09:16:30.894042+0000","last_clean":"2023-10-18T09:16:30.894042+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:16:30.894042+0000","last_undegraded":"2023-10-18T09:16:30.894042+0000","last_fullsized":"2023-10-18T09:16:30.894042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895354+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:16:30.895354+0000","last_peered":"2023-10-18T09:16:30.895354+0000","last_clean":"2023-10-18T09:16:30.895354+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:16:30.895354+0000","last_undegraded":"2023-10-18T09:16:30.895354+0000","last_fullsized":"2023-10-18T09:16:30.895354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882307+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:16:30.882307+0000","last_peered":"2023-10-18T09:16:30.882307+0000","last_clean":"2023-10-18T09:16:30.882307+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:16:30.882307+0000","last_undegraded":"2023-10-18T09:16:30.882307+0000","last_fullsized":"2023-10-18T09:16:30.882307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886859+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:16:30.886859+0000","last_peered":"2023-10-18T09:16:30.886859+0000","last_clean":"2023-10-18T09:16:30.886859+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:16:30.886859+0000","last_undegraded":"2023-10-18T09:16:30.886859+0000","last_fullsized":"2023-10-18T09:16:30.886859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891500+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:16:30.891500+0000","last_peered":"2023-10-18T09:16:30.891500+0000","last_clean":"2023-10-18T09:16:30.891500+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:16:30.891500+0000","last_undegraded":"2023-10-18T09:16:30.891500+0000","last_fullsized":"2023-10-18T09:16:30.891500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886167+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:16:30.886167+0000","last_peered":"2023-10-18T09:16:30.886167+0000","last_clean":"2023-10-18T09:16:30.886167+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:16:30.886167+0000","last_undegraded":"2023-10-18T09:16:30.886167+0000","last_fullsized":"2023-10-18T09:16:30.886167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892991+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:16:30.892991+0000","last_peered":"2023-10-18T09:16:30.892991+0000","last_clean":"2023-10-18T09:16:30.892991+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:16:30.892991+0000","last_undegraded":"2023-10-18T09:16:30.892991+0000","last_fullsized":"2023-10-18T09:16:30.892991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894187+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:16:30.894187+0000","last_peered":"2023-10-18T09:16:30.894187+0000","last_clean":"2023-10-18T09:16:30.894187+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:16:30.894187+0000","last_undegraded":"2023-10-18T09:16:30.894187+0000","last_fullsized":"2023-10-18T09:16:30.894187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894504+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:16:30.894504+0000","last_peered":"2023-10-18T09:16:30.894504+0000","last_clean":"2023-10-18T09:16:30.894504+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:16:30.894504+0000","last_undegraded":"2023-10-18T09:16:30.894504+0000","last_fullsized":"2023-10-18T09:16:30.894504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10164,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":84802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":468315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.624}]},{"osd":1,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.372},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.502},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.39000000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.39900000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.51000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.65300000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":472341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.58099999999999996,"15min":0.55100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.78200000000000003,"15min":0.76500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.63}]},{"osd":3,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.84999999999999998,"15min":0.82499999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.56100000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.5190000000000001,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.107,"5min":0.98199999999999998,"15min":0.96099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.86399999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.99199999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.99399999999999999,"15min":0.96599999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.4130000000000003,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.90700000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75700000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82199999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92900000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92400000000000004,"5min":1.022,"15min":1.022},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.498,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.254,"15min":1.254},"last":0.60999999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71999999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1058364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.88600000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.55300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.51700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97399999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.871}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98799999999999999,"5min":0.998,"15min":0.998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45800000000000002,"15min":0.434},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0169999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0469999999999999,"5min":1.145,"15min":1.145},"last":0.499}]},{"osd":2,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91500000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97399999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.298,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.876},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.88100000000000001,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98499999999999999,"5min":11.112,"15min":11.112},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.66400000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.60599999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.115,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.80100000000000005}]},{"osd":8,"last update":"Wed Oct 18 09:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.84599999999999997,"15min":0.85899999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.077,"5min":1.51,"15min":1.51},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":9,"last update":"Wed Oct 18 09:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.747},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.498,"5min":0.379,"15min":0.379},"max":{"1min":0.85099999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed Oct 18 09:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39300000000000002},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42299999999999999,"15min":0.40899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.48999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.621,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":5.258,"5min":5.258,"15min":5.258},"last":0.85999999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.069,"15min":1.069},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.083,"15min":1.083},"last":0.627}]},{"osd":7,"last update":"Wed Oct 18 09:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.876}]},{"osd":8,"last update":"Wed Oct 18 09:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.55400000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.90600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.92000000000000004}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":468601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.56200000000000006,"15min":0.51700000000000002},"min":{"1min":0.497,"5min":0.314,"15min":0.314},"max":{"1min":0.83399999999999996,"5min":1.042,"15min":1.042},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.38200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.47399999999999998,"15min":0.46000000000000002},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.105},"min":{"1min":0.41399999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3240000000000001,"5min":11.539,"15min":11.539},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.95499999999999996,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.37,"15min":0.37},"max":{"1min":1.071,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.45200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.71199999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0389999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.51000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.748},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.98699999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92700000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.92600000000000005,"15min":0.93500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.0209999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.042},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.96999999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.0600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":512973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.40200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.60799999999999998},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.75,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.70599999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.45500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83399999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.747}]},{"osd":3,"last update":"Wed Oct 18 09:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.503},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.433,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Oct 18 09:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.628},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47399999999999998,"15min":0.44},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.101,"15min":1.101},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69199999999999995,"15min":0.69999999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82299999999999995}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42399999999999999,"15min":0.38900000000000001},"min":{"1min":0.35499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.41699999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61199999999999999,"15min":0.57299999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64900000000000002,"15min":0.63},"min":{"1min":0.52200000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.86199999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65000000000000002,"15min":0.63400000000000001},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.63400000000000001,"15min":0.59199999999999997},"min":{"1min":0.59199999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.628,"15min":0.59499999999999997},"min":{"1min":0.56299999999999994,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.622},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.501}]},{"osd":6,"last update":"Wed Oct 18 09:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78100000000000003,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.748}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":468213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.33400000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.38400000000000001},"min":{"1min":0.314,"5min":0.219,"15min":0.219},"max":{"1min":0.747,"5min":0.753,"15min":0.753},"last":0.436}]},{"osd":2,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73899999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.47799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77600000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.48299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0009999999999999,"5min":1.077,"15min":1.077},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84099999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.41299999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72799999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65700000000000003,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92400000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.46400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69499999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.752},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95399999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.626},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.89900000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1060119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.499}]},{"osd":1,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.38800000000000001},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.628},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.54200000000000004},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80100000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.79900000000000004,"5min":0.873,"15min":0.873},"last":0.82699999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80900000000000005,"5min":0.88,"15min":0.88},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Oct 18 09:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.752},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872}]},{"osd":8,"last update":"Wed Oct 18 09:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71199999999999997}]},{"osd":9,"last update":"Wed Oct 18 09:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.93300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":22975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:17:01.533 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:17:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 -- 172.21.15.61:0/544468274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 msgr2=0x7f8184101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 --2- 172.21.15.61:0/544468274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184101d90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f81780099c0 tx=0x7f8178009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 -- 172.21.15.61:0/544468274 shutdown_connections 2023-10-18T09:17:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 --2- 172.21.15.61:0/544468274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 0x7f8184108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 --2- 172.21.15.61:0/544468274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 0x7f8184100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 --2- 172.21.15.61:0/544468274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184101d90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 -- 172.21.15.61:0/544468274 >> 172.21.15.61:0/544468274 conn(0x7f81840fbc90 msgr2=0x7f81840fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:01.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 -- 172.21.15.61:0/544468274 shutdown_connections 2023-10-18T09:17:01.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.766+0000 7f8188bff700 1 -- 172.21.15.61:0/544468274 wait complete. 2023-10-18T09:17:01.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.767+0000 7f8188bff700 1 Processor -- start 2023-10-18T09:17:01.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.767+0000 7f8188bff700 1 -- start start 2023-10-18T09:17:01.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 0x7f8184198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 0x7f818419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81841a0430 con 0x7f8184101930 2023-10-18T09:17:01.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81841a0570 con 0x7f8184100740 2023-10-18T09:17:01.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8188bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81841a0700 con 0x7f81841022d0 2023-10-18T09:17:01.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41080/0 (socket says 172.21.15.61:41080) 2023-10-18T09:17:01.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 -- 172.21.15.61:0/1718622004 learned_addr learned my addr 172.21.15.61:0/1718622004 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:01.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f818259c700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 0x7f8184198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8182d9d700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 0x7f818419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 -- 172.21.15.61:0/1718622004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 msgr2=0x7f818419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 0x7f818419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 -- 172.21.15.61:0/1718622004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 msgr2=0x7f8184198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:01.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 0x7f8184198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:01.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.768+0000 7f8181d9b700 1 -- 172.21.15.61:0/1718622004 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81780095f0 con 0x7f8184101930 2023-10-18T09:17:01.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f8181d9b700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8178003730 tx=0x7f817800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:01.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817801d070 con 0x7f8184101930 2023-10-18T09:17:01.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f817800fe70 con 0x7f8184101930 2023-10-18T09:17:01.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178004bd0 con 0x7f8184101930 2023-10-18T09:17:01.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81841a0980 con 0x7f8184101930 2023-10-18T09:17:01.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.769+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81841a0d50 con 0x7f8184101930 2023-10-18T09:17:01.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.770+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184067530 con 0x7f8184101930 2023-10-18T09:17:01.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.772+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8178004d30 con 0x7f8184101930 2023-10-18T09:17:01.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.773+0000 7f81737fe700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 0x7f816c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:01.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.773+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f81780cdfd0 con 0x7f8184101930 2023-10-18T09:17:01.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.774+0000 7f818259c700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 0x7f816c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:01.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.774+0000 7f818259c700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 0x7f816c0a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8184101790 tx=0x7f8174009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:01.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.778+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8178097310 con 0x7f8184101930 2023-10-18T09:17:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:01 smithi170 ceph-mon[135150]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:01 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2016492869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:01 smithi170 ceph-mon[135150]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2023-10-18T09:17:01.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:01 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3051101303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:01.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-mon[134818]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:01.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2016492869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:01.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-mon[134818]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2023-10-18T09:17:01.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:01 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3051101303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:01.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.984+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8184108820 con 0x7f816c0a1510 2023-10-18T09:17:02.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:01.997+0000 7f81737fe700 1 -- 172.21.15.61:0/1718622004 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422552 (secure 0 0 0) 0x7f8184108820 con 0x7f816c0a1510 2023-10-18T09:17:02.005 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:02.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:01 smithi061 ceph-mon[98573]: from='client.24973 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:17:02.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:01 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2016492869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:02.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:01 smithi061 ceph-mon[98573]: pgmap v308: 129 pgs: 129 active+clean; 586 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2023-10-18T09:17:02.008 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:01 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3051101303' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:02.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 msgr2=0x7f816c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:02.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 0x7f816c0a39d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8184101790 tx=0x7f8174009260 comp rx=0 tx=0).stop 2023-10-18T09:17:02.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 msgr2=0x7f8184198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:02.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8178003730 tx=0x7f817800fa00 comp rx=0 tx=0).stop 2023-10-18T09:17:02.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 shutdown_connections 2023-10-18T09:17:02.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f816c0a1510 0x7f816c0a39d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:02.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81841022d0 0x7f818419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:02.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8184100740 0x7f8184198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:02.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 --2- 172.21.15.61:0/1718622004 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8184101930 0x7f8184198e40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:02.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 >> 172.21.15.61:0/1718622004 conn(0x7f81840fbc90 msgr2=0x7f8184107100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:02.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 shutdown_connections 2023-10-18T09:17:02.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:02.003+0000 7f8188bff700 1 -- 172.21.15.61:0/1718622004 wait complete. 2023-10-18T09:17:02.012 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:17:02.025 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-18T09:17:00.839915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600532,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":147,"num_read_kb":126,"num_write":344,"num_write_kb":2996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337540,"kb_used_data":13344,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604284,"statfs":{"total":1151940427776,"available":1151594786816,"internally_reserved":0,"allocated":13664256,"data_stored":7462521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5665,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":116,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003790"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896939+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:16:30.896939+0000","last_peered":"2023-10-18T09:16:30.896939+0000","last_clean":"2023-10-18T09:16:30.896939+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:16:30.896939+0000","last_undegraded":"2023-10-18T09:16:30.896939+0000","last_fullsized":"2023-10-18T09:16:30.896939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899150+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:16:30.899150+0000","last_peered":"2023-10-18T09:16:30.899150+0000","last_clean":"2023-10-18T09:16:30.899150+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:16:30.899150+0000","last_undegraded":"2023-10-18T09:16:30.899150+0000","last_fullsized":"2023-10-18T09:16:30.899150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895178+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:16:30.895178+0000","last_peered":"2023-10-18T09:16:30.895178+0000","last_clean":"2023-10-18T09:16:30.895178+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:16:30.895178+0000","last_undegraded":"2023-10-18T09:16:30.895178+0000","last_fullsized":"2023-10-18T09:16:30.895178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894474+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:16:30.894474+0000","last_peered":"2023-10-18T09:16:30.894474+0000","last_clean":"2023-10-18T09:16:30.894474+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:16:30.894474+0000","last_undegraded":"2023-10-18T09:16:30.894474+0000","last_fullsized":"2023-10-18T09:16:30.894474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.890834+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:16:30.890834+0000","last_peered":"2023-10-18T09:16:30.890834+0000","last_clean":"2023-10-18T09:16:30.890834+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:16:30.890834+0000","last_undegraded":"2023-10-18T09:16:30.890834+0000","last_fullsized":"2023-10-18T09:16:30.890834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913481+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:16:30.913481+0000","last_peered":"2023-10-18T09:16:30.913481+0000","last_clean":"2023-10-18T09:16:30.913481+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:16:30.913481+0000","last_undegraded":"2023-10-18T09:16:30.913481+0000","last_fullsized":"2023-10-18T09:16:30.913481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881754+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:16:30.881754+0000","last_peered":"2023-10-18T09:16:30.881754+0000","last_clean":"2023-10-18T09:16:30.881754+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:16:30.881754+0000","last_undegraded":"2023-10-18T09:16:30.881754+0000","last_fullsized":"2023-10-18T09:16:30.881754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.885710+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:16:30.885710+0000","last_peered":"2023-10-18T09:16:30.885710+0000","last_clean":"2023-10-18T09:16:30.885710+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:16:30.885710+0000","last_undegraded":"2023-10-18T09:16:30.885710+0000","last_fullsized":"2023-10-18T09:16:30.885710+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892514+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:16:30.892514+0000","last_peered":"2023-10-18T09:16:30.892514+0000","last_clean":"2023-10-18T09:16:30.892514+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:16:30.892514+0000","last_undegraded":"2023-10-18T09:16:30.892514+0000","last_fullsized":"2023-10-18T09:16:30.892514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913945+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:16:30.913945+0000","last_peered":"2023-10-18T09:16:30.913945+0000","last_clean":"2023-10-18T09:16:30.913945+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:16:30.913945+0000","last_undegraded":"2023-10-18T09:16:30.913945+0000","last_fullsized":"2023-10-18T09:16:30.913945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882374+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:16:30.882374+0000","last_peered":"2023-10-18T09:16:30.882374+0000","last_clean":"2023-10-18T09:16:30.882374+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:16:30.882374+0000","last_undegraded":"2023-10-18T09:16:30.882374+0000","last_fullsized":"2023-10-18T09:16:30.882374+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894398+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:16:30.894398+0000","last_peered":"2023-10-18T09:16:30.894398+0000","last_clean":"2023-10-18T09:16:30.894398+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:16:30.894398+0000","last_undegraded":"2023-10-18T09:16:30.894398+0000","last_fullsized":"2023-10-18T09:16:30.894398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894242+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:16:30.894242+0000","last_peered":"2023-10-18T09:16:30.894242+0000","last_clean":"2023-10-18T09:16:30.894242+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:16:30.894242+0000","last_undegraded":"2023-10-18T09:16:30.894242+0000","last_fullsized":"2023-10-18T09:16:30.894242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894421+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:16:30.894421+0000","last_peered":"2023-10-18T09:16:30.894421+0000","last_clean":"2023-10-18T09:16:30.894421+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:16:30.894421+0000","last_undegraded":"2023-10-18T09:16:30.894421+0000","last_fullsized":"2023-10-18T09:16:30.894421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894348+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:16:30.894348+0000","last_peered":"2023-10-18T09:16:30.894348+0000","last_clean":"2023-10-18T09:16:30.894348+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:16:30.894348+0000","last_undegraded":"2023-10-18T09:16:30.894348+0000","last_fullsized":"2023-10-18T09:16:30.894348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891879+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:16:30.891879+0000","last_peered":"2023-10-18T09:16:30.891879+0000","last_clean":"2023-10-18T09:16:30.891879+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:16:30.891879+0000","last_undegraded":"2023-10-18T09:16:30.891879+0000","last_fullsized":"2023-10-18T09:16:30.891879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896852+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:16:30.896852+0000","last_peered":"2023-10-18T09:16:30.896852+0000","last_clean":"2023-10-18T09:16:30.896852+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:16:30.896852+0000","last_undegraded":"2023-10-18T09:16:30.896852+0000","last_fullsized":"2023-10-18T09:16:30.896852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.904116+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:16:30.904116+0000","last_peered":"2023-10-18T09:16:30.904116+0000","last_clean":"2023-10-18T09:16:30.904116+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:16:30.904116+0000","last_undegraded":"2023-10-18T09:16:30.904116+0000","last_fullsized":"2023-10-18T09:16:30.904116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882826+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:16:30.882826+0000","last_peered":"2023-10-18T09:16:30.882826+0000","last_clean":"2023-10-18T09:16:30.882826+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:16:30.882826+0000","last_undegraded":"2023-10-18T09:16:30.882826+0000","last_fullsized":"2023-10-18T09:16:30.882826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499565+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:16:31.499565+0000","last_peered":"2023-10-18T09:16:31.499565+0000","last_clean":"2023-10-18T09:16:31.499565+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:16:31.499565+0000","last_undegraded":"2023-10-18T09:16:31.499565+0000","last_fullsized":"2023-10-18T09:16:31.499565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535374+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:16:58.535374+0000","last_peered":"2023-10-18T09:16:58.535374+0000","last_clean":"2023-10-18T09:16:58.535374+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:16:58.535374+0000","last_undegraded":"2023-10-18T09:16:58.535374+0000","last_fullsized":"2023-10-18T09:16:58.535374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886035+0000","last_change":"2023-10-18T09:16:26.510858+0000","last_active":"2023-10-18T09:16:30.886035+0000","last_peered":"2023-10-18T09:16:30.886035+0000","last_clean":"2023-10-18T09:16:30.886035+0000","last_became_active":"2023-10-18T09:16:26.509795+0000","last_became_peered":"2023-10-18T09:16:26.509795+0000","last_unstale":"2023-10-18T09:16:30.886035+0000","last_undegraded":"2023-10-18T09:16:30.886035+0000","last_fullsized":"2023-10-18T09:16:30.886035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:01.458412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978286+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:16:31.978286+0000","last_peered":"2023-10-18T09:16:31.978286+0000","last_clean":"2023-10-18T09:16:31.978286+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:16:31.978286+0000","last_undegraded":"2023-10-18T09:16:31.978286+0000","last_fullsized":"2023-10-18T09:16:31.978286+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.994315+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:16:31.994315+0000","last_peered":"2023-10-18T09:16:31.994315+0000","last_clean":"2023-10-18T09:16:31.994315+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:16:31.994315+0000","last_undegraded":"2023-10-18T09:16:31.994315+0000","last_fullsized":"2023-10-18T09:16:31.994315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899318+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:16:30.899318+0000","last_peered":"2023-10-18T09:16:30.899318+0000","last_clean":"2023-10-18T09:16:30.899318+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:16:30.899318+0000","last_undegraded":"2023-10-18T09:16:30.899318+0000","last_fullsized":"2023-10-18T09:16:30.899318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979121+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:16:31.979121+0000","last_peered":"2023-10-18T09:16:31.979121+0000","last_clean":"2023-10-18T09:16:31.979121+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:16:31.979121+0000","last_undegraded":"2023-10-18T09:16:31.979121+0000","last_fullsized":"2023-10-18T09:16:31.979121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'11","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535432+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:16:58.535432+0000","last_peered":"2023-10-18T09:16:58.535432+0000","last_clean":"2023-10-18T09:16:58.535432+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:16:58.535432+0000","last_undegraded":"2023-10-18T09:16:58.535432+0000","last_fullsized":"2023-10-18T09:16:58.535432+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892543+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:16:30.892543+0000","last_peered":"2023-10-18T09:16:30.892543+0000","last_clean":"2023-10-18T09:16:30.892543+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:16:30.892543+0000","last_undegraded":"2023-10-18T09:16:30.892543+0000","last_fullsized":"2023-10-18T09:16:30.892543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978412+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:16:31.978412+0000","last_peered":"2023-10-18T09:16:31.978412+0000","last_clean":"2023-10-18T09:16:31.978412+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:16:31.978412+0000","last_undegraded":"2023-10-18T09:16:31.978412+0000","last_fullsized":"2023-10-18T09:16:31.978412+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535988+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:16:58.535988+0000","last_peered":"2023-10-18T09:16:58.535988+0000","last_clean":"2023-10-18T09:16:58.535988+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:16:58.535988+0000","last_undegraded":"2023-10-18T09:16:58.535988+0000","last_fullsized":"2023-10-18T09:16:58.535988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894243+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:16:30.894243+0000","last_peered":"2023-10-18T09:16:30.894243+0000","last_clean":"2023-10-18T09:16:30.894243+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:16:30.894243+0000","last_undegraded":"2023-10-18T09:16:30.894243+0000","last_fullsized":"2023-10-18T09:16:30.894243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499907+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:16:31.499907+0000","last_peered":"2023-10-18T09:16:31.499907+0000","last_clean":"2023-10-18T09:16:31.499907+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:16:31.499907+0000","last_undegraded":"2023-10-18T09:16:31.499907+0000","last_fullsized":"2023-10-18T09:16:31.499907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.520136+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:16:58.520136+0000","last_peered":"2023-10-18T09:16:58.520136+0000","last_clean":"2023-10-18T09:16:58.520136+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:16:58.520136+0000","last_undegraded":"2023-10-18T09:16:58.520136+0000","last_fullsized":"2023-10-18T09:16:58.520136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891722+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:16:30.891722+0000","last_peered":"2023-10-18T09:16:30.891722+0000","last_clean":"2023-10-18T09:16:30.891722+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:16:30.891722+0000","last_undegraded":"2023-10-18T09:16:30.891722+0000","last_fullsized":"2023-10-18T09:16:30.891722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.500073+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:16:31.500073+0000","last_peered":"2023-10-18T09:16:31.500073+0000","last_clean":"2023-10-18T09:16:31.500073+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:16:31.500073+0000","last_undegraded":"2023-10-18T09:16:31.500073+0000","last_fullsized":"2023-10-18T09:16:31.500073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"83'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913415+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:16:30.913415+0000","last_peered":"2023-10-18T09:16:30.913415+0000","last_clean":"2023-10-18T09:16:30.913415+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:16:30.913415+0000","last_undegraded":"2023-10-18T09:16:30.913415+0000","last_fullsized":"2023-10-18T09:16:30.913415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891407+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:16:30.891407+0000","last_peered":"2023-10-18T09:16:30.891407+0000","last_clean":"2023-10-18T09:16:30.891407+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:16:30.891407+0000","last_undegraded":"2023-10-18T09:16:30.891407+0000","last_fullsized":"2023-10-18T09:16:30.891407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499032+0000","last_change":"2023-10-18T09:16:30.910525+0000","last_active":"2023-10-18T09:16:31.499032+0000","last_peered":"2023-10-18T09:16:31.499032+0000","last_clean":"2023-10-18T09:16:31.499032+0000","last_became_active":"2023-10-18T09:16:30.909960+0000","last_became_peered":"2023-10-18T09:16:30.909960+0000","last_unstale":"2023-10-18T09:16:31.499032+0000","last_undegraded":"2023-10-18T09:16:31.499032+0000","last_fullsized":"2023-10-18T09:16:31.499032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:16:21.919981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894017+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:16:30.894017+0000","last_peered":"2023-10-18T09:16:30.894017+0000","last_clean":"2023-10-18T09:16:30.894017+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:16:30.894017+0000","last_undegraded":"2023-10-18T09:16:30.894017+0000","last_fullsized":"2023-10-18T09:16:30.894017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.885819+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:16:30.885819+0000","last_peered":"2023-10-18T09:16:30.885819+0000","last_clean":"2023-10-18T09:16:30.885819+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:16:30.885819+0000","last_undegraded":"2023-10-18T09:16:30.885819+0000","last_fullsized":"2023-10-18T09:16:30.885819+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979112+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:16:31.979112+0000","last_peered":"2023-10-18T09:16:31.979112+0000","last_clean":"2023-10-18T09:16:31.979112+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:16:31.979112+0000","last_undegraded":"2023-10-18T09:16:31.979112+0000","last_fullsized":"2023-10-18T09:16:31.979112+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.524545+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:16:57.524545+0000","last_peered":"2023-10-18T09:16:57.524545+0000","last_clean":"2023-10-18T09:16:57.524545+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:16:57.524545+0000","last_undegraded":"2023-10-18T09:16:57.524545+0000","last_fullsized":"2023-10-18T09:16:57.524545+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.901444+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:16:30.901444+0000","last_peered":"2023-10-18T09:16:30.901444+0000","last_clean":"2023-10-18T09:16:30.901444+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:16:30.901444+0000","last_undegraded":"2023-10-18T09:16:30.901444+0000","last_fullsized":"2023-10-18T09:16:30.901444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.993802+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:16:31.993802+0000","last_peered":"2023-10-18T09:16:31.993802+0000","last_clean":"2023-10-18T09:16:31.993802+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:16:31.993802+0000","last_undegraded":"2023-10-18T09:16:31.993802+0000","last_fullsized":"2023-10-18T09:16:31.993802+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.914017+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:16:30.914017+0000","last_peered":"2023-10-18T09:16:30.914017+0000","last_clean":"2023-10-18T09:16:30.914017+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:16:30.914017+0000","last_undegraded":"2023-10-18T09:16:30.914017+0000","last_fullsized":"2023-10-18T09:16:30.914017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893229+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:16:30.893229+0000","last_peered":"2023-10-18T09:16:30.893229+0000","last_clean":"2023-10-18T09:16:30.893229+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:16:30.893229+0000","last_undegraded":"2023-10-18T09:16:30.893229+0000","last_fullsized":"2023-10-18T09:16:30.893229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499129+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:16:31.499129+0000","last_peered":"2023-10-18T09:16:31.499129+0000","last_clean":"2023-10-18T09:16:31.499129+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:16:31.499129+0000","last_undegraded":"2023-10-18T09:16:31.499129+0000","last_fullsized":"2023-10-18T09:16:31.499129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.921053+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:16:55.921053+0000","last_peered":"2023-10-18T09:16:55.921053+0000","last_clean":"2023-10-18T09:16:55.921053+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:16:55.921053+0000","last_undegraded":"2023-10-18T09:16:55.921053+0000","last_fullsized":"2023-10-18T09:16:55.921053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899414+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:16:30.899414+0000","last_peered":"2023-10-18T09:16:30.899414+0000","last_clean":"2023-10-18T09:16:30.899414+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:16:30.899414+0000","last_undegraded":"2023-10-18T09:16:30.899414+0000","last_fullsized":"2023-10-18T09:16:30.899414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979419+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:16:31.979419+0000","last_peered":"2023-10-18T09:16:31.979419+0000","last_clean":"2023-10-18T09:16:31.979419+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:16:31.979419+0000","last_undegraded":"2023-10-18T09:16:31.979419+0000","last_fullsized":"2023-10-18T09:16:31.979419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:56.513262+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:16:56.513262+0000","last_peered":"2023-10-18T09:16:56.513262+0000","last_clean":"2023-10-18T09:16:56.513262+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:16:56.513262+0000","last_undegraded":"2023-10-18T09:16:56.513262+0000","last_fullsized":"2023-10-18T09:16:56.513262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892709+0000","last_change":"2023-10-18T09:16:26.520057+0000","last_active":"2023-10-18T09:16:30.892709+0000","last_peered":"2023-10-18T09:16:30.892709+0000","last_clean":"2023-10-18T09:16:30.892709+0000","last_became_active":"2023-10-18T09:16:26.518997+0000","last_became_peered":"2023-10-18T09:16:26.518997+0000","last_unstale":"2023-10-18T09:16:30.892709+0000","last_undegraded":"2023-10-18T09:16:30.892709+0000","last_fullsized":"2023-10-18T09:16:30.892709+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:38:54.779476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499191+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:16:31.499191+0000","last_peered":"2023-10-18T09:16:31.499191+0000","last_clean":"2023-10-18T09:16:31.499191+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:16:31.499191+0000","last_undegraded":"2023-10-18T09:16:31.499191+0000","last_fullsized":"2023-10-18T09:16:31.499191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'10","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:46.614761+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:16:46.614761+0000","last_peered":"2023-10-18T09:16:46.614761+0000","last_clean":"2023-10-18T09:16:46.614761+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:16:46.614761+0000","last_undegraded":"2023-10-18T09:16:46.614761+0000","last_fullsized":"2023-10-18T09:16:46.614761+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891040+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:16:30.891040+0000","last_peered":"2023-10-18T09:16:30.891040+0000","last_clean":"2023-10-18T09:16:30.891040+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:16:30.891040+0000","last_undegraded":"2023-10-18T09:16:30.891040+0000","last_fullsized":"2023-10-18T09:16:30.891040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979353+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:16:31.979353+0000","last_peered":"2023-10-18T09:16:31.979353+0000","last_clean":"2023-10-18T09:16:31.979353+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:16:31.979353+0000","last_undegraded":"2023-10-18T09:16:31.979353+0000","last_fullsized":"2023-10-18T09:16:31.979353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.535609+0000","last_change":"2023-10-18T09:16:25.864359+0000","last_active":"2023-10-18T09:16:58.535609+0000","last_peered":"2023-10-18T09:16:58.535609+0000","last_clean":"2023-10-18T09:16:58.535609+0000","last_became_active":"2023-10-18T09:16:25.864019+0000","last_became_peered":"2023-10-18T09:16:25.864019+0000","last_unstale":"2023-10-18T09:16:58.535609+0000","last_undegraded":"2023-10-18T09:16:58.535609+0000","last_fullsized":"2023-10-18T09:16:58.535609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:30:36.407133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.897598+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:16:30.897598+0000","last_peered":"2023-10-18T09:16:30.897598+0000","last_clean":"2023-10-18T09:16:30.897598+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:16:30.897598+0000","last_undegraded":"2023-10-18T09:16:30.897598+0000","last_fullsized":"2023-10-18T09:16:30.897598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.498769+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:16:31.498769+0000","last_peered":"2023-10-18T09:16:31.498769+0000","last_clean":"2023-10-18T09:16:31.498769+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:16:31.498769+0000","last_undegraded":"2023-10-18T09:16:31.498769+0000","last_fullsized":"2023-10-18T09:16:31.498769+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526285+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:16:57.526285+0000","last_peered":"2023-10-18T09:16:57.526285+0000","last_clean":"2023-10-18T09:16:57.526285+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:16:57.526285+0000","last_undegraded":"2023-10-18T09:16:57.526285+0000","last_fullsized":"2023-10-18T09:16:57.526285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882100+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:16:30.882100+0000","last_peered":"2023-10-18T09:16:30.882100+0000","last_clean":"2023-10-18T09:16:30.882100+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:16:30.882100+0000","last_undegraded":"2023-10-18T09:16:30.882100+0000","last_fullsized":"2023-10-18T09:16:30.882100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499442+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:16:31.499442+0000","last_peered":"2023-10-18T09:16:31.499442+0000","last_clean":"2023-10-18T09:16:31.499442+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:16:31.499442+0000","last_undegraded":"2023-10-18T09:16:31.499442+0000","last_fullsized":"2023-10-18T09:16:31.499442+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.524500+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:16:57.524500+0000","last_peered":"2023-10-18T09:16:57.524500+0000","last_clean":"2023-10-18T09:16:57.524500+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:16:57.524500+0000","last_undegraded":"2023-10-18T09:16:57.524500+0000","last_fullsized":"2023-10-18T09:16:57.524500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881818+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:16:30.881818+0000","last_peered":"2023-10-18T09:16:30.881818+0000","last_clean":"2023-10-18T09:16:30.881818+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:16:30.881818+0000","last_undegraded":"2023-10-18T09:16:30.881818+0000","last_fullsized":"2023-10-18T09:16:30.881818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978777+0000","last_change":"2023-10-18T09:16:30.910478+0000","last_active":"2023-10-18T09:16:31.978777+0000","last_peered":"2023-10-18T09:16:31.978777+0000","last_clean":"2023-10-18T09:16:31.978777+0000","last_became_active":"2023-10-18T09:16:30.910036+0000","last_became_peered":"2023-10-18T09:16:30.910036+0000","last_unstale":"2023-10-18T09:16:31.978777+0000","last_undegraded":"2023-10-18T09:16:31.978777+0000","last_fullsized":"2023-10-18T09:16:31.978777+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:12:55.671867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.525217+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:16:57.525217+0000","last_peered":"2023-10-18T09:16:57.525217+0000","last_clean":"2023-10-18T09:16:57.525217+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:16:57.525217+0000","last_undegraded":"2023-10-18T09:16:57.525217+0000","last_fullsized":"2023-10-18T09:16:57.525217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893351+0000","last_change":"2023-10-18T09:16:26.516754+0000","last_active":"2023-10-18T09:16:30.893351+0000","last_peered":"2023-10-18T09:16:30.893351+0000","last_clean":"2023-10-18T09:16:30.893351+0000","last_became_active":"2023-10-18T09:16:26.516350+0000","last_became_peered":"2023-10-18T09:16:26.516350+0000","last_unstale":"2023-10-18T09:16:30.893351+0000","last_undegraded":"2023-10-18T09:16:30.893351+0000","last_fullsized":"2023-10-18T09:16:30.893351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:37.396217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979040+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:16:31.979040+0000","last_peered":"2023-10-18T09:16:31.979040+0000","last_clean":"2023-10-18T09:16:31.979040+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:16:31.979040+0000","last_undegraded":"2023-10-18T09:16:31.979040+0000","last_fullsized":"2023-10-18T09:16:31.979040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978360+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:16:31.978360+0000","last_peered":"2023-10-18T09:16:31.978360+0000","last_clean":"2023-10-18T09:16:31.978360+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:16:31.978360+0000","last_undegraded":"2023-10-18T09:16:31.978360+0000","last_fullsized":"2023-10-18T09:16:31.978360+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895086+0000","last_change":"2023-10-18T09:16:26.517973+0000","last_active":"2023-10-18T09:16:30.895086+0000","last_peered":"2023-10-18T09:16:30.895086+0000","last_clean":"2023-10-18T09:16:30.895086+0000","last_became_active":"2023-10-18T09:16:26.517708+0000","last_became_peered":"2023-10-18T09:16:26.517708+0000","last_unstale":"2023-10-18T09:16:30.895086+0000","last_undegraded":"2023-10-18T09:16:30.895086+0000","last_fullsized":"2023-10-18T09:16:30.895086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:07.086081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978393+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:16:31.978393+0000","last_peered":"2023-10-18T09:16:31.978393+0000","last_clean":"2023-10-18T09:16:31.978393+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:16:31.978393+0000","last_undegraded":"2023-10-18T09:16:31.978393+0000","last_fullsized":"2023-10-18T09:16:31.978393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499981+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:16:31.499981+0000","last_peered":"2023-10-18T09:16:31.499981+0000","last_clean":"2023-10-18T09:16:31.499981+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:16:31.499981+0000","last_undegraded":"2023-10-18T09:16:31.499981+0000","last_fullsized":"2023-10-18T09:16:31.499981+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913739+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:16:30.913739+0000","last_peered":"2023-10-18T09:16:30.913739+0000","last_clean":"2023-10-18T09:16:30.913739+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:16:30.913739+0000","last_undegraded":"2023-10-18T09:16:30.913739+0000","last_fullsized":"2023-10-18T09:16:30.913739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978082+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:16:31.978082+0000","last_peered":"2023-10-18T09:16:31.978082+0000","last_clean":"2023-10-18T09:16:31.978082+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:16:31.978082+0000","last_undegraded":"2023-10-18T09:16:31.978082+0000","last_fullsized":"2023-10-18T09:16:31.978082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978370+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:16:31.978370+0000","last_peered":"2023-10-18T09:16:31.978370+0000","last_clean":"2023-10-18T09:16:31.978370+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:16:31.978370+0000","last_undegraded":"2023-10-18T09:16:31.978370+0000","last_fullsized":"2023-10-18T09:16:31.978370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.913574+0000","last_change":"2023-10-18T09:16:26.518651+0000","last_active":"2023-10-18T09:16:30.913574+0000","last_peered":"2023-10-18T09:16:30.913574+0000","last_clean":"2023-10-18T09:16:30.913574+0000","last_became_active":"2023-10-18T09:16:26.518365+0000","last_became_peered":"2023-10-18T09:16:26.518365+0000","last_unstale":"2023-10-18T09:16:30.913574+0000","last_undegraded":"2023-10-18T09:16:30.913574+0000","last_fullsized":"2023-10-18T09:16:30.913574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:12:02.868372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979109+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:16:31.979109+0000","last_peered":"2023-10-18T09:16:31.979109+0000","last_clean":"2023-10-18T09:16:31.979109+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:16:31.979109+0000","last_undegraded":"2023-10-18T09:16:31.979109+0000","last_fullsized":"2023-10-18T09:16:31.979109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891585+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:16:30.891585+0000","last_peered":"2023-10-18T09:16:30.891585+0000","last_clean":"2023-10-18T09:16:30.891585+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:16:30.891585+0000","last_undegraded":"2023-10-18T09:16:30.891585+0000","last_fullsized":"2023-10-18T09:16:30.891585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'1","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.977610+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:16:31.977610+0000","last_peered":"2023-10-18T09:16:31.977610+0000","last_clean":"2023-10-18T09:16:31.977610+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:16:31.977610+0000","last_undegraded":"2023-10-18T09:16:31.977610+0000","last_fullsized":"2023-10-18T09:16:31.977610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979926+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:16:31.979926+0000","last_peered":"2023-10-18T09:16:31.979926+0000","last_clean":"2023-10-18T09:16:31.979926+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:16:31.979926+0000","last_undegraded":"2023-10-18T09:16:31.979926+0000","last_fullsized":"2023-10-18T09:16:31.979926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886038+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:16:30.886038+0000","last_peered":"2023-10-18T09:16:30.886038+0000","last_clean":"2023-10-18T09:16:30.886038+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:16:30.886038+0000","last_undegraded":"2023-10-18T09:16:30.886038+0000","last_fullsized":"2023-10-18T09:16:30.886038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.527172+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:16:57.527172+0000","last_peered":"2023-10-18T09:16:57.527172+0000","last_clean":"2023-10-18T09:16:57.527172+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:16:57.527172+0000","last_undegraded":"2023-10-18T09:16:57.527172+0000","last_fullsized":"2023-10-18T09:16:57.527172+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":311,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881635+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:16:30.881635+0000","last_peered":"2023-10-18T09:16:30.881635+0000","last_clean":"2023-10-18T09:16:30.881635+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:16:30.881635+0000","last_undegraded":"2023-10-18T09:16:30.881635+0000","last_fullsized":"2023-10-18T09:16:30.881635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979156+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:16:31.979156+0000","last_peered":"2023-10-18T09:16:31.979156+0000","last_clean":"2023-10-18T09:16:31.979156+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:16:31.979156+0000","last_undegraded":"2023-10-18T09:16:31.979156+0000","last_fullsized":"2023-10-18T09:16:31.979156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.893884+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:16:30.893884+0000","last_peered":"2023-10-18T09:16:30.893884+0000","last_clean":"2023-10-18T09:16:30.893884+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:16:30.893884+0000","last_undegraded":"2023-10-18T09:16:30.893884+0000","last_fullsized":"2023-10-18T09:16:30.893884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:56.517486+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:16:56.517486+0000","last_peered":"2023-10-18T09:16:56.517486+0000","last_clean":"2023-10-18T09:16:56.517486+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:16:56.517486+0000","last_undegraded":"2023-10-18T09:16:56.517486+0000","last_fullsized":"2023-10-18T09:16:56.517486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.498729+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:16:31.498729+0000","last_peered":"2023-10-18T09:16:31.498729+0000","last_clean":"2023-10-18T09:16:31.498729+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:16:31.498729+0000","last_undegraded":"2023-10-18T09:16:31.498729+0000","last_fullsized":"2023-10-18T09:16:31.498729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.527439+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:16:57.527439+0000","last_peered":"2023-10-18T09:16:57.527439+0000","last_clean":"2023-10-18T09:16:57.527439+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:16:57.527439+0000","last_undegraded":"2023-10-18T09:16:57.527439+0000","last_fullsized":"2023-10-18T09:16:57.527439+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894211+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:16:30.894211+0000","last_peered":"2023-10-18T09:16:30.894211+0000","last_clean":"2023-10-18T09:16:30.894211+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:16:30.894211+0000","last_undegraded":"2023-10-18T09:16:30.894211+0000","last_fullsized":"2023-10-18T09:16:30.894211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.993743+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:16:31.993743+0000","last_peered":"2023-10-18T09:16:31.993743+0000","last_clean":"2023-10-18T09:16:31.993743+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:16:31.993743+0000","last_undegraded":"2023-10-18T09:16:31.993743+0000","last_fullsized":"2023-10-18T09:16:31.993743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.528421+0000","last_change":"2023-10-18T09:16:25.866900+0000","last_active":"2023-10-18T09:16:57.528421+0000","last_peered":"2023-10-18T09:16:57.528421+0000","last_clean":"2023-10-18T09:16:57.528421+0000","last_became_active":"2023-10-18T09:16:25.866545+0000","last_became_peered":"2023-10-18T09:16:25.866545+0000","last_unstale":"2023-10-18T09:16:57.528421+0000","last_undegraded":"2023-10-18T09:16:57.528421+0000","last_fullsized":"2023-10-18T09:16:57.528421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:38:01.437689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886952+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:16:30.886952+0000","last_peered":"2023-10-18T09:16:30.886952+0000","last_clean":"2023-10-18T09:16:30.886952+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:16:30.886952+0000","last_undegraded":"2023-10-18T09:16:30.886952+0000","last_fullsized":"2023-10-18T09:16:30.886952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499444+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:16:31.499444+0000","last_peered":"2023-10-18T09:16:31.499444+0000","last_clean":"2023-10-18T09:16:31.499444+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:16:31.499444+0000","last_undegraded":"2023-10-18T09:16:31.499444+0000","last_fullsized":"2023-10-18T09:16:31.499444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:58.536235+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:16:58.536235+0000","last_peered":"2023-10-18T09:16:58.536235+0000","last_clean":"2023-10-18T09:16:58.536235+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:16:58.536235+0000","last_undegraded":"2023-10-18T09:16:58.536235+0000","last_fullsized":"2023-10-18T09:16:58.536235+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.899243+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:16:30.899243+0000","last_peered":"2023-10-18T09:16:30.899243+0000","last_clean":"2023-10-18T09:16:30.899243+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:16:30.899243+0000","last_undegraded":"2023-10-18T09:16:30.899243+0000","last_fullsized":"2023-10-18T09:16:30.899243+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979193+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:16:31.979193+0000","last_peered":"2023-10-18T09:16:31.979193+0000","last_clean":"2023-10-18T09:16:31.979193+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:16:31.979193+0000","last_undegraded":"2023-10-18T09:16:31.979193+0000","last_fullsized":"2023-10-18T09:16:31.979193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.922535+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:16:55.922535+0000","last_peered":"2023-10-18T09:16:55.922535+0000","last_clean":"2023-10-18T09:16:55.922535+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:16:55.922535+0000","last_undegraded":"2023-10-18T09:16:55.922535+0000","last_fullsized":"2023-10-18T09:16:55.922535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.890715+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:16:30.890715+0000","last_peered":"2023-10-18T09:16:30.890715+0000","last_clean":"2023-10-18T09:16:30.890715+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:16:30.890715+0000","last_undegraded":"2023-10-18T09:16:30.890715+0000","last_fullsized":"2023-10-18T09:16:30.890715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.499562+0000","last_change":"2023-10-18T09:16:30.908660+0000","last_active":"2023-10-18T09:16:31.499562+0000","last_peered":"2023-10-18T09:16:31.499562+0000","last_clean":"2023-10-18T09:16:31.499562+0000","last_became_active":"2023-10-18T09:16:30.906449+0000","last_became_peered":"2023-10-18T09:16:30.906449+0000","last_unstale":"2023-10-18T09:16:31.499562+0000","last_undegraded":"2023-10-18T09:16:31.499562+0000","last_fullsized":"2023-10-18T09:16:31.499562+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:03:22.596758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.923681+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:16:55.923681+0000","last_peered":"2023-10-18T09:16:55.923681+0000","last_clean":"2023-10-18T09:16:55.923681+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:16:55.923681+0000","last_undegraded":"2023-10-18T09:16:55.923681+0000","last_fullsized":"2023-10-18T09:16:55.923681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894142+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:16:30.894142+0000","last_peered":"2023-10-18T09:16:30.894142+0000","last_clean":"2023-10-18T09:16:30.894142+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:16:30.894142+0000","last_undegraded":"2023-10-18T09:16:30.894142+0000","last_fullsized":"2023-10-18T09:16:30.894142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.979153+0000","last_change":"2023-10-18T09:16:30.909709+0000","last_active":"2023-10-18T09:16:31.979153+0000","last_peered":"2023-10-18T09:16:31.979153+0000","last_clean":"2023-10-18T09:16:31.979153+0000","last_became_active":"2023-10-18T09:16:30.909187+0000","last_became_peered":"2023-10-18T09:16:30.909187+0000","last_unstale":"2023-10-18T09:16:31.979153+0000","last_undegraded":"2023-10-18T09:16:31.979153+0000","last_fullsized":"2023-10-18T09:16:31.979153+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:14:01.975160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526205+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:16:57.526205+0000","last_peered":"2023-10-18T09:16:57.526205+0000","last_clean":"2023-10-18T09:16:57.526205+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:16:57.526205+0000","last_undegraded":"2023-10-18T09:16:57.526205+0000","last_fullsized":"2023-10-18T09:16:57.526205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886513+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:16:30.886513+0000","last_peered":"2023-10-18T09:16:30.886513+0000","last_clean":"2023-10-18T09:16:30.886513+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:16:30.886513+0000","last_undegraded":"2023-10-18T09:16:30.886513+0000","last_fullsized":"2023-10-18T09:16:30.886513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978879+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:16:31.978879+0000","last_peered":"2023-10-18T09:16:31.978879+0000","last_clean":"2023-10-18T09:16:31.978879+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:16:31.978879+0000","last_undegraded":"2023-10-18T09:16:31.978879+0000","last_fullsized":"2023-10-18T09:16:31.978879+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:55.924188+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:16:55.924188+0000","last_peered":"2023-10-18T09:16:55.924188+0000","last_clean":"2023-10-18T09:16:55.924188+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:16:55.924188+0000","last_undegraded":"2023-10-18T09:16:55.924188+0000","last_fullsized":"2023-10-18T09:16:55.924188+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881949+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:16:30.881949+0000","last_peered":"2023-10-18T09:16:30.881949+0000","last_clean":"2023-10-18T09:16:30.881949+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:16:30.881949+0000","last_undegraded":"2023-10-18T09:16:30.881949+0000","last_fullsized":"2023-10-18T09:16:30.881949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.680106+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:16:31.680106+0000","last_peered":"2023-10-18T09:16:31.680106+0000","last_clean":"2023-10-18T09:16:31.680106+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:16:31.680106+0000","last_undegraded":"2023-10-18T09:16:31.680106+0000","last_fullsized":"2023-10-18T09:16:31.680106+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:59.551894+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:16:59.551894+0000","last_peered":"2023-10-18T09:16:59.551894+0000","last_clean":"2023-10-18T09:16:59.551894+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:16:59.551894+0000","last_undegraded":"2023-10-18T09:16:59.551894+0000","last_fullsized":"2023-10-18T09:16:59.551894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882434+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:16:30.882434+0000","last_peered":"2023-10-18T09:16:30.882434+0000","last_clean":"2023-10-18T09:16:30.882434+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:16:30.882434+0000","last_undegraded":"2023-10-18T09:16:30.882434+0000","last_fullsized":"2023-10-18T09:16:30.882434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978869+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:16:31.978869+0000","last_peered":"2023-10-18T09:16:31.978869+0000","last_clean":"2023-10-18T09:16:31.978869+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:16:31.978869+0000","last_undegraded":"2023-10-18T09:16:31.978869+0000","last_fullsized":"2023-10-18T09:16:31.978869+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:57.526493+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:16:57.526493+0000","last_peered":"2023-10-18T09:16:57.526493+0000","last_clean":"2023-10-18T09:16:57.526493+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:16:57.526493+0000","last_undegraded":"2023-10-18T09:16:57.526493+0000","last_fullsized":"2023-10-18T09:16:57.526493+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882212+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:16:30.882212+0000","last_peered":"2023-10-18T09:16:30.882212+0000","last_clean":"2023-10-18T09:16:30.882212+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:16:30.882212+0000","last_undegraded":"2023-10-18T09:16:30.882212+0000","last_fullsized":"2023-10-18T09:16:30.882212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-18T09:16:31.978052+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:16:31.978052+0000","last_peered":"2023-10-18T09:16:31.978052+0000","last_clean":"2023-10-18T09:16:31.978052+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:16:31.978052+0000","last_undegraded":"2023-10-18T09:16:31.978052+0000","last_fullsized":"2023-10-18T09:16:31.978052+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896459+0000","last_change":"2023-10-18T09:16:25.867090+0000","last_active":"2023-10-18T09:16:30.896459+0000","last_peered":"2023-10-18T09:16:30.896459+0000","last_clean":"2023-10-18T09:16:30.896459+0000","last_became_active":"2023-10-18T09:16:25.864322+0000","last_became_peered":"2023-10-18T09:16:25.864322+0000","last_unstale":"2023-10-18T09:16:30.896459+0000","last_undegraded":"2023-10-18T09:16:30.896459+0000","last_fullsized":"2023-10-18T09:16:30.896459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:23:31.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882392+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:16:30.882392+0000","last_peered":"2023-10-18T09:16:30.882392+0000","last_clean":"2023-10-18T09:16:30.882392+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:16:30.882392+0000","last_undegraded":"2023-10-18T09:16:30.882392+0000","last_fullsized":"2023-10-18T09:16:30.882392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894259+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:16:30.894259+0000","last_peered":"2023-10-18T09:16:30.894259+0000","last_clean":"2023-10-18T09:16:30.894259+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:16:30.894259+0000","last_undegraded":"2023-10-18T09:16:30.894259+0000","last_fullsized":"2023-10-18T09:16:30.894259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.883101+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:16:30.883101+0000","last_peered":"2023-10-18T09:16:30.883101+0000","last_clean":"2023-10-18T09:16:30.883101+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:16:30.883101+0000","last_undegraded":"2023-10-18T09:16:30.883101+0000","last_fullsized":"2023-10-18T09:16:30.883101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.881853+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:16:30.881853+0000","last_peered":"2023-10-18T09:16:30.881853+0000","last_clean":"2023-10-18T09:16:30.881853+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:16:30.881853+0000","last_undegraded":"2023-10-18T09:16:30.881853+0000","last_fullsized":"2023-10-18T09:16:30.881853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.896602+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:16:30.896602+0000","last_peered":"2023-10-18T09:16:30.896602+0000","last_clean":"2023-10-18T09:16:30.896602+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:16:30.896602+0000","last_undegraded":"2023-10-18T09:16:30.896602+0000","last_fullsized":"2023-10-18T09:16:30.896602+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894042+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:16:30.894042+0000","last_peered":"2023-10-18T09:16:30.894042+0000","last_clean":"2023-10-18T09:16:30.894042+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:16:30.894042+0000","last_undegraded":"2023-10-18T09:16:30.894042+0000","last_fullsized":"2023-10-18T09:16:30.894042+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.895354+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:16:30.895354+0000","last_peered":"2023-10-18T09:16:30.895354+0000","last_clean":"2023-10-18T09:16:30.895354+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:16:30.895354+0000","last_undegraded":"2023-10-18T09:16:30.895354+0000","last_fullsized":"2023-10-18T09:16:30.895354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.882307+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:16:30.882307+0000","last_peered":"2023-10-18T09:16:30.882307+0000","last_clean":"2023-10-18T09:16:30.882307+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:16:30.882307+0000","last_undegraded":"2023-10-18T09:16:30.882307+0000","last_fullsized":"2023-10-18T09:16:30.882307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886859+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:16:30.886859+0000","last_peered":"2023-10-18T09:16:30.886859+0000","last_clean":"2023-10-18T09:16:30.886859+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:16:30.886859+0000","last_undegraded":"2023-10-18T09:16:30.886859+0000","last_fullsized":"2023-10-18T09:16:30.886859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.891500+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:16:30.891500+0000","last_peered":"2023-10-18T09:16:30.891500+0000","last_clean":"2023-10-18T09:16:30.891500+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:16:30.891500+0000","last_undegraded":"2023-10-18T09:16:30.891500+0000","last_fullsized":"2023-10-18T09:16:30.891500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.886167+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:16:30.886167+0000","last_peered":"2023-10-18T09:16:30.886167+0000","last_clean":"2023-10-18T09:16:30.886167+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:16:30.886167+0000","last_undegraded":"2023-10-18T09:16:30.886167+0000","last_fullsized":"2023-10-18T09:16:30.886167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.892991+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:16:30.892991+0000","last_peered":"2023-10-18T09:16:30.892991+0000","last_clean":"2023-10-18T09:16:30.892991+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:16:30.892991+0000","last_undegraded":"2023-10-18T09:16:30.892991+0000","last_fullsized":"2023-10-18T09:16:30.892991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894187+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:16:30.894187+0000","last_peered":"2023-10-18T09:16:30.894187+0000","last_clean":"2023-10-18T09:16:30.894187+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:16:30.894187+0000","last_undegraded":"2023-10-18T09:16:30.894187+0000","last_fullsized":"2023-10-18T09:16:30.894187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-18T09:16:30.894504+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:16:30.894504+0000","last_peered":"2023-10-18T09:16:30.894504+0000","last_clean":"2023-10-18T09:16:30.894504+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:16:30.894504+0000","last_undegraded":"2023-10-18T09:16:30.894504+0000","last_fullsized":"2023-10-18T09:16:30.894504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10164,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":364544,"data_stored":84802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":468315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.624}]},{"osd":1,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.372},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57999999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.502},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.39000000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.73999999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.753}]}]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1061196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.39900000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.55200000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.51000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.65300000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808859,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":472341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.58099999999999996,"15min":0.55100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.78200000000000003,"15min":0.76500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.63}]},{"osd":3,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.84999999999999998,"15min":0.82499999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.81299999999999994}]},{"osd":4,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.56100000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.5190000000000001,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.107,"5min":0.98199999999999998,"15min":0.96099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.86399999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.99199999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.99399999999999999,"15min":0.96599999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.4130000000000003,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.90700000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:16:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.98999999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]}]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":467861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57099999999999995,"15min":0.57699999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73299999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68200000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58599999999999997,"15min":0.58399999999999996},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.77900000000000003,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.75700000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78100000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.623},"min":{"1min":0.439,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.54000000000000004,"15min":0.54100000000000004},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.82199999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.92900000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.436,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.70799999999999996,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.92400000000000004,"5min":1.022,"15min":1.022},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:16:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69299999999999995,"15min":0.68899999999999995},"min":{"1min":0.498,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90200000000000002,"5min":1.254,"15min":1.254},"last":0.60999999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:16:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71999999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.86399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1058364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.40600000000000003,"15min":0.39600000000000002},"min":{"1min":0.29599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.56699999999999995},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.33600000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60699999999999998,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.61399999999999999,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.88600000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.55300000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.51700000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.67300000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68899999999999995,"15min":0.68300000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.97399999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.871}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98799999999999999,"5min":0.998,"15min":0.998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.89900000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.92300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":467123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45800000000000002,"15min":0.434},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0169999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0469999999999999,"5min":1.145,"15min":1.145},"last":0.499}]},{"osd":2,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91500000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.44600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97399999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.76100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.298,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.876},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.88100000000000001,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98499999999999999,"5min":11.112,"15min":11.112},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.66400000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.60599999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.115,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.80100000000000005}]},{"osd":8,"last update":"Wed Oct 18 09:16:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.84599999999999997,"15min":0.85899999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.077,"5min":1.51,"15min":1.51},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.79600000000000004},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.95499999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":9,"last update":"Wed Oct 18 09:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.747},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":490098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.51400000000000001,"15min":0.502},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:16:14 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.73999999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.50700000000000001},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.498,"5min":0.379,"15min":0.379},"max":{"1min":0.85099999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66900000000000004}]},{"osd":4,"last update":"Wed Oct 18 09:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.39300000000000002},"min":{"1min":0.29599999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.42299999999999999,"15min":0.40899999999999997},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.84499999999999997}]},{"osd":5,"last update":"Wed Oct 18 09:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55300000000000005,"15min":0.53300000000000003},"min":{"1min":0.48999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.621,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":5.258,"5min":5.258,"15min":5.258},"last":0.85999999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.069,"15min":1.069},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.083,"15min":1.083},"last":0.627}]},{"osd":7,"last update":"Wed Oct 18 09:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.34999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.60699999999999998},"min":{"1min":0.34699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.876}]},{"osd":8,"last update":"Wed Oct 18 09:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.47699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.55400000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.90600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:15:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.92000000000000004}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.95799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":468601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:15:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.56200000000000006,"15min":0.51700000000000002},"min":{"1min":0.497,"5min":0.314,"15min":0.314},"max":{"1min":0.83399999999999996,"5min":1.042,"15min":1.042},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.55500000000000005,"15min":0.53100000000000003},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.84899999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63200000000000001,"15min":0.59199999999999997},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.626,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:14:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.38200000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.47399999999999998,"15min":0.46000000000000002},"min":{"1min":0.35099999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.97999999999999998,"15min":1.105},"min":{"1min":0.41399999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3240000000000001,"5min":11.539,"15min":11.539},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.95499999999999996,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.37,"15min":0.37},"max":{"1min":1.071,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.45200000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.73499999999999999,"15min":0.71199999999999997},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0389999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.51000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.78900000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81999999999999995,"15min":0.81599999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.748},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.98699999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.85899999999999999,"15min":0.86899999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.92700000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.92600000000000005,"15min":0.93500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.0209999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.042},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.96999999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.0600000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":512973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52800000000000002,"15min":0.53900000000000003},"min":{"1min":0.40200000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.84999999999999998,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.60799999999999998},"min":{"1min":0.36699999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.75,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":0.70599999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.45500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54400000000000004,"15min":0.52600000000000002},"min":{"1min":0.434,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83399999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.747}]},{"osd":3,"last update":"Wed Oct 18 09:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56399999999999995,"15min":0.56299999999999994},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.503},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.53300000000000003},"min":{"1min":0.433,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88500000000000001,"5min":1.056,"15min":1.056},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Oct 18 09:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56000000000000005,"15min":0.53600000000000003},"min":{"1min":0.44,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.49299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73099999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:16:23 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41099999999999998,"15min":0.39600000000000002},"min":{"1min":0.30599999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.628},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.47399999999999998,"15min":0.44},"min":{"1min":0.28199999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.101,"15min":1.101},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.76400000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:16:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.66600000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.69199999999999995,"15min":0.69999999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85399999999999998}]},{"osd":9,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.82299999999999995}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":467317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.42399999999999999,"15min":0.38900000000000001},"min":{"1min":0.35499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.44900000000000001,"15min":0.41699999999999998},"min":{"1min":0.33500000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.68600000000000005,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.41599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89800000000000002,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.61199999999999999,"15min":0.57299999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.86899999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58899999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64900000000000002,"15min":0.63},"min":{"1min":0.52200000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.86199999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65000000000000002,"15min":0.63400000000000001},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.63400000000000001,"15min":0.59199999999999997},"min":{"1min":0.59199999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.628,"15min":0.59499999999999997},"min":{"1min":0.56299999999999994,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:16:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65900000000000003,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.622},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68400000000000005,"15min":0.66600000000000004},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":0.94199999999999995,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.501}]},{"osd":6,"last update":"Wed Oct 18 09:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.78100000000000003,"15min":0.77700000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.748}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.76600000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:16:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.78300000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":468213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.36099999999999999},"min":{"1min":0.33400000000000002,"5min":0.186,"15min":0.186},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.39800000000000002,"15min":0.38400000000000001},"min":{"1min":0.314,"5min":0.219,"15min":0.219},"max":{"1min":0.747,"5min":0.753,"15min":0.753},"last":0.436}]},{"osd":2,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.55000000000000004,"15min":0.53700000000000003},"min":{"1min":0.42199999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.73899999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.56399999999999995},"min":{"1min":0.47799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77600000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.48299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0009999999999999,"5min":1.077,"15min":1.077},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84099999999999997,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.41299999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:16:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.49099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96799999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.60599999999999998,"15min":0.57799999999999996},"min":{"1min":0.443,"5min":0.374,"15min":0.374},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72799999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65700000000000003,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92400000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.48999999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.46400000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.60699999999999998,"15min":0.58699999999999997},"min":{"1min":0.50600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.69499999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:16:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.67200000000000004,"15min":0.66900000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.89700000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.752,"15min":0.752},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76600000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95399999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.626},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.89900000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.71199999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:16:17 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68200000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1060119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53000000000000003,"15min":0.51100000000000001},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81399999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.499}]},{"osd":1,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.38800000000000001,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.38800000000000001},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56000000000000005,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.628},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56599999999999995,"15min":0.54200000000000004},"min":{"1min":0.45200000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80100000000000005,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76200000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.76400000000000001,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.79900000000000004,"5min":0.873,"15min":0.873},"last":0.82699999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80900000000000005,"5min":0.88,"15min":0.88},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Oct 18 09:16:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.85399999999999998,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.752},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63,"15min":0.628},"min":{"1min":0.40400000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872}]},{"osd":8,"last update":"Wed Oct 18 09:16:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60499999999999998,"15min":0.59899999999999998},"min":{"1min":0.42699999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71199999999999997}]},{"osd":9,"last update":"Wed Oct 18 09:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:16:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.93300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":22975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":46123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:17:02.034 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-18T09:17:02.034 INFO:tasks.cephadm:Setting up client nodes... 2023-10-18T09:17:02.035 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-18T09:17:02.814 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:02.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:02 smithi170 ceph-mon[135150]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:02.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:02 smithi110 ceph-mon[134818]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:03.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:02 smithi061 ceph-mon[98573]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:03 smithi061 ceph-mon[98573]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:03 smithi061 ceph-mon[98573]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2023-10-18T09:17:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:03 smithi170 ceph-mon[135150]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:03 smithi170 ceph-mon[135150]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2023-10-18T09:17:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:03 smithi110 ceph-mon[134818]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:03.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:03 smithi110 ceph-mon[134818]: pgmap v309: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2023-10-18T09:17:04.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 -- 172.21.15.61:0/2230746859 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 msgr2=0x7fae88100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:04.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 --2- 172.21.15.61:0/2230746859 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88100c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fae780099c0 tx=0x7fae78009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 -- 172.21.15.61:0/2230746859 shutdown_connections 2023-10-18T09:17:04.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 --2- 172.21.15.61:0/2230746859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 0x7fae88108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 --2- 172.21.15.61:0/2230746859 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88100c20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 --2- 172.21.15.61:0/2230746859 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 -- 172.21.15.61:0/2230746859 >> 172.21.15.61:0/2230746859 conn(0x7fae880fbda0 msgr2=0x7fae880fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:04.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 -- 172.21.15.61:0/2230746859 shutdown_connections 2023-10-18T09:17:04.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.312+0000 7fae8e049700 1 -- 172.21.15.61:0/2230746859 wait complete. 2023-10-18T09:17:04.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 Processor -- start 2023-10-18T09:17:04.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 -- start start 2023-10-18T09:17:04.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:04.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:04.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 0x7fae8819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:04.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819e270 con 0x7fae88100800 2023-10-18T09:17:04.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819e3b0 con 0x7fae88101a00 2023-10-18T09:17:04.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.313+0000 7fae8e049700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819e540 con 0x7fae881023a0 2023-10-18T09:17:04.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae877fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:04.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae877fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:41106/0 (socket says 172.21.15.61:41106) 2023-10-18T09:17:04.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae877fe700 1 -- 172.21.15.61:0/3857253282 learned_addr learned my addr 172.21.15.61:0/3857253282 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:04.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:04.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae87fff700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 0x7fae8819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:04.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 -- 172.21.15.61:0/3857253282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 msgr2=0x7fae8819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:04.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 0x7fae8819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 -- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 msgr2=0x7fae88196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:04.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 -- 172.21.15.61:0/3857253282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780095f0 con 0x7fae88101a00 2023-10-18T09:17:04.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.314+0000 7fae86ffd700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88196c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9f0 tx=0x7fae7c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:04.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.315+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c00eee0 con 0x7fae88101a00 2023-10-18T09:17:04.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.315+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8819e820 con 0x7fae88101a00 2023-10-18T09:17:04.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.315+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae7c00cbd0 con 0x7fae88101a00 2023-10-18T09:17:04.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.315+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c018680 con 0x7fae88101a00 2023-10-18T09:17:04.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.315+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8819ed70 con 0x7fae88101a00 2023-10-18T09:17:04.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.316+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae88107cd0 con 0x7fae88101a00 2023-10-18T09:17:04.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.318+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fae7c00cd40 con 0x7fae88101a00 2023-10-18T09:17:04.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.319+0000 7fae84ff9700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 0x7fae700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:04.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.319+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fae7c014070 con 0x7fae88101a00 2023-10-18T09:17:04.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.320+0000 7fae877fe700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 0x7fae700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:04.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.320+0000 7fae877fe700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 0x7fae700a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fae78000c00 tx=0x7fae78009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:04.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.324+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae7c052100 con 0x7fae88101a00 2023-10-18T09:17:04.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.617+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fae8810c050 con 0x7fae88101a00 2023-10-18T09:17:04.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.623+0000 7fae84ff9700 1 -- 172.21.15.61:0/3857253282 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fae7c096540 con 0x7fae88101a00 2023-10-18T09:17:04.626 INFO:teuthology.orchestra.run.smithi061.stdout:[client.0] 2023-10-18T09:17:04.626 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCQoi9lkvDwJBAALjUnjiMnWvUsvH7f6+uhbA== 2023-10-18T09:17:04.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 msgr2=0x7fae700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:04.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 0x7fae700a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fae78000c00 tx=0x7fae78009530 comp rx=0 tx=0).stop 2023-10-18T09:17:04.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 msgr2=0x7fae88196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:04.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88196c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00e9f0 tx=0x7fae7c00ed00 comp rx=0 tx=0).stop 2023-10-18T09:17:04.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 shutdown_connections 2023-10-18T09:17:04.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae700a1510 0x7fae700a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae881023a0 0x7fae8819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae88100800 0x7fae88196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 --2- 172.21.15.61:0/3857253282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae88101a00 0x7fae88196c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:04.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 >> 172.21.15.61:0/3857253282 conn(0x7fae880fbda0 msgr2=0x7fae88104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:04.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 shutdown_connections 2023-10-18T09:17:04.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:04.626+0000 7fae8e049700 1 -- 172.21.15.61:0/3857253282 wait complete. 2023-10-18T09:17:04.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:04 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3857253282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:04.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:04 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:04.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:04 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T09:17:05.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:04 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3857253282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:05.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:04 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:05.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:04 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T09:17:05.123 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:17:05.124 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-18T09:17:05.124 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-18T09:17:05.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:04 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3857253282' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:05.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:04 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-18T09:17:05.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:04 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-18T09:17:05.227 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph config log 1 --format=json 2023-10-18T09:17:05.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:05 smithi110 ceph-mon[134818]: pgmap v310: 129 pgs: 129 active+clean; 627 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-10-18T09:17:06.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:05 smithi061 ceph-mon[98573]: pgmap v310: 129 pgs: 129 active+clean; 627 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-10-18T09:17:06.012 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:06.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:05 smithi170 ceph-mon[135150]: pgmap v310: 129 pgs: 129 active+clean; 627 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-10-18T09:17:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 -- 172.21.15.61:0/4282991501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e4073340 msgr2=0x7f85e410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/4282991501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e4073340 0x7f85e410cd10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009990 tx=0x7f85d0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 -- 172.21.15.61:0/4282991501 shutdown_connections 2023-10-18T09:17:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/4282991501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e410d250 0x7f85e410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/4282991501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 0x7f85e4072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/4282991501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e4073340 0x7f85e410cd10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 -- 172.21.15.61:0/4282991501 >> 172.21.15.61:0/4282991501 conn(0x7f85e40779e0 msgr2=0x7f85e4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 -- 172.21.15.61:0/4282991501 shutdown_connections 2023-10-18T09:17:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.521+0000 7f85eaf3d700 1 -- 172.21.15.61:0/4282991501 wait complete. 2023-10-18T09:17:07.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.522+0000 7f85eaf3d700 1 Processor -- start 2023-10-18T09:17:07.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 -- start start 2023-10-18T09:17:07.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 0x7f85e4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:07.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e410d250 0x7f85e419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419ddc0 con 0x7f85e410d250 2023-10-18T09:17:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419df30 con 0x7f85e4072a20 2023-10-18T09:17:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85eaf3d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419e0f0 con 0x7f85e4073340 2023-10-18T09:17:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85e3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85e94da700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e410d250 0x7f85e419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85e3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:40108/0 (socket says 172.21.15.61:40108) 2023-10-18T09:17:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85e3fff700 1 -- 172.21.15.61:0/57662945 learned_addr learned my addr 172.21.15.61:0/57662945 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.523+0000 7f85e8cd9700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 0x7f85e4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 -- 172.21.15.61:0/57662945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 msgr2=0x7f85e4198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 0x7f85e4198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 -- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e410d250 msgr2=0x7f85e419d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e410d250 0x7f85e419d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 -- 172.21.15.61:0/57662945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00095f0 con 0x7f85e4073340 2023-10-18T09:17:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e3fff700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005910 tx=0x7f85d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d001d070 con 0x7f85e4073340 2023-10-18T09:17:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e419e370 con 0x7f85e4073340 2023-10-18T09:17:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.524+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85d0022470 con 0x7f85e4073340 2023-10-18T09:17:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.525+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0017650 con 0x7f85e4073340 2023-10-18T09:17:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.525+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e41a5c30 con 0x7f85e4073340 2023-10-18T09:17:07.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.526+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e4108460 con 0x7f85e4073340 2023-10-18T09:17:07.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.528+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f85d00225e0 con 0x7f85e4073340 2023-10-18T09:17:07.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.529+0000 7f85e1ffb700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 0x7f85d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:07.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.529+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f85d00ce360 con 0x7f85e4073340 2023-10-18T09:17:07.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.529+0000 7f85e8cd9700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 0x7f85d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:07.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.530+0000 7f85e8cd9700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 0x7f85d40a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005950 tx=0x7f85d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:07.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.531+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f85d00976a0 con 0x7f85e4073340 2023-10-18T09:17:07.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.741+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7f85e404e4c0 con 0x7f85e4073340 2023-10-18T09:17:07.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.742+0000 7f85e1ffb700 1 -- 172.21.15.61:0/57662945 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v43) v1 ==== 86+0+133 (secure 0 0 0) 0x7f85d009acc0 con 0x7f85e4073340 2023-10-18T09:17:07.745 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:07.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 msgr2=0x7f85d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:07.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 0x7f85d40a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f85d8005950 tx=0x7f85d800b410 comp rx=0 tx=0).stop 2023-10-18T09:17:07.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 msgr2=0x7f85e4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:07.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85d0005910 tx=0x7f85d0004730 comp rx=0 tx=0).stop 2023-10-18T09:17:07.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 shutdown_connections 2023-10-18T09:17:07.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f85d40a1510 0x7f85d40a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f85e410d250 0x7f85e419d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85e4072a20 0x7f85e4198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 --2- 172.21.15.61:0/57662945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e4073340 0x7f85e4199400 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:07.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 >> 172.21.15.61:0/57662945 conn(0x7f85e40779e0 msgr2=0x7f85e40fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:07.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 shutdown_connections 2023-10-18T09:17:07.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:07.745+0000 7f85eaf3d700 1 -- 172.21.15.61:0/57662945 wait complete. 2023-10-18T09:17:07.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:07 smithi061 ceph-mon[98573]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:07.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:07 smithi170 ceph-mon[135150]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:07.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:07 smithi110 ceph-mon[134818]: pgmap v311: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:08.213 INFO:teuthology.orchestra.run.smithi061.stdout:[{"version":43,"timestamp":"2023-10-18T09:16:20.269002+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2023-10-18T09:17:08.214 INFO:tasks.ceph_manager:config epoch is 43 2023-10-18T09:17:08.214 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-18T09:17:08.215 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-18T09:17:08.215 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-18T09:17:08.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:08 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/57662945' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T09:17:08.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:08 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/57662945' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T09:17:08.963 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:09.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:08 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/57662945' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2023-10-18T09:17:09.843 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:09 smithi061 ceph-mon[98573]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:09.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:09 smithi170 ceph-mon[135150]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:09.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:09 smithi110 ceph-mon[134818]: pgmap v312: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2023-10-18T09:17:10.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 -- 172.21.15.61:0/3351762555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf681057c0 msgr2=0x7fdf68105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:10.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/3351762555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf681057c0 0x7fdf68105ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009990 tx=0x7fdf64009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 -- 172.21.15.61:0/3351762555 shutdown_connections 2023-10-18T09:17:10.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/3351762555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf68101260 0x7fdf6810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/3351762555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf681057c0 0x7fdf68105ba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/3351762555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 0x7fdf68100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 -- 172.21.15.61:0/3351762555 >> 172.21.15.61:0/3351762555 conn(0x7fdf680779e0 msgr2=0x7fdf68077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:10.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 -- 172.21.15.61:0/3351762555 shutdown_connections 2023-10-18T09:17:10.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.467+0000 7fdf6fde4700 1 -- 172.21.15.61:0/3351762555 wait complete. 2023-10-18T09:17:10.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.468+0000 7fdf6fde4700 1 Processor -- start 2023-10-18T09:17:10.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 -- start start 2023-10-18T09:17:10.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 0x7fdf6819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:10.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:10.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf681057c0 0x7fdf68196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:10.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681970b0 con 0x7fdf68101260 2023-10-18T09:17:10.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68197220 con 0x7fdf681008a0 2023-10-18T09:17:10.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6fde4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681973e0 con 0x7fdf681057c0 2023-10-18T09:17:10.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6d37f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:10.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6d37f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53564/0 (socket says 172.21.15.61:53564) 2023-10-18T09:17:10.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6d37f700 1 -- 172.21.15.61:0/2582000386 learned_addr learned my addr 172.21.15.61:0/2582000386 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:10.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.469+0000 7fdf6db80700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 0x7fdf6819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:10.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6e381700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf681057c0 0x7fdf68196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:10.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 -- 172.21.15.61:0/2582000386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf681057c0 msgr2=0x7fdf68196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:10.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf681057c0 0x7fdf68196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 -- 172.21.15.61:0/2582000386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 msgr2=0x7fdf6819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:10.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 0x7fdf6819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 -- 172.21.15.61:0/2582000386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf640095f0 con 0x7fdf68101260 2023-10-18T09:17:10.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6d37f700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d950 tx=0x7fdf5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:10.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800f890 con 0x7fdf68101260 2023-10-18T09:17:10.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf58010460 con 0x7fdf68101260 2023-10-18T09:17:10.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf681976c0 con 0x7fdf68101260 2023-10-18T09:17:10.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.470+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800e610 con 0x7fdf68101260 2023-10-18T09:17:10.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.471+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a3900 con 0x7fdf68101260 2023-10-18T09:17:10.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.472+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68108d80 con 0x7fdf68101260 2023-10-18T09:17:10.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.474+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fdf580108d0 con 0x7fdf68101260 2023-10-18T09:17:10.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.475+0000 7fdf5effd700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 0x7fdf540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:10.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.475+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdf580cce70 con 0x7fdf68101260 2023-10-18T09:17:10.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.475+0000 7fdf6db80700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 0x7fdf540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:10.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.476+0000 7fdf6db80700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 0x7fdf540a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:10.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.480+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf58096120 con 0x7fdf68101260 2023-10-18T09:17:10.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.749+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fdf68101f30 con 0x7fdf68101260 2023-10-18T09:17:10.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.754+0000 7fdf5effd700 1 -- 172.21.15.61:0/2582000386 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260405 (secure 0 0 0) 0x7fdf58016090 con 0x7fdf68101260 2023-10-18T09:17:10.762 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:10.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 msgr2=0x7fdf540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:10.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 0x7fdf540a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006010 tx=0x7fdf64005960 comp rx=0 tx=0).stop 2023-10-18T09:17:10.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 msgr2=0x7fdf6819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:10.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800d950 tx=0x7fdf5800dc60 comp rx=0 tx=0).stop 2023-10-18T09:17:10.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 shutdown_connections 2023-10-18T09:17:10.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf540a1510 0x7fdf540a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf681057c0 0x7fdf68196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf681008a0 0x7fdf6819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 --2- 172.21.15.61:0/2582000386 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf68101260 0x7fdf6819ce50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:10.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 >> 172.21.15.61:0/2582000386 conn(0x7fdf680779e0 msgr2=0x7fdf680fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:10.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 shutdown_connections 2023-10-18T09:17:10.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:10.759+0000 7fdf6fde4700 1 -- 172.21.15.61:0/2582000386 wait complete. 2023-10-18T09:17:11.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:10 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2582000386' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T09:17:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:10 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2582000386' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T09:17:11.235 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:10 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2582000386' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-18T09:17:11.264 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":3155127227},{"type":"v1","addr":"172.21.15.61:6801","nonce":3155127227}]},"active_addr":"172.21.15.61:6801/3155127227","active_change":"2023-10-18T09:09:20.739876+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.61:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":283719276}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":217188384}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":1868139328}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":3233637384}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":923012263}]}]} 2023-10-18T09:17:11.271 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-18T09:17:11.271 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-18T09:17:11.271 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T09:17:12.020 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:12.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:11 smithi170 ceph-mon[135150]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-18T09:17:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:11 smithi110 ceph-mon[134818]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-18T09:17:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:11 smithi061 ceph-mon[98573]: pgmap v313: 129 pgs: 129 active+clean; 635 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.0 KiB/s wr, 25 op/s 2023-10-18T09:17:13.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 -- 172.21.15.61:0/3888621166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 msgr2=0x7f7d10105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:13.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 --2- 172.21.15.61:0/3888621166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d10105ba0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009990 tx=0x7f7d0c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 -- 172.21.15.61:0/3888621166 shutdown_connections 2023-10-18T09:17:13.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 --2- 172.21.15.61:0/3888621166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d1010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 --2- 172.21.15.61:0/3888621166 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d10105ba0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 --2- 172.21.15.61:0/3888621166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 0x7f7d10100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 -- 172.21.15.61:0/3888621166 >> 172.21.15.61:0/3888621166 conn(0x7f7d100779e0 msgr2=0x7f7d10077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:13.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 -- 172.21.15.61:0/3888621166 shutdown_connections 2023-10-18T09:17:13.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.559+0000 7f7d18442700 1 -- 172.21.15.61:0/3888621166 wait complete. 2023-10-18T09:17:13.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.560+0000 7f7d18442700 1 Processor -- start 2023-10-18T09:17:13.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.560+0000 7f7d18442700 1 -- start start 2023-10-18T09:17:13.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 0x7f7d10198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:13.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:13.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d1019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019dd50 con 0x7f7d101057c0 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019dec0 con 0x7f7d101008a0 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d18442700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1019e080 con 0x7f7d10101260 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d159dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d169df700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d1019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:13.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d159dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:49266/0 (socket says 172.21.15.61:49266) 2023-10-18T09:17:13.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d159dd700 1 -- 172.21.15.61:0/1590466193 learned_addr learned my addr 172.21.15.61:0/1590466193 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:13.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.561+0000 7f7d161de700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 0x7f7d10198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:13.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 -- 172.21.15.61:0/1590466193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 msgr2=0x7f7d10198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:13.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 0x7f7d10198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 -- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 msgr2=0x7f7d1019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d1019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 -- 172.21.15.61:0/1590466193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0c0095f0 con 0x7f7d10101260 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d169df700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d1019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d159dd700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000e9a0 tx=0x7f7d0000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00009790 con 0x7f7d10101260 2023-10-18T09:17:13.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d00010430 con 0x7f7d10101260 2023-10-18T09:17:13.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1019e360 con 0x7f7d10101260 2023-10-18T09:17:13.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00018680 con 0x7f7d10101260 2023-10-18T09:17:13.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.562+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d101a5bc0 con 0x7f7d10101260 2023-10-18T09:17:13.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.563+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10101f30 con 0x7f7d10101260 2023-10-18T09:17:13.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.565+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f7d000105a0 con 0x7f7d10101260 2023-10-18T09:17:13.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.567+0000 7f7d077fe700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 0x7f7cfc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:13.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.567+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7d00014070 con 0x7f7d10101260 2023-10-18T09:17:13.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.567+0000 7f7d161de700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 0x7f7cfc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:13.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.568+0000 7f7d161de700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 0x7f7cfc0a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b2e0 tx=0x7f7d0c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:13.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.569+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7d00096510 con 0x7f7d10101260 2023-10-18T09:17:13.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.777+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d10110760 con 0x7f7d10101260 2023-10-18T09:17:13.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.779+0000 7f7d077fe700 1 -- 172.21.15.61:0/1590466193 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7f7d00099b30 con 0x7f7d10101260 2023-10-18T09:17:13.782 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:13.782 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:17:13.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.781+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 msgr2=0x7f7cfc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:13.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.781+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 0x7f7cfc0a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00b2e0 tx=0x7f7d0c005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.781+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 msgr2=0x7f7d10199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.781+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000e9a0 tx=0x7f7d0000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 shutdown_connections 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7cfc0a1510 0x7f7cfc0a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7d101057c0 0x7f7d1019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d101008a0 0x7f7d10198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 --2- 172.21.15.61:0/1590466193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d10101260 0x7f7d10199390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:13.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 >> 172.21.15.61:0/1590466193 conn(0x7f7d100779e0 msgr2=0x7f7d100fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:13.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 shutdown_connections 2023-10-18T09:17:13.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:13.782+0000 7f7d18442700 1 -- 172.21.15.61:0/1590466193 wait complete. 2023-10-18T09:17:14.175 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:13 smithi061 ceph-mon[98573]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-18T09:17:14.176 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:13 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1590466193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:13 smithi110 ceph-mon[134818]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-18T09:17:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:13 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1590466193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:14.220 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-18T09:17:14.221 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-18T09:17:14.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:13 smithi170 ceph-mon[135150]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2023-10-18T09:17:14.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:13 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1590466193' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:14.971 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:15 smithi110 ceph-mon[134818]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-10-18T09:17:16.238 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:15 smithi061 ceph-mon[98573]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-10-18T09:17:16.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:15 smithi170 ceph-mon[135150]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-10-18T09:17:16.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 -- 172.21.15.61:0/4135684323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb381057c0 msgr2=0x7fbb38105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:16.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 --2- 172.21.15.61:0/4135684323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb381057c0 0x7fbb38105ba0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009990 tx=0x7fbb34009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 -- 172.21.15.61:0/4135684323 shutdown_connections 2023-10-18T09:17:16.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 --2- 172.21.15.61:0/4135684323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb38101260 0x7fbb3810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 --2- 172.21.15.61:0/4135684323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb381057c0 0x7fbb38105ba0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 --2- 172.21.15.61:0/4135684323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381008a0 0x7fbb38100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 -- 172.21.15.61:0/4135684323 >> 172.21.15.61:0/4135684323 conn(0x7fbb380779e0 msgr2=0x7fbb38077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:16.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 -- 172.21.15.61:0/4135684323 shutdown_connections 2023-10-18T09:17:16.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.484+0000 7fbb404fd700 1 -- 172.21.15.61:0/4135684323 wait complete. 2023-10-18T09:17:16.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 Processor -- start 2023-10-18T09:17:16.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 -- start start 2023-10-18T09:17:16.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381008a0 0x7fbb38198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:16.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:16.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381057c0 0x7fbb3819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:16.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3819dda0 con 0x7fbb38101260 2023-10-18T09:17:16.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3819df10 con 0x7fbb381057c0 2023-10-18T09:17:16.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.485+0000 7fbb404fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3819e0d0 con 0x7fbb381008a0 2023-10-18T09:17:16.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:16.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:53602/0 (socket says 172.21.15.61:53602) 2023-10-18T09:17:16.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 -- 172.21.15.61:0/92949608 learned_addr learned my addr 172.21.15.61:0/92949608 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:16.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3e299700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381008a0 0x7fbb38198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:16.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3ea9a700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381057c0 0x7fbb3819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:16.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 -- 172.21.15.61:0/92949608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381008a0 msgr2=0x7fbb38198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:16.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381008a0 0x7fbb38198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 -- 172.21.15.61:0/92949608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381057c0 msgr2=0x7fbb3819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:16.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381057c0 0x7fbb3819d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 -- 172.21.15.61:0/92949608 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340095f0 con 0x7fbb38101260 2023-10-18T09:17:16.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.486+0000 7fbb3da98700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800d950 tx=0x7fbb2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:16.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.487+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800f890 con 0x7fbb38101260 2023-10-18T09:17:16.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.487+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3819e3b0 con 0x7fbb38101260 2023-10-18T09:17:16.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.487+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb28010460 con 0x7fbb38101260 2023-10-18T09:17:16.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.487+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800e610 con 0x7fbb38101260 2023-10-18T09:17:16.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.487+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb381a5c10 con 0x7fbb38101260 2023-10-18T09:17:16.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.488+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb38101f30 con 0x7fbb38101260 2023-10-18T09:17:16.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.490+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbb280108d0 con 0x7fbb38101260 2023-10-18T09:17:16.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.491+0000 7fbb2f7fe700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 0x7fbb240a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:16.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.492+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb280cce70 con 0x7fbb38101260 2023-10-18T09:17:16.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.492+0000 7fbb3e299700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 0x7fbb240a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:16.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.492+0000 7fbb3e299700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 0x7fbb240a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400b5c0 tx=0x7fbb34005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:16.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.496+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb28096120 con 0x7fbb38101260 2023-10-18T09:17:16.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.712+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb3804e4c0 con 0x7fbb38101260 2023-10-18T09:17:16.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.713+0000 7fbb2f7fe700 1 -- 172.21.15.61:0/92949608 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23925 (secure 0 0 0) 0x7fbb28016090 con 0x7fbb38101260 2023-10-18T09:17:16.716 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:16.716 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":88,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:16:31.407031+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:17:16.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 msgr2=0x7fbb240a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 0x7fbb240a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbb3400b5c0 tx=0x7fbb34005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 msgr2=0x7fbb381993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800d950 tx=0x7fbb2800dc60 comp rx=0 tx=0).stop 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 shutdown_connections 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1510 0x7fbb240a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb381057c0 0x7fbb3819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381008a0 0x7fbb38198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 --2- 172.21.15.61:0/92949608 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38101260 0x7fbb381993e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:16.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 >> 172.21.15.61:0/92949608 conn(0x7fbb380779e0 msgr2=0x7fbb3810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:16.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 shutdown_connections 2023-10-18T09:17:16.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:16.716+0000 7fbb404fd700 1 -- 172.21.15.61:0/92949608 wait complete. 2023-10-18T09:17:17.173 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-18T09:17:17.174 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-18T09:17:17.175 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-18T09:17:17.175 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-18T09:17:17.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-18T09:17:17.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-18T09:17:17.177 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-18T09:17:17.177 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-18T09:17:17.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-18T09:17:17.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-18T09:17:17.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-18T09:17:17.180 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-18T09:17:17.760 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:17 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/92949608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:17.760 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:17 smithi061 ceph-mon[98573]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-10-18T09:17:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:17 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/92949608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:17 smithi170 ceph-mon[135150]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-10-18T09:17:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:17 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/92949608' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:17 smithi110 ceph-mon[134818]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2023-10-18T09:17:18.864 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:19.027 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:19.055 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:19 smithi061 ceph-mon[98573]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:17:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:19 smithi110 ceph-mon[134818]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:17:20.262 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.392 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:19 smithi170 ceph-mon[135150]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:17:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.581 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.781 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:20.902 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:21.186 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-18T09:17:21.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T09:17:21.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-10-18T09:17:21.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-18T09:17:21.187 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2023-10-18T09:17:21.188 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T09:17:21.188 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-10-18T09:17:21.188 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2023-10-18T09:17:21.188 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.188 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-18T09:17:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T09:17:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-10-18T09:17:21.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-18T09:17:21.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2023-10-18T09:17:21.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T09:17:21.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-10-18T09:17:21.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2023-10-18T09:17:21.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.223 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-10-18T09:17:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2023-10-18T09:17:21.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2023-10-18T09:17:21.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-18T09:17:21.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2023-10-18T09:17:21.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2023-10-18T09:17:21.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-10-18T09:17:21.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2023-10-18T09:17:21.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-10-18T09:17:21.456 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:22.178 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-18T09:17:22.179 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-18T09:17:22.179 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T09:17:22.179 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-10-18T09:17:22.179 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-18T09:17:22.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2023-10-18T09:17:22.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T09:17:22.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-10-18T09:17:22.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2023-10-18T09:17:22.180 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-10-18T09:17:22.181 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-10-18T09:17:22.181 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:21 smithi061 ceph-mon[98573]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T09:17:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-18T09:17:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-18T09:17:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-10-18T09:17:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2023-10-18T09:17:22.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-10-18T09:17:22.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-10-18T09:17:22.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:21 smithi110 ceph-mon[134818]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T09:17:22.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2023-10-18T09:17:22.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-10-18T09:17:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2023-10-18T09:17:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2023-10-18T09:17:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-18T09:17:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2023-10-18T09:17:22.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2023-10-18T09:17:22.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-10-18T09:17:22.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2023-10-18T09:17:22.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2023-10-18T09:17:22.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-10-18T09:17:22.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:21 smithi170 ceph-mon[135150]: osdmap e89: 12 total, 12 up, 12 in 2023-10-18T09:17:22.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 -- 172.21.15.61:0/3949068390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c410ce30 msgr2=0x7fe1c410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3949068390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c410ce30 0x7fe1c410f2a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00b0a0 tx=0x7fe1bc00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 -- 172.21.15.61:0/3949068390 shutdown_connections 2023-10-18T09:17:22.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3949068390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c410ce30 0x7fe1c410f2a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3949068390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4072c20 0x7fe1c4073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3949068390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1c4073540 0x7fe1c410c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 -- 172.21.15.61:0/3949068390 >> 172.21.15.61:0/3949068390 conn(0x7fe1c40fc120 msgr2=0x7fe1c40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:22.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 -- 172.21.15.61:0/3949068390 shutdown_connections 2023-10-18T09:17:22.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.882+0000 7fe1c359e700 1 -- 172.21.15.61:0/3949068390 wait complete. 2023-10-18T09:17:22.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.883+0000 7fe1c359e700 1 Processor -- start 2023-10-18T09:17:22.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.883+0000 7fe1c359e700 1 -- start start 2023-10-18T09:17:22.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 0x7fe1c4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:22.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4073540 0x7fe1c4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:22.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1c410ce30 0x7fe1c419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:22.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c419dca0 con 0x7fe1c4072c20 2023-10-18T09:17:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c419de10 con 0x7fe1c4073540 2023-10-18T09:17:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c419dfd0 con 0x7fe1c410ce30 2023-10-18T09:17:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 0x7fe1c4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 0x7fe1c4198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:40304/0 (socket says 172.21.15.61:40304) 2023-10-18T09:17:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c259c700 1 -- 172.21.15.61:0/3998810626 learned_addr learned my addr 172.21.15.61:0/3998810626 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c2d9d700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1c410ce30 0x7fe1c419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c1d9b700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4073540 0x7fe1c4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.884+0000 7fe1c259c700 1 -- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1c410ce30 msgr2=0x7fe1c419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c259c700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1c410ce30 0x7fe1c419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c259c700 1 -- 172.21.15.61:0/3998810626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4073540 msgr2=0x7fe1c4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c259c700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4073540 0x7fe1c4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c259c700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc009d00 con 0x7fe1c4072c20 2023-10-18T09:17:22.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c1d9b700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe1c4073540 0x7fe1c4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:17:22.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c259c700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 0x7fe1c4198df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9a0 tx=0x7fe1b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:22.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4009790 con 0x7fe1c4072c20 2023-10-18T09:17:22.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c419e2b0 con 0x7fe1c4072c20 2023-10-18T09:17:22.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1b400cbd0 con 0x7fe1c4072c20 2023-10-18T09:17:22.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4018680 con 0x7fe1c4072c20 2023-10-18T09:17:22.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.885+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c41a5b10 con 0x7fe1c4072c20 2023-10-18T09:17:22.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.886+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1c4112a20 con 0x7fe1c4072c20 2023-10-18T09:17:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.889+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe1b400cd40 con 0x7fe1c4072c20 2023-10-18T09:17:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1b37fe700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1ac0a1510 0x7fe1ac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe1b4014070 con 0x7fe1c4072c20 2023-10-18T09:17:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1c1d9b700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1ac0a1510 0x7fe1ac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1b37fe700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] conn(0x7fe1ac0ac530 0x7fe1ac0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:22.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1ac0af0a0 con 0x7fe1ac0ac530 2023-10-18T09:17:22.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.890+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1b40cdb80 con 0x7fe1c4072c20 2023-10-18T09:17:22.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.891+0000 7fe1c2d9d700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] conn(0x7fe1ac0ac530 0x7fe1ac0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:22.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.891+0000 7fe1c1d9b700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1ac0a1510 0x7fe1ac0a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1c419a300 tx=0x7fe1b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:22.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.891+0000 7fe1c2d9d700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] conn(0x7fe1ac0ac530 0x7fe1ac0ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:22.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.892+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== osd.8 v2:172.21.15.170:6816/3321062874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fe1ac0af0a0 con 0x7fe1ac0ac530 2023-10-18T09:17:22.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.918+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 --> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1c40742d0 con 0x7fe1ac0ac530 2023-10-18T09:17:22.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.919+0000 7fe1b37fe700 1 -- 172.21.15.61:0/3998810626 <== osd.8 v2:172.21.15.170:6816/3321062874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe1c40742d0 con 0x7fe1ac0ac530 2023-10-18T09:17:22.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] conn(0x7fe1ac0ac530 msgr2=0x7fe1ac0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] conn(0x7fe1ac0ac530 0x7fe1ac0ae9d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1ac0a1510 msgr2=0x7fe1ac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1ac0a1510 0x7fe1ac0a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1c419a300 tx=0x7fe1b8009260 comp rx=0 tx=0).stop 2023-10-18T09:17:22.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 msgr2=0x7fe1c4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:22.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 --2- 172.21.15.61:0/3998810626 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe1c4072c20 0x7fe1c4198df0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9a0 tx=0x7fe1b400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:22.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c2d9d700 1 -- 172.21.15.61:0/3998810626 reap_dead start 2023-10-18T09:17:22.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 shutdown_connections 2023-10-18T09:17:22.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.920+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 >> 172.21.15.61:0/3998810626 conn(0x7fe1c40fc120 msgr2=0x7fe1c40fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:22.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.921+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 shutdown_connections 2023-10-18T09:17:22.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:21.921+0000 7fe1c359e700 1 -- 172.21.15.61:0/3998810626 wait complete. 2023-10-18T09:17:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:22 smithi110 ceph-mon[134818]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T09:17:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:22 smithi061 ceph-mon[98573]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T09:17:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:22 smithi170 ceph-mon[135150]: osdmap e90: 12 total, 12 up, 12 in 2023-10-18T09:17:24.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 -- 172.21.15.61:0/2848612544 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee54072a20 msgr2=0x7fee54072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/2848612544 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee54072a20 0x7fee54072e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fee44009990 tx=0x7fee44009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 -- 172.21.15.61:0/2848612544 shutdown_connections 2023-10-18T09:17:24.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/2848612544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee5410d250 0x7fee5410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/2848612544 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee54072a20 0x7fee54072e00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/2848612544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 0x7fee5410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.105+0000 7fee5a0b4700 1 -- 172.21.15.61:0/2848612544 >> 172.21.15.61:0/2848612544 conn(0x7fee540779e0 msgr2=0x7fee54077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:24.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.106+0000 7fee5a0b4700 1 -- 172.21.15.61:0/2848612544 shutdown_connections 2023-10-18T09:17:24.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.106+0000 7fee5a0b4700 1 -- 172.21.15.61:0/2848612544 wait complete. 2023-10-18T09:17:24.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.107+0000 7fee5a0b4700 1 Processor -- start 2023-10-18T09:17:24.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.107+0000 7fee5a0b4700 1 -- start start 2023-10-18T09:17:24.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54072a20 0x7fee54198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 0x7fee54199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 0x7fee5419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419ddb0 con 0x7fee5410d250 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419df20 con 0x7fee54073340 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee5a0b4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5419e0e0 con 0x7fee54072a20 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee53fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 0x7fee5419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee53fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 0x7fee5419d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:40320/0 (socket says 172.21.15.61:40320) 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54072a20 0x7fee54198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee53fff700 1 -- 172.21.15.61:0/3354111864 learned_addr learned my addr 172.21.15.61:0/3354111864 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:24.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.108+0000 7fee52ffd700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 0x7fee54199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 -- 172.21.15.61:0/3354111864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54072a20 msgr2=0x7fee54198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54072a20 0x7fee54198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 -- 172.21.15.61:0/3354111864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 msgr2=0x7fee54199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 0x7fee54199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee440095f0 con 0x7fee5410d250 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee537fe700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee54072a20 0x7fee54198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee52ffd700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee54073340 0x7fee54199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee53fff700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 0x7fee5419d7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00efc0 tx=0x7fee4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c018070 con 0x7fee5410d250 2023-10-18T09:17:24.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee4c00f040 con 0x7fee5410d250 2023-10-18T09:17:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5419e360 con 0x7fee5410d250 2023-10-18T09:17:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c00cd00 con 0x7fee5410d250 2023-10-18T09:17:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.109+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee541a5c20 con 0x7fee5410d250 2023-10-18T09:17:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.110+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee54112d10 con 0x7fee5410d250 2023-10-18T09:17:24.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.112+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fee4c003a00 con 0x7fee5410d250 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.115+0000 7fee50ff9700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fee3c0a1590 0x7fee3c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.115+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee4c0cc870 con 0x7fee5410d250 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.115+0000 7fee537fe700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fee3c0a1590 0x7fee3c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee50ff9700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] conn(0x7fee3c0ac5b0 0x7fee3c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee3c0af120 con 0x7fee3c0ac5b0 2023-10-18T09:17:24.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee4c0cee50 con 0x7fee5410d250 2023-10-18T09:17:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee537fe700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fee3c0a1590 0x7fee3c0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b2e0 tx=0x7fee44005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee52ffd700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] conn(0x7fee3c0ac5b0 0x7fee3c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.116+0000 7fee52ffd700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] conn(0x7fee3c0ac5b0 0x7fee3c0aea50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.117+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== osd.9 v2:172.21.15.61:6826/65349699 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fee3c0af120 con 0x7fee3c0ac5b0 2023-10-18T09:17:24.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.146+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 --> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee540740b0 con 0x7fee3c0ac5b0 2023-10-18T09:17:24.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee50ff9700 1 -- 172.21.15.61:0/3354111864 <== osd.9 v2:172.21.15.61:6826/65349699 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fee540740b0 con 0x7fee3c0ac5b0 2023-10-18T09:17:24.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] conn(0x7fee3c0ac5b0 msgr2=0x7fee3c0aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] conn(0x7fee3c0ac5b0 0x7fee3c0aea50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fee3c0a1590 msgr2=0x7fee3c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fee3c0a1590 0x7fee3c0a3a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fee4400b2e0 tx=0x7fee44005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 msgr2=0x7fee5419d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee5a0b4700 1 --2- 172.21.15.61:0/3354111864 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fee5410d250 0x7fee5419d7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00efc0 tx=0x7fee4c00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.147+0000 7fee53fff700 1 -- 172.21.15.61:0/3354111864 reap_dead start 2023-10-18T09:17:24.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.148+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 shutdown_connections 2023-10-18T09:17:24.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.148+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 >> 172.21.15.61:0/3354111864 conn(0x7fee540779e0 msgr2=0x7fee540fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:24.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.148+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 shutdown_connections 2023-10-18T09:17:24.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.148+0000 7fee5a0b4700 1 -- 172.21.15.61:0/3354111864 wait complete. 2023-10-18T09:17:24.174 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:23 smithi061 ceph-mon[98573]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-18T09:17:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:23 smithi110 ceph-mon[134818]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-18T09:17:24.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 -- 172.21.15.61:0/3113251398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 msgr2=0x7f8d640685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/3113251398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d640685f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d540098f0 tx=0x7f8d54009c00 comp rx=0 tx=0).stop 2023-10-18T09:17:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 -- 172.21.15.61:0/3113251398 shutdown_connections 2023-10-18T09:17:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/3113251398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 0x7f8d6410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/3113251398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d64106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/3113251398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d640685f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 -- 172.21.15.61:0/3113251398 >> 172.21.15.61:0/3113251398 conn(0x7f8d64075790 msgr2=0x7f8d64075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 -- 172.21.15.61:0/3113251398 shutdown_connections 2023-10-18T09:17:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.369+0000 7f8d6bbce700 1 -- 172.21.15.61:0/3113251398 wait complete. 2023-10-18T09:17:24.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.370+0000 7f8d6bbce700 1 Processor -- start 2023-10-18T09:17:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.370+0000 7f8d6bbce700 1 -- start start 2023-10-18T09:17:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d64196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 0x7f8d641972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d6419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6419bbb0 con 0x7f8d64068190 2023-10-18T09:17:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6419bd20 con 0x7f8d641067a0 2023-10-18T09:17:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6bbce700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6419bee0 con 0x7f8d64068b30 2023-10-18T09:17:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6996a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d64196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6a16b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d6419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6a16b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d6419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43508/0 (socket says 172.21.15.61:43508) 2023-10-18T09:17:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.371+0000 7f8d6a16b700 1 -- 172.21.15.61:0/747902167 learned_addr learned my addr 172.21.15.61:0/747902167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d69169700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 0x7f8d641972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 -- 172.21.15.61:0/747902167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 msgr2=0x7f8d641972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 0x7f8d641972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 -- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 msgr2=0x7f8d64196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d64196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c009d00 con 0x7f8d641067a0 2023-10-18T09:17:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6996a700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8d64068190 0x7f8d64196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d69169700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d64068b30 0x7f8d641972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:17:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d6a16b700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d6419b5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e3c0 tx=0x7f8d5c00e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.372+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0092d0 con 0x7f8d641067a0 2023-10-18T09:17:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.373+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c00f040 con 0x7f8d641067a0 2023-10-18T09:17:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.373+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c007d70 con 0x7f8d641067a0 2023-10-18T09:17:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.373+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d54009520 con 0x7f8d641067a0 2023-10-18T09:17:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.373+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d641a3870 con 0x7f8d641067a0 2023-10-18T09:17:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.374+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d64110980 con 0x7f8d641067a0 2023-10-18T09:17:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.376+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8d5c003a00 con 0x7f8d641067a0 2023-10-18T09:17:24.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.378+0000 7f8d5affd700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8d500a1510 0x7f8d500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d5c0cd560 con 0x7f8d641067a0 2023-10-18T09:17:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d6996a700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8d500a1510 0x7f8d500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d5affd700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] conn(0x7f8d500ac530 0x7f8d500ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d500af0a0 con 0x7f8d500ac530 2023-10-18T09:17:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d5c0cd8d0 con 0x7f8d641067a0 2023-10-18T09:17:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.379+0000 7f8d69169700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] conn(0x7f8d500ac530 0x7f8d500ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.380+0000 7f8d6996a700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8d500a1510 0x7f8d500a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8d64105c10 tx=0x7f8d60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.380+0000 7f8d69169700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] conn(0x7f8d500ac530 0x7f8d500ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:24.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.381+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== osd.3 v2:172.21.15.61:6810/671964640 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8d500af0a0 con 0x7f8d500ac530 2023-10-18T09:17:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:23 smithi170 ceph-mon[135150]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 511 B/s wr, 11 op/s 2023-10-18T09:17:24.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.398+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 --> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d64060ef0 con 0x7f8d500ac530 2023-10-18T09:17:24.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d5affd700 1 -- 172.21.15.61:0/747902167 <== osd.3 v2:172.21.15.61:6810/671964640 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8d64060ef0 con 0x7f8d500ac530 2023-10-18T09:17:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] conn(0x7f8d500ac530 msgr2=0x7f8d500ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] conn(0x7f8d500ac530 0x7f8d500ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8d500a1510 msgr2=0x7f8d500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8d500a1510 0x7f8d500a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8d64105c10 tx=0x7f8d60009260 comp rx=0 tx=0).stop 2023-10-18T09:17:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 msgr2=0x7f8d6419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 --2- 172.21.15.61:0/747902167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8d641067a0 0x7f8d6419b5a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e3c0 tx=0x7f8d5c00e780 comp rx=0 tx=0).stop 2023-10-18T09:17:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6a16b700 1 -- 172.21.15.61:0/747902167 reap_dead start 2023-10-18T09:17:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 shutdown_connections 2023-10-18T09:17:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 >> 172.21.15.61:0/747902167 conn(0x7f8d64075790 msgr2=0x7f8d640fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.399+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 shutdown_connections 2023-10-18T09:17:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:24.400+0000 7f8d6bbce700 1 -- 172.21.15.61:0/747902167 wait complete. 2023-10-18T09:17:24.962 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:24 smithi061 ceph-mon[98573]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:24 smithi110 ceph-mon[134818]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:24 smithi170 ceph-mon[135150]: pgmap v322: 129 pgs: 3 peering, 126 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:25.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 -- 172.21.15.61:0/901801070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 msgr2=0x7f0cc410d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/901801070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc410d630 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00b0a0 tx=0x7f0cbc00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 -- 172.21.15.61:0/901801070 shutdown_connections 2023-10-18T09:17:25.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/901801070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc410d630 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/901801070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 0x7f0cc4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/901801070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cc4100840 0x7f0cc4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 -- 172.21.15.61:0/901801070 >> 172.21.15.61:0/901801070 conn(0x7f0cc4077950 msgr2=0x7f0cc4077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:25.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 -- 172.21.15.61:0/901801070 shutdown_connections 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.659+0000 7f0ccc75b700 1 -- 172.21.15.61:0/901801070 wait complete. 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.660+0000 7f0ccc75b700 1 Processor -- start 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 -- start start 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cc4100840 0x7f0cc4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc4199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:25.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 0x7f0cc419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:25.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419de30 con 0x7f0cc4100840 2023-10-18T09:17:25.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419dfa0 con 0x7f0cc41056a0 2023-10-18T09:17:25.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.661+0000 7f0ccc75b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc419e160 con 0x7f0cc4101200 2023-10-18T09:17:25.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cca4f7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cc4100840 0x7f0cc4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:25.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc4199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:25.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc4199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48238/0 (socket says 172.21.15.61:48238) 2023-10-18T09:17:25.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0ccacf8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 0x7f0cc419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:25.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 -- 172.21.15.61:0/3215046081 learned_addr learned my addr 172.21.15.61:0/3215046081 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:25.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 -- 172.21.15.61:0/3215046081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 msgr2=0x7f0cc419d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 0x7f0cc419d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 -- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cc4100840 msgr2=0x7f0cc4198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0cc4100840 0x7f0cc4198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009d00 con 0x7f0cc4101200 2023-10-18T09:17:25.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0ccacf8700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cc41056a0 0x7f0cc419d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:25.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.662+0000 7f0cc9cf6700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc4199470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9a0 tx=0x7f0cb400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:25.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.663+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4009790 con 0x7f0cc4101200 2023-10-18T09:17:25.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.663+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cb4010430 con 0x7f0cc4101200 2023-10-18T09:17:25.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.663+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc419e440 con 0x7f0cc4101200 2023-10-18T09:17:25.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.663+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4018680 con 0x7f0cc4101200 2023-10-18T09:17:25.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.663+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc41a5ca0 con 0x7f0cc4101200 2023-10-18T09:17:25.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.664+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cc41109d0 con 0x7f0cc4101200 2023-10-18T09:17:25.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.666+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0cb40105a0 con 0x7f0cc4101200 2023-10-18T09:17:25.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cbb7fe700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0cb00a1590 0x7f0cb00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:25.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0cb4014070 con 0x7f0cc4101200 2023-10-18T09:17:25.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cca4f7700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0cb00a1590 0x7f0cb00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:25.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cbb7fe700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] conn(0x7f0cb00ac5b0 0x7f0cb00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cb00af120 con 0x7f0cb00ac5b0 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.669+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cb40cd970 con 0x7f0cc4101200 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.670+0000 7f0cca4f7700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0cb00a1590 0x7f0cb00a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009be0 tx=0x7f0cc0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.670+0000 7f0ccacf8700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] conn(0x7f0cb00ac5b0 0x7f0cb00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.670+0000 7f0ccacf8700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] conn(0x7f0cb00ac5b0 0x7f0cb00aea50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:25.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.671+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== osd.5 v2:172.21.15.170:6808/1107914770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f0cb00af120 con 0x7f0cb00ac5b0 2023-10-18T09:17:25.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.693+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 --> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cc40ff000 con 0x7f0cb00ac5b0 2023-10-18T09:17:25.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.693+0000 7f0cbb7fe700 1 -- 172.21.15.61:0/3215046081 <== osd.5 v2:172.21.15.170:6808/1107914770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0cc40ff000 con 0x7f0cb00ac5b0 2023-10-18T09:17:25.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] conn(0x7f0cb00ac5b0 msgr2=0x7f0cb00aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] conn(0x7f0cb00ac5b0 0x7f0cb00aea50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0cb00a1590 msgr2=0x7f0cb00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0cb00a1590 0x7f0cb00a3a50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009be0 tx=0x7f0cc0009260 comp rx=0 tx=0).stop 2023-10-18T09:17:25.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 msgr2=0x7f0cc4199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 --2- 172.21.15.61:0/3215046081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cc4101200 0x7f0cc4199470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400e9a0 tx=0x7f0cb400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccacf8700 1 -- 172.21.15.61:0/3215046081 reap_dead start 2023-10-18T09:17:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 shutdown_connections 2023-10-18T09:17:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 >> 172.21.15.61:0/3215046081 conn(0x7f0cc4077950 msgr2=0x7f0cc40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:25.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 shutdown_connections 2023-10-18T09:17:25.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:25.694+0000 7f0ccc75b700 1 -- 172.21.15.61:0/3215046081 wait complete. 2023-10-18T09:17:25.857 INFO:teuthology.orchestra.run.smithi061.stdout:253403070501 2023-10-18T09:17:25.857 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-18T09:17:26.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 -- 172.21.15.61:0/3901566871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 msgr2=0x7efd4010d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 --2- 172.21.15.61:0/3901566871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 0x7efd4010d650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd3800b0a0 tx=0x7efd3800b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 -- 172.21.15.61:0/3901566871 shutdown_connections 2023-10-18T09:17:26.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 --2- 172.21.15.61:0/3901566871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 0x7efd4010d650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 --2- 172.21.15.61:0/3901566871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd401057c0 0x7efd40105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 --2- 172.21.15.61:0/3901566871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 -- 172.21.15.61:0/3901566871 >> 172.21.15.61:0/3901566871 conn(0x7efd400779e0 msgr2=0x7efd40077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:26.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.834+0000 7efd47645700 1 -- 172.21.15.61:0/3901566871 shutdown_connections 2023-10-18T09:17:26.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.835+0000 7efd47645700 1 -- 172.21.15.61:0/3901566871 wait complete. 2023-10-18T09:17:26.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 Processor -- start 2023-10-18T09:17:26.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 -- start start 2023-10-18T09:17:26.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 0x7efd40073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd401057c0 0x7efd40073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd401a5100 con 0x7efd40101260 2023-10-18T09:17:26.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd401a5270 con 0x7efd401008a0 2023-10-18T09:17:26.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.836+0000 7efd47645700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd401a5430 con 0x7efd401057c0 2023-10-18T09:17:26.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd44be0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 0x7efd40073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43574/0 (socket says 172.21.15.61:43574) 2023-10-18T09:17:26.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 -- 172.21.15.61:0/3350415455 learned_addr learned my addr 172.21.15.61:0/3350415455 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:26.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd45be2700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd401057c0 0x7efd40073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 -- 172.21.15.61:0/3350415455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd401057c0 msgr2=0x7efd40073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd401057c0 0x7efd40073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 -- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 msgr2=0x7efd40073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efd40101260 0x7efd40073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38009d00 con 0x7efd401008a0 2023-10-18T09:17:26.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.837+0000 7efd453e1700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40072af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e970 tx=0x7efd3c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.838+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c009790 con 0x7efd401008a0 2023-10-18T09:17:26.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.838+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd401a5710 con 0x7efd401008a0 2023-10-18T09:17:26.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.838+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd3c00c7e0 con 0x7efd401008a0 2023-10-18T09:17:26.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.838+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c003c70 con 0x7efd401008a0 2023-10-18T09:17:26.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.838+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd401a5c60 con 0x7efd401008a0 2023-10-18T09:17:26.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.839+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd401109f0 con 0x7efd401008a0 2023-10-18T09:17:26.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.841+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7efd3c00cc50 con 0x7efd401008a0 2023-10-18T09:17:26.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.842+0000 7efd367fc700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efd2c0a1510 0x7efd2c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.842+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efd3c014070 con 0x7efd401008a0 2023-10-18T09:17:26.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd44be0700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efd2c0a1510 0x7efd2c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd367fc700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] conn(0x7efd2c0ac530 0x7efd2c0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd2c0af0a0 con 0x7efd2c0ac530 2023-10-18T09:17:26.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd3c0cd7b0 con 0x7efd401008a0 2023-10-18T09:17:26.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd45be2700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] conn(0x7efd2c0ac530 0x7efd2c0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.843+0000 7efd44be0700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efd2c0a1510 0x7efd2c0a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd40074220 tx=0x7efd300073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.844+0000 7efd45be2700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] conn(0x7efd2c0ac530 0x7efd2c0ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.844+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== osd.0 v2:172.21.15.61:6802/3245715235 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7efd2c0af0a0 con 0x7efd2c0ac530 2023-10-18T09:17:26.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 -- 172.21.15.61:0/2860292203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c1008d0 msgr2=0x7f538c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 --2- 172.21.15.61:0/2860292203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c1008d0 0x7f538c100d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5378009990 tx=0x7f5378009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 -- 172.21.15.61:0/2860292203 shutdown_connections 2023-10-18T09:17:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 --2- 172.21.15.61:0/2860292203 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c101290 0x7f538c10d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 --2- 172.21.15.61:0/2860292203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1067d0 0x7f538c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 --2- 172.21.15.61:0/2860292203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c1008d0 0x7f538c100d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.118+0000 7f53931a0700 1 -- 172.21.15.61:0/2860292203 >> 172.21.15.61:0/2860292203 conn(0x7f538c0778d0 msgr2=0x7f538c077ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:26.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.119+0000 7f53931a0700 1 -- 172.21.15.61:0/2860292203 shutdown_connections 2023-10-18T09:17:26.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.119+0000 7f53931a0700 1 -- 172.21.15.61:0/2860292203 wait complete. 2023-10-18T09:17:26.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 Processor -- start 2023-10-18T09:17:26.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 -- start start 2023-10-18T09:17:26.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 0x7f538c198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c101290 0x7f538c1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c1067d0 0x7f538c19d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19dd60 con 0x7f538c1067d0 2023-10-18T09:17:26.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19ded0 con 0x7f538c101290 2023-10-18T09:17:26.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.120+0000 7f53931a0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c19e090 con 0x7f538c1008d0 2023-10-18T09:17:26.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 0x7f538c198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f539173d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c1067d0 0x7f538c19d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 0x7f538c198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48270/0 (socket says 172.21.15.61:48270) 2023-10-18T09:17:26.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 -- 172.21.15.61:0/2809118733 learned_addr learned my addr 172.21.15.61:0/2809118733 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:26.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f538bfff700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c101290 0x7f538c1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 -- 172.21.15.61:0/2809118733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c101290 msgr2=0x7f538c1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c101290 0x7f538c1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 -- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c1067d0 msgr2=0x7f538c19d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c1067d0 0x7f538c19d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.121+0000 7f5390f3c700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780095f0 con 0x7f538c1008d0 2023-10-18T09:17:26.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f538bfff700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f538c101290 0x7f538c1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:26.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f539173d700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f538c1067d0 0x7f538c19d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:26.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f5390f3c700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 0x7f538c198e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f538000e9a0 tx=0x7f538000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380009790 con 0x7f538c1008d0 2023-10-18T09:17:26.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c19e370 con 0x7f538c1008d0 2023-10-18T09:17:26.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1a5bd0 con 0x7f538c1008d0 2023-10-18T09:17:26.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.122+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5380010430 con 0x7f538c1008d0 2023-10-18T09:17:26.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.123+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380018680 con 0x7f538c1008d0 2023-10-18T09:17:26.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.123+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5370000fc0 con 0x7f538c1008d0 2023-10-18T09:17:26.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.126+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f53800105a0 con 0x7f538c1008d0 2023-10-18T09:17:26.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.127+0000 7f5389ffb700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f537c0a1590 0x7f537c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.127+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5380014070 con 0x7f538c1008d0 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f538bfff700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f537c0a1590 0x7f537c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f5389ffb700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] conn(0x7f537c0ac5a0 0x7f537c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f537c0af110 con 0x7f537c0ac5a0 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53800cdee0 con 0x7f538c1008d0 2023-10-18T09:17:26.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f539173d700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] conn(0x7f537c0ac5a0 0x7f537c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:26.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.128+0000 7f538bfff700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f537c0a1590 0x7f537c0a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f538c19a3c0 tx=0x7f537800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.129+0000 7f539173d700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] conn(0x7f537c0ac5a0 0x7f537c0aea40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:26.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.130+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== osd.6 v2:172.21.15.61:6818/2592494554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f537c0af110 con 0x7f537c0ac5a0 2023-10-18T09:17:26.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.148+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 --> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5370002d40 con 0x7f537c0ac5a0 2023-10-18T09:17:26.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.148+0000 7f5389ffb700 1 -- 172.21.15.61:0/2809118733 <== osd.6 v2:172.21.15.61:6818/2592494554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5370002d40 con 0x7f537c0ac5a0 2023-10-18T09:17:26.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] conn(0x7f537c0ac5a0 msgr2=0x7f537c0aea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] conn(0x7f537c0ac5a0 0x7f537c0aea40 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f537c0a1590 msgr2=0x7f537c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f537c0a1590 0x7f537c0a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f538c19a3c0 tx=0x7f537800b260 comp rx=0 tx=0).stop 2023-10-18T09:17:26.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 msgr2=0x7f538c198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.149+0000 7f53931a0700 1 --2- 172.21.15.61:0/2809118733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f538c1008d0 0x7f538c198e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f538000e9a0 tx=0x7f538000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.150+0000 7f539173d700 1 -- 172.21.15.61:0/2809118733 reap_dead start 2023-10-18T09:17:26.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.150+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 shutdown_connections 2023-10-18T09:17:26.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.150+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 >> 172.21.15.61:0/2809118733 conn(0x7f538c0778d0 msgr2=0x7f538c072d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:26.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.150+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 shutdown_connections 2023-10-18T09:17:26.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.150+0000 7f53931a0700 1 -- 172.21.15.61:0/2809118733 wait complete. 2023-10-18T09:17:26.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.871+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 --> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd40101f30 con 0x7efd2c0ac530 2023-10-18T09:17:26.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.872+0000 7efd367fc700 1 -- 172.21.15.61:0/3350415455 <== osd.0 v2:172.21.15.61:6802/3245715235 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd40101f30 con 0x7efd2c0ac530 2023-10-18T09:17:26.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.872+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] conn(0x7efd2c0ac530 msgr2=0x7efd2c0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.872+0000 7efd47645700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] conn(0x7efd2c0ac530 0x7efd2c0ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efd2c0a1510 msgr2=0x7efd2c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efd2c0a1510 0x7efd2c0a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7efd40074220 tx=0x7efd300073d0 comp rx=0 tx=0).stop 2023-10-18T09:17:26.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 msgr2=0x7efd40072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:26.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 --2- 172.21.15.61:0/3350415455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efd401008a0 0x7efd40072af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e970 tx=0x7efd3c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:26.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd45be2700 1 -- 172.21.15.61:0/3350415455 reap_dead start 2023-10-18T09:17:26.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 shutdown_connections 2023-10-18T09:17:26.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 >> 172.21.15.61:0/3350415455 conn(0x7efd400779e0 msgr2=0x7efd400fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:26.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 shutdown_connections 2023-10-18T09:17:26.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:26.873+0000 7efd47645700 1 -- 172.21.15.61:0/3350415455 wait complete. 2023-10-18T09:17:26.932 INFO:teuthology.orchestra.run.smithi061.stdout:167503724596 2023-10-18T09:17:26.932 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-18T09:17:27.627 INFO:teuthology.orchestra.run.smithi061.stdout:111669149755 2023-10-18T09:17:27.628 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-18T09:17:27.783 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:27 smithi061 ceph-mon[98573]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:27 smithi170 ceph-mon[135150]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:27.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:27 smithi110 ceph-mon[134818]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:28.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 -- 172.21.15.61:0/3985421896 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 msgr2=0x7f26f80685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 --2- 172.21.15.61:0/3985421896 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f80685f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009990 tx=0x7f26e8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 -- 172.21.15.61:0/3985421896 shutdown_connections 2023-10-18T09:17:28.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 --2- 172.21.15.61:0/3985421896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f8068b30 0x7f26f810d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 --2- 172.21.15.61:0/3985421896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 0x7f26f8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 --2- 172.21.15.61:0/3985421896 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f80685f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 -- 172.21.15.61:0/3985421896 >> 172.21.15.61:0/3985421896 conn(0x7f26f8075790 msgr2=0x7f26f8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:28.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 -- 172.21.15.61:0/3985421896 shutdown_connections 2023-10-18T09:17:28.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.146+0000 7f26ffdea700 1 -- 172.21.15.61:0/3985421896 wait complete. 2023-10-18T09:17:28.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.147+0000 7f26ffdea700 1 Processor -- start 2023-10-18T09:17:28.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.147+0000 7f26ffdea700 1 -- start start 2023-10-18T09:17:28.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:28.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f8068b30 0x7f26f81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:28.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 0x7f26f819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:28.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819dd60 con 0x7f26f8068190 2023-10-18T09:17:28.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819ded0 con 0x7f26f81067a0 2023-10-18T09:17:28.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26ffdea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819e090 con 0x7f26f8068b30 2023-10-18T09:17:28.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26fdb86700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:28.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26fdb86700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f8198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:40388/0 (socket says 172.21.15.61:40388) 2023-10-18T09:17:28.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26fdb86700 1 -- 172.21.15.61:0/1022503671 learned_addr learned my addr 172.21.15.61:0/1022503671 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:28.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26fd385700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f8068b30 0x7f26f81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:28.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.148+0000 7f26fe387700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 0x7f26f819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:28.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 -- 172.21.15.61:0/1022503671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f8068b30 msgr2=0x7f26f81993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26f8068b30 0x7f26f81993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 -- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 msgr2=0x7f26f819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 0x7f26f819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80095f0 con 0x7f26f8068190 2023-10-18T09:17:28.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fe387700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26f81067a0 0x7f26f819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:28.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26fdb86700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f8198eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e9a0 tx=0x7f26f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:28.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4009790 con 0x7f26f8068190 2023-10-18T09:17:28.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f819e370 con 0x7f26f8068190 2023-10-18T09:17:28.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26f400cbd0 con 0x7f26f8068190 2023-10-18T09:17:28.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4018680 con 0x7f26f8068190 2023-10-18T09:17:28.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.149+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81a5bd0 con 0x7f26f8068190 2023-10-18T09:17:28.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.150+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26f8110980 con 0x7f26f8068190 2023-10-18T09:17:28.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.152+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f26f400cd40 con 0x7f26f8068190 2023-10-18T09:17:28.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26eeffd700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26e40a1590 0x7f26e40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:28.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26f4014070 con 0x7f26f8068190 2023-10-18T09:17:28.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26fd385700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26e40a1590 0x7f26e40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:28.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26eeffd700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] conn(0x7f26e40ac5b0 0x7f26e40aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:28.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26e40af120 con 0x7f26e40ac5b0 2023-10-18T09:17:28.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.154+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26f40cda80 con 0x7f26f8068190 2023-10-18T09:17:28.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.155+0000 7f26fe387700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] conn(0x7f26e40ac5b0 0x7f26e40aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:28.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.155+0000 7f26fd385700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26e40a1590 0x7f26e40a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26f819a3c0 tx=0x7f26e800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:28.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.155+0000 7f26fe387700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] conn(0x7f26e40ac5b0 0x7f26e40aea50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:28.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.156+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== osd.4 v2:172.21.15.110:6808/113917828 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f26e40af120 con 0x7f26e40ac5b0 2023-10-18T09:17:28.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.174+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 --> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26f8101ee0 con 0x7f26e40ac5b0 2023-10-18T09:17:28.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.175+0000 7f26eeffd700 1 -- 172.21.15.61:0/1022503671 <== osd.4 v2:172.21.15.110:6808/113917828 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f26f8101ee0 con 0x7f26e40ac5b0 2023-10-18T09:17:28.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.175+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] conn(0x7f26e40ac5b0 msgr2=0x7f26e40aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.175+0000 7f26ffdea700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] conn(0x7f26e40ac5b0 0x7f26e40aea50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.175+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26e40a1590 msgr2=0x7f26e40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.175+0000 7f26ffdea700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26e40a1590 0x7f26e40a3a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26f819a3c0 tx=0x7f26e800b260 comp rx=0 tx=0).stop 2023-10-18T09:17:28.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 msgr2=0x7f26f8198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:28.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 --2- 172.21.15.61:0/1022503671 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26f8068190 0x7f26f8198eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e9a0 tx=0x7f26f400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:28.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26fe387700 1 -- 172.21.15.61:0/1022503671 reap_dead start 2023-10-18T09:17:28.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 shutdown_connections 2023-10-18T09:17:28.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 >> 172.21.15.61:0/1022503671 conn(0x7f26f8075790 msgr2=0x7f26f80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:28.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 shutdown_connections 2023-10-18T09:17:28.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:28.176+0000 7f26ffdea700 1 -- 172.21.15.61:0/1022503671 wait complete. 2023-10-18T09:17:28.197 INFO:teuthology.orchestra.run.smithi061.stdout:201863462959 2023-10-18T09:17:28.198 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-18T09:17:28.562 INFO:teuthology.orchestra.run.smithi061.stdout:287762808864 2023-10-18T09:17:28.562 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-18T09:17:29.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 -- 172.21.15.61:0/1203684657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 msgr2=0x7f884c10d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 --2- 172.21.15.61:0/1203684657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c10d630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8844009f90 tx=0x7f884400b330 comp rx=0 tx=0).stop 2023-10-18T09:17:29.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 -- 172.21.15.61:0/1203684657 shutdown_connections 2023-10-18T09:17:29.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 --2- 172.21.15.61:0/1203684657 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c10d630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 --2- 172.21.15.61:0/1203684657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c1056a0 0x7f884c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 --2- 172.21.15.61:0/1203684657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 0x7f884c100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 -- 172.21.15.61:0/1203684657 >> 172.21.15.61:0/1203684657 conn(0x7f884c077950 msgr2=0x7f884c077d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 -- 172.21.15.61:0/1203684657 shutdown_connections 2023-10-18T09:17:29.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.323+0000 7f8851a79700 1 -- 172.21.15.61:0/1203684657 wait complete. 2023-10-18T09:17:29.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.325+0000 7f8851a79700 1 Processor -- start 2023-10-18T09:17:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.325+0000 7f8851a79700 1 -- start start 2023-10-18T09:17:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 0x7f884c198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c1056a0 0x7f884c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19deb0 con 0x7f884c101200 2023-10-18T09:17:29.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19e020 con 0x7f884c1056a0 2023-10-18T09:17:29.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f8851a79700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19e1e0 con 0x7f884c100840 2023-10-18T09:17:29.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c1994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37072/0 (socket says 172.21.15.61:37072) 2023-10-18T09:17:29.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 0x7f884c198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884affd700 1 -- 172.21.15.61:0/3210978856 learned_addr learned my addr 172.21.15.61:0/3210978856 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884bfff700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c1056a0 0x7f884c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 -- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c1056a0 msgr2=0x7f884c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f884c1056a0 0x7f884c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 -- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 msgr2=0x7f884c1994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f884c101200 0x7f884c1994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.326+0000 7f884b7fe700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400095f0 con 0x7f884c100840 2023-10-18T09:17:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f884b7fe700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 0x7f884c198fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00c9c0 con 0x7f884c100840 2023-10-18T09:17:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8844009c40 con 0x7f884c100840 2023-10-18T09:17:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883c010430 con 0x7f884c100840 2023-10-18T09:17:29.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c018680 con 0x7f884c100840 2023-10-18T09:17:29.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.327+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c1a5b50 con 0x7f884c100840 2023-10-18T09:17:29.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.328+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f884c1109c0 con 0x7f884c100840 2023-10-18T09:17:29.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.332+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f883c010860 con 0x7f884c100840 2023-10-18T09:17:29.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f8848ff9700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88340a1510 0x7f88340a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f883c014070 con 0x7f884c100840 2023-10-18T09:17:29.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f8848ff9700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] conn(0x7f88340ac530 0x7f88340ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88340af0a0 con 0x7f88340ac530 2023-10-18T09:17:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883c0cd8a0 con 0x7f884c100840 2023-10-18T09:17:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.335+0000 7f884affd700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88340a1510 0x7f88340a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.336+0000 7f884bfff700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] conn(0x7f88340ac530 0x7f88340ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.337+0000 7f884affd700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88340a1510 0x7f88340a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f884c19a510 tx=0x7f8840009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.337+0000 7f884bfff700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] conn(0x7f88340ac530 0x7f88340ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.338+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== osd.1 v2:172.21.15.110:6800/4194494003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f88340af0a0 con 0x7f88340ac530 2023-10-18T09:17:29.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.356+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 --> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f884c101ec0 con 0x7f88340ac530 2023-10-18T09:17:29.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8848ff9700 1 -- 172.21.15.61:0/3210978856 <== osd.1 v2:172.21.15.110:6800/4194494003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f884c101ec0 con 0x7f88340ac530 2023-10-18T09:17:29.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] conn(0x7f88340ac530 msgr2=0x7f88340ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] conn(0x7f88340ac530 0x7f88340ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88340a1510 msgr2=0x7f88340a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88340a1510 0x7f88340a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f884c19a510 tx=0x7f8840009310 comp rx=0 tx=0).stop 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 msgr2=0x7f884c198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f8851a79700 1 --2- 172.21.15.61:0/3210978856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f884c100840 0x7f884c198fb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e9f0 tx=0x7f883c00ed00 comp rx=0 tx=0).stop 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.357+0000 7f884bfff700 1 -- 172.21.15.61:0/3210978856 reap_dead start 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.358+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 shutdown_connections 2023-10-18T09:17:29.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.358+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 >> 172.21.15.61:0/3210978856 conn(0x7f884c077950 msgr2=0x7f884c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.358+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 shutdown_connections 2023-10-18T09:17:29.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.358+0000 7f8851a79700 1 -- 172.21.15.61:0/3210978856 wait complete. 2023-10-18T09:17:29.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 -- 172.21.15.61:0/982274844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 msgr2=0x7f8c48068570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/982274844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48068570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009990 tx=0x7f8c38009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 -- 172.21.15.61:0/982274844 shutdown_connections 2023-10-18T09:17:29.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/982274844 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c4810d300 0x7f8c4810f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/982274844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48068570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/982274844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c48068ab0 0x7f8c4810cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 -- 172.21.15.61:0/982274844 >> 172.21.15.61:0/982274844 conn(0x7f8c48075790 msgr2=0x7f8c48075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.415+0000 7f8c4dd88700 1 -- 172.21.15.61:0/982274844 shutdown_connections 2023-10-18T09:17:29.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.416+0000 7f8c4dd88700 1 -- 172.21.15.61:0/982274844 wait complete. 2023-10-18T09:17:29.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 Processor -- start 2023-10-18T09:17:29.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 -- start start 2023-10-18T09:17:29.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 0x7f8c481993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4810d300 0x7f8c4819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4819dd90 con 0x7f8c48068ab0 2023-10-18T09:17:29.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4819df00 con 0x7f8c48068190 2023-10-18T09:17:29.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.417+0000 7f8c4dd88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4819e0c0 con 0x7f8c4810d300 2023-10-18T09:17:29.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 0x7f8c481993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c477fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 0x7f8c481993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37100/0 (socket says 172.21.15.61:37100) 2023-10-18T09:17:29.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 -- 172.21.15.61:0/1199601906 learned_addr learned my addr 172.21.15.61:0/1199601906 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c47fff700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4810d300 0x7f8c4819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 -- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4810d300 msgr2=0x7f8c4819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4810d300 0x7f8c4819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 -- 172.21.15.61:0/1199601906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 msgr2=0x7f8c48198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380095f0 con 0x7f8c48068ab0 2023-10-18T09:17:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c47fff700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c4810d300 0x7f8c4819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c477fe700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c48068190 0x7f8c48198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.418+0000 7f8c46ffd700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 0x7f8c481993d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e9c0 tx=0x7f8c3c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.419+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0097b0 con 0x7f8c48068ab0 2023-10-18T09:17:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.419+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3c00ee60 con 0x7f8c48068ab0 2023-10-18T09:17:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.419+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0054f0 con 0x7f8c48068ab0 2023-10-18T09:17:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.419+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4819e3a0 con 0x7f8c48068ab0 2023-10-18T09:17:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.419+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c481a5c00 con 0x7f8c48068ab0 2023-10-18T09:17:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.420+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8c48112c20 con 0x7f8c48068ab0 2023-10-18T09:17:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.422+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8c3c004750 con 0x7f8c48068ab0 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.424+0000 7f8c44ff9700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c300a1590 0x7f8c300a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.424+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c3c014070 con 0x7f8c48068ab0 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.424+0000 7f8c477fe700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c300a1590 0x7f8c300a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.425+0000 7f8c44ff9700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] conn(0x7f8c300ac5b0 0x7f8c300aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.425+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8c300af120 con 0x7f8c300ac5b0 2023-10-18T09:17:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.425+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c3c0cd970 con 0x7f8c48068ab0 2023-10-18T09:17:29.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.425+0000 7f8c47fff700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] conn(0x7f8c300ac5b0 0x7f8c300aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.425+0000 7f8c477fe700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c300a1590 0x7f8c300a3a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006010 tx=0x7f8c3800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.426+0000 7f8c47fff700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] conn(0x7f8c300ac5b0 0x7f8c300aea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.427+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== osd.11 v2:172.21.15.170:6824/4083010574 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f8c300af120 con 0x7f8c300ac5b0 2023-10-18T09:17:29.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.459+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 --> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8c48060ef0 con 0x7f8c300ac5b0 2023-10-18T09:17:29.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.460+0000 7f8c44ff9700 1 -- 172.21.15.61:0/1199601906 <== osd.11 v2:172.21.15.170:6824/4083010574 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8c48060ef0 con 0x7f8c300ac5b0 2023-10-18T09:17:29.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] conn(0x7f8c300ac5b0 msgr2=0x7f8c300aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] conn(0x7f8c300ac5b0 0x7f8c300aea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c300a1590 msgr2=0x7f8c300a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c300a1590 0x7f8c300a3a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006010 tx=0x7f8c3800b260 comp rx=0 tx=0).stop 2023-10-18T09:17:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 msgr2=0x7f8c481993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 --2- 172.21.15.61:0/1199601906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c48068ab0 0x7f8c481993d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00e9c0 tx=0x7f8c3c00ecd0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c47fff700 1 -- 172.21.15.61:0/1199601906 reap_dead start 2023-10-18T09:17:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 shutdown_connections 2023-10-18T09:17:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.461+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 >> 172.21.15.61:0/1199601906 conn(0x7f8c48075790 msgr2=0x7f8c4810e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.462+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 shutdown_connections 2023-10-18T09:17:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.462+0000 7f8c4dd88700 1 -- 172.21.15.61:0/1199601906 wait complete. 2023-10-18T09:17:29.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 -- 172.21.15.61:0/2531480356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c1057c0 msgr2=0x7fca2c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 --2- 172.21.15.61:0/2531480356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c1057c0 0x7fca2c105ba0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca28009990 tx=0x7fca28009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 -- 172.21.15.61:0/2531480356 shutdown_connections 2023-10-18T09:17:29.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 --2- 172.21.15.61:0/2531480356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c101260 0x7fca2c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 --2- 172.21.15.61:0/2531480356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c1057c0 0x7fca2c105ba0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 --2- 172.21.15.61:0/2531480356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1008a0 0x7fca2c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.001+0000 7fca337c3700 1 -- 172.21.15.61:0/2531480356 >> 172.21.15.61:0/2531480356 conn(0x7fca2c0779e0 msgr2=0x7fca2c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.002+0000 7fca337c3700 1 -- 172.21.15.61:0/2531480356 shutdown_connections 2023-10-18T09:17:29.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.002+0000 7fca337c3700 1 -- 172.21.15.61:0/2531480356 wait complete. 2023-10-18T09:17:29.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 Processor -- start 2023-10-18T09:17:29.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 -- start start 2023-10-18T09:17:29.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c1008a0 0x7fca2c072af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 0x7fca2c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1057c0 0x7fca2c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c074a60 con 0x7fca2c101260 2023-10-18T09:17:29.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1a9660 con 0x7fca2c1008a0 2023-10-18T09:17:29.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.003+0000 7fca337c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1a97d0 con 0x7fca2c1057c0 2023-10-18T09:17:29.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 0x7fca2c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 0x7fca2c073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37044/0 (socket says 172.21.15.61:37044) 2023-10-18T09:17:29.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 -- 172.21.15.61:0/3506177737 learned_addr learned my addr 172.21.15.61:0/3506177737 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca3155f700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c1008a0 0x7fca2c072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca31d60700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1057c0 0x7fca2c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 -- 172.21.15.61:0/3506177737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1057c0 msgr2=0x7fca2c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1057c0 0x7fca2c073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 -- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c1008a0 msgr2=0x7fca2c072af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c1008a0 0x7fca2c072af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca30d5e700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280095f0 con 0x7fca2c101260 2023-10-18T09:17:29.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca3155f700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca2c1008a0 0x7fca2c072af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:29.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.004+0000 7fca31d60700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2c1057c0 0x7fca2c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:29.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca30d5e700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 0x7fca2c073030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e9a0 tx=0x7fca1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c009790 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c1a9ab0 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1c00ce70 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c018420 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.005+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c1aa000 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.006+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca2c1109e0 con 0x7fca2c101260 2023-10-18T09:17:29.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.008+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fca1c0106f0 con 0x7fca2c101260 2023-10-18T09:17:29.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca227fc700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca180a1510 0x7fca180a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fca1c014070 con 0x7fca2c101260 2023-10-18T09:17:29.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca3155f700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca180a1510 0x7fca180a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca227fc700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] conn(0x7fca180ac530 0x7fca180ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:29.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca180af0a0 con 0x7fca180ac530 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.010+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1c0cdd70 con 0x7fca2c101260 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.011+0000 7fca31d60700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] conn(0x7fca180ac530 0x7fca180ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.011+0000 7fca3155f700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca180a1510 0x7fca180a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca28006010 tx=0x7fca2800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.011+0000 7fca31d60700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] conn(0x7fca180ac530 0x7fca180ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.012+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== osd.10 v2:172.21.15.110:6824/508893070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fca180af0a0 con 0x7fca180ac530 2023-10-18T09:17:29.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.029+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 --> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca2c060e70 con 0x7fca180ac530 2023-10-18T09:17:29.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.030+0000 7fca227fc700 1 -- 172.21.15.61:0/3506177737 <== osd.10 v2:172.21.15.110:6824/508893070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fca2c060e70 con 0x7fca180ac530 2023-10-18T09:17:29.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.030+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] conn(0x7fca180ac530 msgr2=0x7fca180ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.030+0000 7fca337c3700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] conn(0x7fca180ac530 0x7fca180ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca180a1510 msgr2=0x7fca180a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fca180a1510 0x7fca180a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca28006010 tx=0x7fca2800b260 comp rx=0 tx=0).stop 2023-10-18T09:17:29.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 msgr2=0x7fca2c073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:29.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 --2- 172.21.15.61:0/3506177737 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fca2c101260 0x7fca2c073030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e9a0 tx=0x7fca1c00ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:29.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca31d60700 1 -- 172.21.15.61:0/3506177737 reap_dead start 2023-10-18T09:17:29.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 shutdown_connections 2023-10-18T09:17:29.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 >> 172.21.15.61:0/3506177737 conn(0x7fca2c0779e0 msgr2=0x7fca2c0fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:29.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 shutdown_connections 2023-10-18T09:17:29.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:29.031+0000 7fca337c3700 1 -- 172.21.15.61:0/3506177737 wait complete. 2023-10-18T09:17:29.979 INFO:teuthology.orchestra.run.smithi061.stdout:34359738443 2023-10-18T09:17:29.980 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-18T09:17:30.036 INFO:teuthology.orchestra.run.smithi061.stdout:146028888119 2023-10-18T09:17:30.037 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-18T09:17:30.060 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:29 smithi061 ceph-mon[98573]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:29 smithi110 ceph-mon[134818]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:30.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:29 smithi170 ceph-mon[135150]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 11 op/s; 23 B/s, 2 keys/s, 1 objects/s recovering 2023-10-18T09:17:30.675 INFO:teuthology.orchestra.run.smithi061.stdout:339302416409 2023-10-18T09:17:30.675 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-18T09:17:30.676 INFO:teuthology.orchestra.run.smithi061.stdout:55834574919 2023-10-18T09:17:30.676 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-18T09:17:30.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/3038809716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc111570 msgr2=0x7fd1cc1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/3038809716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc111570 0x7fd1cc1139a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b0a0 tx=0x7fd1c400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/3038809716 shutdown_connections 2023-10-18T09:17:30.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/3038809716 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc111570 0x7fd1cc1139a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/3038809716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1cc074770 0x7fd1cc072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/3038809716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc072f60 0x7fd1cc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.925+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/3038809716 >> 172.21.15.61:0/3038809716 conn(0x7fd1cc0779e0 msgr2=0x7fd1cc077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:30.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.926+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/3038809716 shutdown_connections 2023-10-18T09:17:30.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.926+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/3038809716 wait complete. 2023-10-18T09:17:30.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.927+0000 7fd1d0ef7700 1 Processor -- start 2023-10-18T09:17:30.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.927+0000 7fd1d0ef7700 1 -- start start 2023-10-18T09:17:30.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1cc072f60 0x7fd1cc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:30.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc074770 0x7fd1cc19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:30.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 0x7fd1cc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:30.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1a22a0 con 0x7fd1cc074770 2023-10-18T09:17:30.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1a2410 con 0x7fd1cc072f60 2023-10-18T09:17:30.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.928+0000 7fd1d0ef7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1a25d0 con 0x7fd1cc111570 2023-10-18T09:17:30.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 0x7fd1cc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:30.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1cc072f60 0x7fd1cc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:30.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 0x7fd1cc1a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:33146/0 (socket says 172.21.15.61:33146) 2023-10-18T09:17:30.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 -- 172.21.15.61:0/2605966235 learned_addr learned my addr 172.21.15.61:0/2605966235 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:30.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1c9d9b700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc074770 0x7fd1cc19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:30.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 -- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1cc072f60 msgr2=0x7fd1cc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1cc072f60 0x7fd1cc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 -- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc074770 msgr2=0x7fd1cc19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd1cc074770 0x7fd1cc19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.929+0000 7fd1cad9d700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4009d00 con 0x7fd1cc111570 2023-10-18T09:17:30.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.930+0000 7fd1cad9d700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 0x7fd1cc1a1c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b070 tx=0x7fd1c40078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:30.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.931+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400e040 con 0x7fd1cc111570 2023-10-18T09:17:30.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.931+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c401d470 con 0x7fd1cc111570 2023-10-18T09:17:30.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.931+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40052a0 con 0x7fd1cc111570 2023-10-18T09:17:30.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.931+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc1a2850 con 0x7fd1cc111570 2023-10-18T09:17:30.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.931+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc1aa110 con 0x7fd1cc111570 2023-10-18T09:17:30.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.933+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1cc0740b0 con 0x7fd1cc111570 2023-10-18T09:17:30.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.938+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fd1c401d5e0 con 0x7fd1cc111570 2023-10-18T09:17:30.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1c37fe700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd1b80a1590 0x7fd1b80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:30.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1c40ce920 con 0x7fd1cc111570 2023-10-18T09:17:30.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1ca59c700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd1b80a1590 0x7fd1b80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:30.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1c37fe700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] conn(0x7fd1b80ac5b0 0x7fd1b80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:30.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1b80af120 con 0x7fd1b80ac5b0 2023-10-18T09:17:30.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.941+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c40ced30 con 0x7fd1cc111570 2023-10-18T09:17:30.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.942+0000 7fd1c9d9b700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] conn(0x7fd1b80ac5b0 0x7fd1b80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:30.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.943+0000 7fd1ca59c700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd1b80a1590 0x7fd1b80a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005950 tx=0x7fd1b400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:30.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.943+0000 7fd1c9d9b700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] conn(0x7fd1b80ac5b0 0x7fd1b80aea50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:30.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.944+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== osd.7 v2:172.21.15.110:6816/3683648719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd1b80af120 con 0x7fd1b80ac5b0 2023-10-18T09:17:30.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.979+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 --> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1cc117040 con 0x7fd1b80ac5b0 2023-10-18T09:17:30.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.980+0000 7fd1c37fe700 1 -- 172.21.15.61:0/2605966235 <== osd.7 v2:172.21.15.110:6816/3683648719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd1cc117040 con 0x7fd1b80ac5b0 2023-10-18T09:17:30.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] conn(0x7fd1b80ac5b0 msgr2=0x7fd1b80aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] conn(0x7fd1b80ac5b0 0x7fd1b80aea50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd1b80a1590 msgr2=0x7fd1b80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd1b80a1590 0x7fd1b80a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4005950 tx=0x7fd1b400a300 comp rx=0 tx=0).stop 2023-10-18T09:17:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 msgr2=0x7fd1cc1a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.981+0000 7fd1d0ef7700 1 --2- 172.21.15.61:0/2605966235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1cc111570 0x7fd1cc1a1c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400b070 tx=0x7fd1c40078e0 comp rx=0 tx=0).stop 2023-10-18T09:17:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.982+0000 7fd1cad9d700 1 -- 172.21.15.61:0/2605966235 reap_dead start 2023-10-18T09:17:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.982+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 shutdown_connections 2023-10-18T09:17:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.982+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 >> 172.21.15.61:0/2605966235 conn(0x7fd1cc0779e0 msgr2=0x7fd1cc1101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.982+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 shutdown_connections 2023-10-18T09:17:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:30.982+0000 7fd1d0ef7700 1 -- 172.21.15.61:0/2605966235 wait complete. 2023-10-18T09:17:31.088 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:30 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:17:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:30 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:17:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:30 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:17:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 -- 172.21.15.61:0/2159665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644072f60 msgr2=0x7f46440733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 --2- 172.21.15.61:0/2159665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644072f60 0x7f46440733e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4634009990 tx=0x7f4634009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 -- 172.21.15.61:0/2159665098 shutdown_connections 2023-10-18T09:17:31.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 --2- 172.21.15.61:0/2159665098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644111570 0x7f46441139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 --2- 172.21.15.61:0/2159665098 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644074770 0x7f4644072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 --2- 172.21.15.61:0/2159665098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644072f60 0x7f46440733e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 -- 172.21.15.61:0/2159665098 >> 172.21.15.61:0/2159665098 conn(0x7f46440779e0 msgr2=0x7f4644077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:31.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.544+0000 7f464b744700 1 -- 172.21.15.61:0/2159665098 shutdown_connections 2023-10-18T09:17:31.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.545+0000 7f464b744700 1 -- 172.21.15.61:0/2159665098 wait complete. 2023-10-18T09:17:31.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.546+0000 7f464b744700 1 Processor -- start 2023-10-18T09:17:31.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.546+0000 7f464b744700 1 -- start start 2023-10-18T09:17:31.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 0x7f464419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:31.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644074770 0x7f464419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:31.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644111570 0x7f46441a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:31.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441a21d0 con 0x7f4644072f60 2023-10-18T09:17:31.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441a2340 con 0x7f4644074770 2023-10-18T09:17:31.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f464b744700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46441a2500 con 0x7f4644111570 2023-10-18T09:17:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f46494e0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 0x7f464419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f46494e0700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 0x7f464419d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37126/0 (socket says 172.21.15.61:37126) 2023-10-18T09:17:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f46494e0700 1 -- 172.21.15.61:0/4119777209 learned_addr learned my addr 172.21.15.61:0/4119777209 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.547+0000 7f4649ce1700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644111570 0x7f46441a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 -- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644111570 msgr2=0x7f46441a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4644111570 0x7f46441a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 -- 172.21.15.61:0/4119777209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644074770 msgr2=0x7f464419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4644074770 0x7f464419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340095f0 con 0x7f4644072f60 2023-10-18T09:17:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f46494e0700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 0x7f464419d320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f464000e9a0 tx=0x7f464000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640009790 con 0x7f4644072f60 2023-10-18T09:17:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46441a27e0 con 0x7f4644072f60 2023-10-18T09:17:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f464000cbd0 con 0x7f4644072f60 2023-10-18T09:17:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640018680 con 0x7f4644072f60 2023-10-18T09:17:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.548+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46441aa040 con 0x7f4644072f60 2023-10-18T09:17:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.549+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4644117110 con 0x7f4644072f60 2023-10-18T09:17:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.552+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f464000cd40 con 0x7f4644072f60 2023-10-18T09:17:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.553+0000 7f463a7fc700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46300a1510 0x7f46300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.553+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4640014070 con 0x7f4644072f60 2023-10-18T09:17:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f4648cdf700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46300a1510 0x7f46300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f463a7fc700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] conn(0x7f46300ac530 0x7f46300ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46300af0a0 con 0x7f46300ac530 2023-10-18T09:17:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46400cdb90 con 0x7f4644072f60 2023-10-18T09:17:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f4649ce1700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] conn(0x7f46300ac530 0x7f46300ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.554+0000 7f4648cdf700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46300a1510 0x7f46300a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46340098e0 tx=0x7f4634005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.555+0000 7f4649ce1700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] conn(0x7f46300ac530 0x7f46300ae9d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.556+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== osd.2 v2:172.21.15.170:6800/696261473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f46300af0a0 con 0x7f46300ac530 2023-10-18T09:17:31.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.574+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 --> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4644110b30 con 0x7f46300ac530 2023-10-18T09:17:31.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.574+0000 7f463a7fc700 1 -- 172.21.15.61:0/4119777209 <== osd.2 v2:172.21.15.170:6800/696261473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4644110b30 con 0x7f46300ac530 2023-10-18T09:17:31.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.574+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] conn(0x7f46300ac530 msgr2=0x7f46300ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.574+0000 7f464b744700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] conn(0x7f46300ac530 0x7f46300ae9d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46300a1510 msgr2=0x7f46300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46300a1510 0x7f46300a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f46340098e0 tx=0x7f4634005fb0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 msgr2=0x7f464419d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:31.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 --2- 172.21.15.61:0/4119777209 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4644072f60 0x7f464419d320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f464000e9a0 tx=0x7f464000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:31.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f4649ce1700 1 -- 172.21.15.61:0/4119777209 reap_dead start 2023-10-18T09:17:31.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 shutdown_connections 2023-10-18T09:17:31.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 >> 172.21.15.61:0/4119777209 conn(0x7f46440779e0 msgr2=0x7f46441101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:31.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 shutdown_connections 2023-10-18T09:17:31.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:31.575+0000 7f464b744700 1 -- 172.21.15.61:0/4119777209 wait complete. 2023-10-18T09:17:31.681 INFO:teuthology.orchestra.run.smithi061.stdout:309237645341 2023-10-18T09:17:31.682 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-18T09:17:31.921 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:31.997 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:32.026 INFO:teuthology.orchestra.run.smithi061.stdout:223338299434 2023-10-18T09:17:32.027 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-18T09:17:32.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:31 smithi110 ceph-mon[134818]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 309 B/s wr, 11 op/s; 19 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:32.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:31 smithi061 ceph-mon[98573]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 309 B/s wr, 11 op/s; 19 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:32.380 INFO:teuthology.orchestra.run.smithi061.stdout:77309411396 2023-10-18T09:17:32.381 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-18T09:17:32.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:31 smithi170 ceph-mon[135150]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 309 B/s wr, 11 op/s; 19 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:32.462 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:32.519 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:33.320 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:33.734 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:33.735 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:33.838 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:33.838 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:33.838 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:33.839 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:33.839 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:33 smithi061 ceph-mon[98573]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 280 B/s wr, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:33 smithi170 ceph-mon[135150]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 280 B/s wr, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:34.211 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:34.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:33 smithi110 ceph-mon[134818]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 280 B/s wr, 9 op/s; 17 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:35.419 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:35.817 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:36.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:35 smithi061 ceph-mon[98573]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 15 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:35 smithi110 ceph-mon[134818]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 15 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:36.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 -- 172.21.15.61:0/3806008752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db41057c0 msgr2=0x7f0db4105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:36.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/3806008752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db41057c0 0x7f0db4105ba0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009990 tx=0x7f0da4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 -- 172.21.15.61:0/3806008752 shutdown_connections 2023-10-18T09:17:36.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/3806008752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db4101260 0x7f0db410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/3806008752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db41057c0 0x7f0db4105ba0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/3806008752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41008a0 0x7f0db4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 -- 172.21.15.61:0/3806008752 >> 172.21.15.61:0/3806008752 conn(0x7f0db40779e0 msgr2=0x7f0db4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:36.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 -- 172.21.15.61:0/3806008752 shutdown_connections 2023-10-18T09:17:36.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.103+0000 7f0dbaa44700 1 -- 172.21.15.61:0/3806008752 wait complete. 2023-10-18T09:17:36.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.105+0000 7f0dbaa44700 1 Processor -- start 2023-10-18T09:17:36.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.105+0000 7f0dbaa44700 1 -- start start 2023-10-18T09:17:36.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.105+0000 7f0dbaa44700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 0x7f0db419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:36.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.105+0000 7f0dbaa44700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0dbaa44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 0x7f0db41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0dbaa44700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4197050 con 0x7f0db4101260 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0dbaa44700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db41971c0 con 0x7f0db41008a0 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0dbaa44700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db4197380 con 0x7f0db41057c0 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:36.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 0x7f0db419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:36.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37138/0 (socket says 172.21.15.61:37138) 2023-10-18T09:17:36.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 0x7f0db419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47902/0 (socket says 172.21.15.61:47902) 2023-10-18T09:17:36.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 -- 172.21.15.61:0/2659677700 learned_addr learned my addr 172.21.15.61:0/2659677700 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:36.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db8fe1700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 0x7f0db41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:36.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 -- 172.21.15.61:0/2659677700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 msgr2=0x7f0db41969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:36.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 0x7f0db41969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 -- 172.21.15.61:0/2659677700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 msgr2=0x7f0db419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:36.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 0x7f0db419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 -- 172.21.15.61:0/2659677700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40095f0 con 0x7f0db4101260 2023-10-18T09:17:36.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db8fe1700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 0x7f0db41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:36.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.106+0000 7f0db37fe700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e9a0 tx=0x7f0da800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:36.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.107+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8009790 con 0x7f0db4101260 2023-10-18T09:17:36.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.107+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0da800cbd0 con 0x7f0db4101260 2023-10-18T09:17:36.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.107+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db4197660 con 0x7f0db4101260 2023-10-18T09:17:36.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.107+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8018680 con 0x7f0db4101260 2023-10-18T09:17:36.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.107+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db41a38a0 con 0x7f0db4101260 2023-10-18T09:17:36.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.108+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db4108d80 con 0x7f0db4101260 2023-10-18T09:17:36.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.110+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f0da800cd40 con 0x7f0db4101260 2023-10-18T09:17:36.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.111+0000 7f0db17fa700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 0x7f0d9c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:36.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.111+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0da8014070 con 0x7f0db4101260 2023-10-18T09:17:36.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.111+0000 7f0db3fff700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 0x7f0d9c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:36.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.112+0000 7f0db3fff700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 0x7f0d9c0a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0da4006010 tx=0x7f0da4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:36.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.116+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0da8052100 con 0x7f0db4101260 2023-10-18T09:17:36.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.379+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f0db4101f30 con 0x7f0db4101260 2023-10-18T09:17:36.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.379+0000 7f0db17fa700 1 -- 172.21.15.61:0/2659677700 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0da8096630 con 0x7f0db4101260 2023-10-18T09:17:36.382 INFO:teuthology.orchestra.run.smithi061.stdout:287762808866 2023-10-18T09:17:36.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 msgr2=0x7f0d9c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:36.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 0x7f0d9c0a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0da4006010 tx=0x7f0da4005960 comp rx=0 tx=0).stop 2023-10-18T09:17:36.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 msgr2=0x7f0db419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:36.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e9a0 tx=0x7f0da800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 shutdown_connections 2023-10-18T09:17:36.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0d9c0a1510 0x7f0d9c0a39d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0db41057c0 0x7f0db41969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db41008a0 0x7f0db419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 --2- 172.21.15.61:0/2659677700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0db4101260 0x7f0db419cdf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:36.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 >> 172.21.15.61:0/2659677700 conn(0x7f0db40779e0 msgr2=0x7f0db40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:36.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 shutdown_connections 2023-10-18T09:17:36.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:36.382+0000 7f0dbaa44700 1 -- 172.21.15.61:0/2659677700 wait complete. 2023-10-18T09:17:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:35 smithi170 ceph-mon[135150]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s; 15 B/s, 1 keys/s, 1 objects/s recovering 2023-10-18T09:17:36.921 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:36 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2659677700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T09:17:37.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:36 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2659677700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T09:17:37.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 -- 172.21.15.61:0/4160803315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 msgr2=0x7f758c103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 --2- 172.21.15.61:0/4160803315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c103700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f758400b0a0 tx=0x7f758400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 -- 172.21.15.61:0/4160803315 shutdown_connections 2023-10-18T09:17:37.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 --2- 172.21.15.61:0/4160803315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c103700 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 --2- 172.21.15.61:0/4160803315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 --2- 172.21.15.61:0/4160803315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c068b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 -- 172.21.15.61:0/4160803315 >> 172.21.15.61:0/4160803315 conn(0x7f758c075790 msgr2=0x7f758c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 -- 172.21.15.61:0/4160803315 shutdown_connections 2023-10-18T09:17:37.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.227+0000 7f7591674700 1 -- 172.21.15.61:0/4160803315 wait complete. 2023-10-18T09:17:37.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.228+0000 7f7591674700 1 Processor -- start 2023-10-18T09:17:37.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.228+0000 7f7591674700 1 -- start start 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a2010 con 0x7f758c103250 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a2180 con 0x7f758c1056f0 2023-10-18T09:17:37.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f7591674700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1a2340 con 0x7f758c0686d0 2023-10-18T09:17:37.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f758b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f758affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f758affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:33198/0 (socket says 172.21.15.61:33198) 2023-10-18T09:17:37.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.229+0000 7f758affd700 1 -- 172.21.15.61:0/710929134 learned_addr learned my addr 172.21.15.61:0/710929134 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:37.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758a7fc700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 -- 172.21.15.61:0/710929134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 msgr2=0x7f758c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 -- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 msgr2=0x7f758c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 -- 172.21.15.61:0/710929134 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009d00 con 0x7f758c0686d0 2023-10-18T09:17:37.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758a7fc700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:37.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758b7fe700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:37.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f758affd700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9a0 tx=0x7f757c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.230+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c009790 con 0x7f758c0686d0 2023-10-18T09:17:37.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.231+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c1a2620 con 0x7f758c0686d0 2023-10-18T09:17:37.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.231+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f757c00cbd0 con 0x7f758c0686d0 2023-10-18T09:17:37.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.231+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c1a9e80 con 0x7f758c0686d0 2023-10-18T09:17:37.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.231+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c018680 con 0x7f758c0686d0 2023-10-18T09:17:37.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.233+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c0ffd30 con 0x7f758c0686d0 2023-10-18T09:17:37.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.236+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f757c00cd40 con 0x7f758c0686d0 2023-10-18T09:17:37.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.238+0000 7f7573fff700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 0x7f75740a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.239+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f757c014070 con 0x7f758c0686d0 2023-10-18T09:17:37.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.239+0000 7f758a7fc700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 0x7f75740a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.240+0000 7f758a7fc700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 0x7f75740a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f758c068e60 tx=0x7f7580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.241+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f757c052100 con 0x7f758c0686d0 2023-10-18T09:17:37.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:36 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2659677700' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-18T09:17:37.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.540+0000 7f4c6c98c700 1 -- 172.21.15.61:0/3148508695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c680686d0 msgr2=0x7f4c68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.540+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/3148508695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c680686d0 0x7f4c68068b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009990 tx=0x7f4c5c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 -- 172.21.15.61:0/3148508695 shutdown_connections 2023-10-18T09:17:37.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/3148508695 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c68103250 0x7f4c68103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/3148508695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c68068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/3148508695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c680686d0 0x7f4c68068b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 -- 172.21.15.61:0/3148508695 >> 172.21.15.61:0/3148508695 conn(0x7f4c68075790 msgr2=0x7f4c68075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 -- 172.21.15.61:0/3148508695 shutdown_connections 2023-10-18T09:17:37.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.541+0000 7f4c6c98c700 1 -- 172.21.15.61:0/3148508695 wait complete. 2023-10-18T09:17:37.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 Processor -- start 2023-10-18T09:17:37.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 -- start start 2023-10-18T09:17:37.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 0x7f4c6819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c681a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681a2130 con 0x7f4c680686d0 2023-10-18T09:17:37.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681a22a0 con 0x7f4c68103250 2023-10-18T09:17:37.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.542+0000 7f4c6c98c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681a2460 con 0x7f4c681056f0 2023-10-18T09:17:37.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 0x7f4c6819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c66d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c681a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47964/0 (socket says 172.21.15.61:47964) 2023-10-18T09:17:37.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c6659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 0x7f4c6819d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37186/0 (socket says 172.21.15.61:37186) 2023-10-18T09:17:37.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 -- 172.21.15.61:0/1256833638 learned_addr learned my addr 172.21.15.61:0/1256833638 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:37.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 -- 172.21.15.61:0/1256833638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 msgr2=0x7f4c681a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c681a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 -- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 msgr2=0x7f4c6819d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 0x7f4c6819d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c65d9b700 1 -- 172.21.15.61:0/1256833638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c0095f0 con 0x7f4c68103250 2023-10-18T09:17:37.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.543+0000 7f4c66d9d700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c681a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:37.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c65d9b700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005720 tx=0x7f4c5c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c01d070 con 0x7f4c68103250 2023-10-18T09:17:37.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c5c022470 con 0x7f4c68103250 2023-10-18T09:17:37.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c017650 con 0x7f4c68103250 2023-10-18T09:17:37.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c681a26e0 con 0x7f4c68103250 2023-10-18T09:17:37.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.544+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c681a9fa0 con 0x7f4c68103250 2023-10-18T09:17:37.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.546+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680ffd30 con 0x7f4c68103250 2023-10-18T09:17:37.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.548+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4c5c0225e0 con 0x7f4c68103250 2023-10-18T09:17:37.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.549+0000 7f4c577fe700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 0x7f4c500a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.549+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4c5c0ce610 con 0x7f4c68103250 2023-10-18T09:17:37.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.549+0000 7f4c6659c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 0x7f4c500a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.550+0000 7f4c6659c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 0x7f4c500a3a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009be0 tx=0x7f4c58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.553+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4c5c05c100 con 0x7f4c68103250 2023-10-18T09:17:37.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.574+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f758c19e490 con 0x7f758c0686d0 2023-10-18T09:17:37.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.575+0000 7f7573fff700 1 -- 172.21.15.61:0/710929134 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f757c0964e0 con 0x7f758c0686d0 2023-10-18T09:17:37.578 INFO:teuthology.orchestra.run.smithi061.stdout:111669149757 2023-10-18T09:17:37.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.577+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 msgr2=0x7f75740a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.577+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 0x7f75740a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f758c068e60 tx=0x7f7580009260 comp rx=0 tx=0).stop 2023-10-18T09:17:37.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.577+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 msgr2=0x7f758c19d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.577+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9a0 tx=0x7f757c00ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 shutdown_connections 2023-10-18T09:17:37.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75740a1590 0x7f75740a3a50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c1056f0 0x7f758c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0686d0 0x7f758c19d180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 --2- 172.21.15.61:0/710929134 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f758c103250 0x7f758c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 >> 172.21.15.61:0/710929134 conn(0x7f758c075790 msgr2=0x7f758c102230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 shutdown_connections 2023-10-18T09:17:37.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.578+0000 7f7591674700 1 -- 172.21.15.61:0/710929134 wait complete. 2023-10-18T09:17:37.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2119801292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc00ff750 msgr2=0x7f9cc00ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2119801292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc00ff750 0x7f9cc00ffbd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009990 tx=0x7f9cb0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2119801292 shutdown_connections 2023-10-18T09:17:37.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2119801292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc0100110 0x7f9cc010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2119801292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc01056b0 0x7f9cc0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2119801292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc00ff750 0x7f9cc00ffbd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2119801292 >> 172.21.15.61:0/2119801292 conn(0x7f9cc0075790 msgr2=0x7f9cc0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2119801292 shutdown_connections 2023-10-18T09:17:37.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.474+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2119801292 wait complete. 2023-10-18T09:17:37.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.475+0000 7f9cc7fa9700 1 Processor -- start 2023-10-18T09:17:37.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 -- start start 2023-10-18T09:17:37.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc0100110 0x7f9cc019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 0x7f9cc0196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01970b0 con 0x7f9cc00ff750 2023-10-18T09:17:37.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0197220 con 0x7f9cc0100110 2023-10-18T09:17:37.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc7fa9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01973e0 con 0x7f9cc01056b0 2023-10-18T09:17:37.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc5d45700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc5d45700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37164/0 (socket says 172.21.15.61:37164) 2023-10-18T09:17:37.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc5d45700 1 -- 172.21.15.61:0/2022651407 learned_addr learned my addr 172.21.15.61:0/2022651407 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:37.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.476+0000 7f9cc5544700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc0100110 0x7f9cc019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc6546700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 0x7f9cc0196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 -- 172.21.15.61:0/2022651407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 msgr2=0x7f9cc0196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 0x7f9cc0196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 -- 172.21.15.61:0/2022651407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc0100110 msgr2=0x7f9cc019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc0100110 0x7f9cc019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 -- 172.21.15.61:0/2022651407 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009520 con 0x7f9cc00ff750 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc6546700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 0x7f9cc0196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc5d45700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00eb30 tx=0x7f9cbc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc009880 con 0x7f9cc00ff750 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc0044e0 con 0x7f9cc00ff750 2023-10-18T09:17:37.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb00095f0 con 0x7f9cc00ff750 2023-10-18T09:17:37.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003e90 con 0x7f9cc00ff750 2023-10-18T09:17:37.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.477+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01a3730 con 0x7f9cc00ff750 2023-10-18T09:17:37.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.479+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0108cc0 con 0x7f9cc00ff750 2023-10-18T09:17:37.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.481+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9cbc004000 con 0x7f9cc00ff750 2023-10-18T09:17:37.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.482+0000 7f9cb6ffd700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 0x7f9cac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:37.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.482+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9cbc014070 con 0x7f9cc00ff750 2023-10-18T09:17:37.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.482+0000 7f9cc5544700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 0x7f9cac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:37.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.483+0000 7f9cc5544700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 0x7f9cac0a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00098e0 tx=0x7f9cb0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:37.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.491+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9cbc0967d0 con 0x7f9cc00ff750 2023-10-18T09:17:37.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.765+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f9cc0100df0 con 0x7f9cc00ff750 2023-10-18T09:17:37.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.765+0000 7f9cb6ffd700 1 -- 172.21.15.61:0/2022651407 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9cbc099df0 con 0x7f9cc00ff750 2023-10-18T09:17:37.768 INFO:teuthology.orchestra.run.smithi061.stdout:253403070504 2023-10-18T09:17:37.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 msgr2=0x7f9cac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 0x7f9cac0a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00098e0 tx=0x7f9cb0005960 comp rx=0 tx=0).stop 2023-10-18T09:17:37.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 msgr2=0x7f9cc019c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00eb30 tx=0x7f9cbc00ee40 comp rx=0 tx=0).stop 2023-10-18T09:17:37.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 shutdown_connections 2023-10-18T09:17:37.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9cac0a1510 0x7f9cac0a39d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9cc01056b0 0x7f9cc0196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9cc00ff750 0x7f9cc019c910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 --2- 172.21.15.61:0/2022651407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cc0100110 0x7f9cc019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 >> 172.21.15.61:0/2022651407 conn(0x7f9cc0075790 msgr2=0x7f9cc0068530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 shutdown_connections 2023-10-18T09:17:37.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.768+0000 7f9cc7fa9700 1 -- 172.21.15.61:0/2022651407 wait complete. 2023-10-18T09:17:37.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.827+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4c681a28d0 con 0x7f4c68103250 2023-10-18T09:17:37.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.827+0000 7f4c577fe700 1 -- 172.21.15.61:0/1256833638 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c5c097950 con 0x7f4c68103250 2023-10-18T09:17:37.830 INFO:teuthology.orchestra.run.smithi061.stdout:167503724598 2023-10-18T09:17:37.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 msgr2=0x7f4c500a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 0x7f4c500a3a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009be0 tx=0x7f4c58009260 comp rx=0 tx=0).stop 2023-10-18T09:17:37.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 msgr2=0x7f4c6819d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:37.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005720 tx=0x7f4c5c005750 comp rx=0 tx=0).stop 2023-10-18T09:17:37.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 shutdown_connections 2023-10-18T09:17:37.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4c500a1590 0x7f4c500a3a50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4c681056f0 0x7f4c681a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4c680686d0 0x7f4c6819d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 --2- 172.21.15.61:0/1256833638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c68103250 0x7f4c6819d770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:37.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 >> 172.21.15.61:0/1256833638 conn(0x7f4c68075790 msgr2=0x7f4c68100220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:37.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.830+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 shutdown_connections 2023-10-18T09:17:37.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:37.831+0000 7f4c6c98c700 1 -- 172.21.15.61:0/1256833638 wait complete. 2023-10-18T09:17:38.128 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:37 smithi061 ceph-mon[98573]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-18T09:17:38.129 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:37 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/710929134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T09:17:38.129 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:37 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2022651407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T09:17:38.129 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:37 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1256833638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T09:17:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:37 smithi110 ceph-mon[134818]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-18T09:17:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:37 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/710929134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T09:17:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:37 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2022651407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T09:17:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:37 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1256833638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T09:17:38.269 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:37 smithi170 ceph-mon[135150]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 255 B/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-10-18T09:17:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:37 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/710929134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-18T09:17:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:37 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2022651407' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-18T09:17:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:37 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1256833638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-18T09:17:38.563 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808866 for osd.9 2023-10-18T09:17:38.563 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:39.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724596 got 167503724598 for osd.5 2023-10-18T09:17:39.642 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:40.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149755 got 111669149757 for osd.3 2023-10-18T09:17:40.085 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:40.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070504 for osd.8 2023-10-18T09:17:40.127 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:40.170 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:39 smithi061 ceph-mon[98573]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:39 smithi110 ceph-mon[134818]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:40.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:39 smithi170 ceph-mon[135150]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:40.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.439+0000 7f5947b62700 1 -- 172.21.15.61:0/3754897398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068b30 msgr2=0x7f594010d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:40.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.439+0000 7f5947b62700 1 --2- 172.21.15.61:0/3754897398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068b30 0x7f594010d670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5938009990 tx=0x7f5938009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 -- 172.21.15.61:0/3754897398 shutdown_connections 2023-10-18T09:17:40.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 --2- 172.21.15.61:0/3754897398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068b30 0x7f594010d670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 --2- 172.21.15.61:0/3754897398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f5940106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 --2- 172.21.15.61:0/3754897398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068190 0x7f59400685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 -- 172.21.15.61:0/3754897398 >> 172.21.15.61:0/3754897398 conn(0x7f5940075790 msgr2=0x7f5940075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:40.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 -- 172.21.15.61:0/3754897398 shutdown_connections 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.440+0000 7f5947b62700 1 -- 172.21.15.61:0/3754897398 wait complete. 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.441+0000 7f5947b62700 1 Processor -- start 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 -- start start 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 0x7f59400ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 0x7f59400ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:40.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f59401002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:40.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401a4f70 con 0x7f59401067a0 2023-10-18T09:17:40.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401a50e0 con 0x7f5940068190 2023-10-18T09:17:40.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.442+0000 7f5947b62700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401a52a0 con 0x7f5940068b30 2023-10-18T09:17:40.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59460ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f59401002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:40.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59460ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f59401002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52176/0 (socket says 172.21.15.61:52176) 2023-10-18T09:17:40.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59460ff700 1 -- 172.21.15.61:0/338911689 learned_addr learned my addr 172.21.15.61:0/338911689 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:40.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 0x7f59400ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:40.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59450fd700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 0x7f59400ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:40.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 -- 172.21.15.61:0/338911689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 msgr2=0x7f59400ffd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:40.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 0x7f59400ffd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 -- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 msgr2=0x7f59401002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f59401002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59458fe700 1 -- 172.21.15.61:0/338911689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59380095f0 con 0x7f5940068190 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.443+0000 7f59450fd700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 0x7f59400ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f59458fe700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 0x7f59400ff820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f593400ef10 tx=0x7f593400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:40.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593400ca70 con 0x7f5940068190 2023-10-18T09:17:40.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59401a5520 con 0x7f5940068190 2023-10-18T09:17:40.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f593400f040 con 0x7f5940068190 2023-10-18T09:17:40.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5934007ea0 con 0x7f5940068190 2023-10-18T09:17:40.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.444+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59401a5a70 con 0x7f5940068190 2023-10-18T09:17:40.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.445+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940108d50 con 0x7f5940068190 2023-10-18T09:17:40.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.448+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5934004ad0 con 0x7f5940068190 2023-10-18T09:17:40.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.450+0000 7f593effd700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 0x7f59300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:40.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.450+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f59340cd480 con 0x7f5940068190 2023-10-18T09:17:40.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.450+0000 7f59450fd700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 0x7f59300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:40.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.451+0000 7f59450fd700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 0x7f59300a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5940100f50 tx=0x7f592c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:40.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.451+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5934096610 con 0x7f5940068190 2023-10-18T09:17:40.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.688+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5940100d00 con 0x7f5940068190 2023-10-18T09:17:40.691 INFO:teuthology.orchestra.run.smithi061.stdout:201863462962 2023-10-18T09:17:40.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.688+0000 7f593effd700 1 -- 172.21.15.61:0/338911689 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5934099c30 con 0x7f5940068190 2023-10-18T09:17:40.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 msgr2=0x7f59300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:40.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 0x7f59300a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5940100f50 tx=0x7f592c009260 comp rx=0 tx=0).stop 2023-10-18T09:17:40.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 msgr2=0x7f59400ff820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:40.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 0x7f59400ff820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f593400ef10 tx=0x7f593400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 shutdown_connections 2023-10-18T09:17:40.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59300a1510 0x7f59300a39d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f59401067a0 0x7f59401002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5940068190 0x7f59400ff820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 --2- 172.21.15.61:0/338911689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5940068b30 0x7f59400ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:40.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.693+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 >> 172.21.15.61:0/338911689 conn(0x7f5940075790 msgr2=0x7f59400fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:40.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.694+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 shutdown_connections 2023-10-18T09:17:40.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.694+0000 7f5947b62700 1 -- 172.21.15.61:0/338911689 wait complete. 2023-10-18T09:17:41.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:40 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/338911689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T09:17:41.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 -- 172.21.15.61:0/1191867259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 msgr2=0x7f3774105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:41.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 --2- 172.21.15.61:0/1191867259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f3774105ba0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3768009990 tx=0x7f3768009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 -- 172.21.15.61:0/1191867259 shutdown_connections 2023-10-18T09:17:41.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 --2- 172.21.15.61:0/1191867259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f377410d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 --2- 172.21.15.61:0/1191867259 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f3774105ba0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 --2- 172.21.15.61:0/1191867259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 0x7f3774100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 -- 172.21.15.61:0/1191867259 >> 172.21.15.61:0/1191867259 conn(0x7f37740779e0 msgr2=0x7f3774077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:41.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 -- 172.21.15.61:0/1191867259 shutdown_connections 2023-10-18T09:17:41.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.896+0000 7f377b016700 1 -- 172.21.15.61:0/1191867259 wait complete. 2023-10-18T09:17:41.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 Processor -- start 2023-10-18T09:17:41.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 -- start start 2023-10-18T09:17:41.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 0x7f3774198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:41.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f37741993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:41.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:41.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.897+0000 7f377b016700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419dda0 con 0x7f37741057c0 2023-10-18T09:17:41.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f377b016700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419df10 con 0x7f37741008a0 2023-10-18T09:17:41.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f377b016700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419e0d0 con 0x7f3774101260 2023-10-18T09:17:41.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:41.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52196/0 (socket says 172.21.15.61:52196) 2023-10-18T09:17:41.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 -- 172.21.15.61:0/2064094412 learned_addr learned my addr 172.21.15.61:0/2064094412 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:41.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f3778db2700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 0x7f3774198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:41.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f3773fff700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f37741993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:41.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 -- 172.21.15.61:0/2064094412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 msgr2=0x7f37741993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:41.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f37741993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 -- 172.21.15.61:0/2064094412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 msgr2=0x7f3774198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:41.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 0x7f3774198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 -- 172.21.15.61:0/2064094412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680095f0 con 0x7f37741057c0 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f3773fff700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f37741993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.898+0000 7f37795b3700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f376c007f10 tx=0x7f376c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.899+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c009ea0 con 0x7f37741057c0 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.899+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f376c00f040 con 0x7f37741057c0 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.899+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c004810 con 0x7f37741057c0 2023-10-18T09:17:41.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.899+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377419e3b0 con 0x7f37741057c0 2023-10-18T09:17:41.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.899+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37741a5c10 con 0x7f37741057c0 2023-10-18T09:17:41.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.900+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774101f30 con 0x7f37741057c0 2023-10-18T09:17:41.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.902+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f376c003a00 con 0x7f37741057c0 2023-10-18T09:17:41.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.903+0000 7f3771ffb700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 0x7f37640a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:41.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.903+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f376c0cc9d0 con 0x7f37741057c0 2023-10-18T09:17:41.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.903+0000 7f3778db2700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 0x7f37640a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:41.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.904+0000 7f3778db2700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 0x7f37640a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f376800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:41.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:40.908+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f376c04f100 con 0x7f37741057c0 2023-10-18T09:17:41.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.120+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f377419a250 con 0x7f37741057c0 2023-10-18T09:17:41.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.120+0000 7f3771ffb700 1 -- 172.21.15.61:0/2064094412 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f376c095b60 con 0x7f37741057c0 2023-10-18T09:17:41.219 INFO:teuthology.orchestra.run.smithi061.stdout:34359738446 2023-10-18T09:17:41.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 msgr2=0x7f37640a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:41.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 0x7f37640a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3768006010 tx=0x7f376800b260 comp rx=0 tx=0).stop 2023-10-18T09:17:41.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 msgr2=0x7f377419d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:41.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f376c007f10 tx=0x7f376c00d3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 shutdown_connections 2023-10-18T09:17:41.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f37640a1510 0x7f37640a39d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f37741057c0 0x7f377419d790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37741008a0 0x7f3774198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 --2- 172.21.15.61:0/2064094412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3774101260 0x7f37741993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:41.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.122+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 >> 172.21.15.61:0/2064094412 conn(0x7f37740779e0 msgr2=0x7f377410ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:41.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.123+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 shutdown_connections 2023-10-18T09:17:41.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:41.123+0000 7f377b016700 1 -- 172.21.15.61:0/2064094412 wait complete. 2023-10-18T09:17:41.222 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:40 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/338911689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T09:17:41.295 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:40 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/338911689' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-18T09:17:42.170 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:41 smithi061 ceph-mon[98573]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:42.170 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2064094412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T09:17:42.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462962 for osd.6 2023-10-18T09:17:42.173 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:41 smithi110 ceph-mon[134818]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2064094412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T09:17:42.217 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2023-10-18T09:17:42.217 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:42.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:41 smithi170 ceph-mon[135150]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:42.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2064094412' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-18T09:17:42.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 -- 172.21.15.61:0/4008256990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 msgr2=0x7f96d8102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 --2- 172.21.15.61:0/4008256990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8102e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0098f0 tx=0x7f96cc009c00 comp rx=0 tx=0).stop 2023-10-18T09:17:42.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 -- 172.21.15.61:0/4008256990 shutdown_connections 2023-10-18T09:17:42.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 --2- 172.21.15.61:0/4008256990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 0x7f96d810f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 --2- 172.21.15.61:0/4008256990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d8108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 --2- 172.21.15.61:0/4008256990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8102e50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 -- 172.21.15.61:0/4008256990 >> 172.21.15.61:0/4008256990 conn(0x7f96d80fe4c0 msgr2=0x7f96d81008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:42.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 -- 172.21.15.61:0/4008256990 shutdown_connections 2023-10-18T09:17:42.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.474+0000 7f96de6c6700 1 -- 172.21.15.61:0/4008256990 wait complete. 2023-10-18T09:17:42.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.475+0000 7f96de6c6700 1 Processor -- start 2023-10-18T09:17:42.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.475+0000 7f96de6c6700 1 -- start start 2023-10-18T09:17:42.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.475+0000 7f96de6c6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.475+0000 7f96de6c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 0x7f96d81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.475+0000 7f96de6c6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96de6c6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819dda0 con 0x7f96d81089d0 2023-10-18T09:17:42.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96de6c6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819df10 con 0x7f96d81029d0 2023-10-18T09:17:42.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96de6c6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d819e0d0 con 0x7f96d8103390 2023-10-18T09:17:42.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52210/0 (socket says 172.21.15.61:52210) 2023-10-18T09:17:42.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 -- 172.21.15.61:0/2271577147 learned_addr learned my addr 172.21.15.61:0/2271577147 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:42.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96d7fff700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96d77fe700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 0x7f96d81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 -- 172.21.15.61:0/2271577147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 msgr2=0x7f96d81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 0x7f96d81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 -- 172.21.15.61:0/2271577147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 msgr2=0x7f96d8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96dcc63700 1 -- 172.21.15.61:0/2271577147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c80095f0 con 0x7f96d81089d0 2023-10-18T09:17:42.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.476+0000 7f96d7fff700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:42.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96dcc63700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ef30 tx=0x7f96d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018070 con 0x7f96d81089d0 2023-10-18T09:17:42.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc009520 con 0x7f96d81089d0 2023-10-18T09:17:42.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d000f040 con 0x7f96d81089d0 2023-10-18T09:17:42.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000cd00 con 0x7f96d81089d0 2023-10-18T09:17:42.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.477+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d81a5a40 con 0x7f96d81089d0 2023-10-18T09:17:42.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.478+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d810aff0 con 0x7f96d81089d0 2023-10-18T09:17:42.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.480+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f96d0003a00 con 0x7f96d81089d0 2023-10-18T09:17:42.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.482+0000 7f96d57fa700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 0x7f96c00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.482+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f96d00cd6a0 con 0x7f96d81089d0 2023-10-18T09:17:42.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.482+0000 7f96d7fff700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 0x7f96c00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.483+0000 7f96d7fff700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 0x7f96c00a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005fd0 tx=0x7f96c8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.489+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96d00968c0 con 0x7f96d81089d0 2023-10-18T09:17:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.566+0000 7fde97c00700 1 -- 172.21.15.61:0/99270225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 msgr2=0x7fde90100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.566+0000 7fde97c00700 1 --2- 172.21.15.61:0/99270225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde90100d20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde80009990 tx=0x7fde80009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 -- 172.21.15.61:0/99270225 shutdown_connections 2023-10-18T09:17:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 --2- 172.21.15.61:0/99270225 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 --2- 172.21.15.61:0/99270225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 0x7fde90105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 --2- 172.21.15.61:0/99270225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde90100d20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 -- 172.21.15.61:0/99270225 >> 172.21.15.61:0/99270225 conn(0x7fde900779e0 msgr2=0x7fde90077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:42.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 -- 172.21.15.61:0/99270225 shutdown_connections 2023-10-18T09:17:42.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.567+0000 7fde97c00700 1 -- 172.21.15.61:0/99270225 wait complete. 2023-10-18T09:17:42.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.568+0000 7fde97c00700 1 Processor -- start 2023-10-18T09:17:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.568+0000 7fde97c00700 1 -- start start 2023-10-18T09:17:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.568+0000 7fde97c00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde97c00700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde97c00700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 0x7fde901a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde97c00700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde901a2100 con 0x7fde90101260 2023-10-18T09:17:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde97c00700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde901a2270 con 0x7fde901057c0 2023-10-18T09:17:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde97c00700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde901a2430 con 0x7fde901008a0 2023-10-18T09:17:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9519b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:53720/0 (socket says 172.21.15.61:53720) 2023-10-18T09:17:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9519b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52230/0 (socket says 172.21.15.61:52230) 2023-10-18T09:17:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 -- 172.21.15.61:0/1695320704 learned_addr learned my addr 172.21.15.61:0/1695320704 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9619d700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 0x7fde901a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 -- 172.21.15.61:0/1695320704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 msgr2=0x7fde901a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 0x7fde901a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 -- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 msgr2=0x7fde9010ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.569+0000 7fde9599c700 1 -- 172.21.15.61:0/1695320704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde800095f0 con 0x7fde901008a0 2023-10-18T09:17:42.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde9599c700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e970 tx=0x7fde8c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c009790 con 0x7fde901008a0 2023-10-18T09:17:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde8c00eec0 con 0x7fde901008a0 2023-10-18T09:17:42.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c003cf0 con 0x7fde901008a0 2023-10-18T09:17:42.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde901a2710 con 0x7fde901008a0 2023-10-18T09:17:42.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.570+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde901a9f90 con 0x7fde901008a0 2023-10-18T09:17:42.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.572+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde90101f30 con 0x7fde901008a0 2023-10-18T09:17:42.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.574+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fde8c00ccd0 con 0x7fde901008a0 2023-10-18T09:17:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.575+0000 7fde86ffd700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 0x7fde7c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.576+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fde8c014070 con 0x7fde901008a0 2023-10-18T09:17:42.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.576+0000 7fde9519b700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 0x7fde7c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.577+0000 7fde9519b700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 0x7fde7c0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde90106c80 tx=0x7fde80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.580+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fde8c096590 con 0x7fde901008a0 2023-10-18T09:17:42.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.719+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f96d8104110 con 0x7f96d81089d0 2023-10-18T09:17:42.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.719+0000 7f96d57fa700 1 -- 172.21.15.61:0/2271577147 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f96d0099ee0 con 0x7f96d81089d0 2023-10-18T09:17:42.722 INFO:teuthology.orchestra.run.smithi061.stdout:309237645343 2023-10-18T09:17:42.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.723+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 msgr2=0x7f96c00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.723+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 0x7f96c00a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005fd0 tx=0x7f96c8009310 comp rx=0 tx=0).stop 2023-10-18T09:17:42.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.723+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 msgr2=0x7f96d819d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.723+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ef30 tx=0x7f96d000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 shutdown_connections 2023-10-18T09:17:42.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f96c00a1590 0x7f96c00a3a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f96d81089d0 0x7f96d819d790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96d81029d0 0x7f96d8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 --2- 172.21.15.61:0/2271577147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d8103390 0x7f96d81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 >> 172.21.15.61:0/2271577147 conn(0x7f96d80fe4c0 msgr2=0x7f96d80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:42.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 shutdown_connections 2023-10-18T09:17:42.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.724+0000 7f96de6c6700 1 -- 172.21.15.61:0/2271577147 wait complete. 2023-10-18T09:17:42.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.807+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fde9004e4c0 con 0x7fde901008a0 2023-10-18T09:17:42.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.807+0000 7fde86ffd700 1 -- 172.21.15.61:0/1695320704 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde8c094e60 con 0x7fde901008a0 2023-10-18T09:17:42.811 INFO:teuthology.orchestra.run.smithi061.stdout:223338299436 2023-10-18T09:17:42.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 msgr2=0x7fde7c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 0x7fde7c0a39d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde90106c80 tx=0x7fde80005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 msgr2=0x7fde9010c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e970 tx=0x7fde8c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:42.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 shutdown_connections 2023-10-18T09:17:42.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fde7c0a1510 0x7fde7c0a39d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde901057c0 0x7fde901a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde901008a0 0x7fde9010c8c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 --2- 172.21.15.61:0/1695320704 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fde90101260 0x7fde9010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 >> 172.21.15.61:0/1695320704 conn(0x7fde900779e0 msgr2=0x7fde900fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:42.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.810+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 shutdown_connections 2023-10-18T09:17:42.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.811+0000 7fde97c00700 1 -- 172.21.15.61:0/1695320704 wait complete. 2023-10-18T09:17:42.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 -- 172.21.15.61:0/3285923688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d601008a0 msgr2=0x7f1d60100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 --2- 172.21.15.61:0/3285923688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d601008a0 0x7f1d60100d20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009990 tx=0x7f1d50009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 -- 172.21.15.61:0/3285923688 shutdown_connections 2023-10-18T09:17:42.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 --2- 172.21.15.61:0/3285923688 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d60101260 0x7f1d6010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 --2- 172.21.15.61:0/3285923688 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601057c0 0x7f1d60105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 --2- 172.21.15.61:0/3285923688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d601008a0 0x7f1d60100d20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 -- 172.21.15.61:0/3285923688 >> 172.21.15.61:0/3285923688 conn(0x7f1d600779e0 msgr2=0x7f1d60077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:42.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 -- 172.21.15.61:0/3285923688 shutdown_connections 2023-10-18T09:17:42.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.887+0000 7f1d67599700 1 -- 172.21.15.61:0/3285923688 wait complete. 2023-10-18T09:17:42.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.889+0000 7f1d67599700 1 Processor -- start 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.889+0000 7f1d67599700 1 -- start start 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 0x7f1d6019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d60101260 0x7f1d6019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 0x7f1d601969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60197050 con 0x7f1d601057c0 2023-10-18T09:17:42.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601971c0 con 0x7f1d601008a0 2023-10-18T09:17:42.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d67599700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60197380 con 0x7f1d60101260 2023-10-18T09:17:42.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d65335700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 0x7f1d6019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d65b36700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 0x7f1d601969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d65335700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 0x7f1d6019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:33986/0 (socket says 172.21.15.61:33986) 2023-10-18T09:17:42.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d65335700 1 -- 172.21.15.61:0/1386097635 learned_addr learned my addr 172.21.15.61:0/1386097635 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:42.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.890+0000 7f1d64b34700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d60101260 0x7f1d6019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 -- 172.21.15.61:0/1386097635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d60101260 msgr2=0x7f1d6019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d60101260 0x7f1d6019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 -- 172.21.15.61:0/1386097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 msgr2=0x7f1d6019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:42.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 0x7f1d6019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:42.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 -- 172.21.15.61:0/1386097635 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d601057c0 2023-10-18T09:17:42.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.891+0000 7f1d65b36700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 0x7f1d601969f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800efc0 tx=0x7f1d5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.892+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58018070 con 0x7f1d601057c0 2023-10-18T09:17:42.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.892+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5800f040 con 0x7f1d601057c0 2023-10-18T09:17:42.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.892+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d60197600 con 0x7f1d601057c0 2023-10-18T09:17:42.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.892+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d601a38a0 con 0x7f1d601057c0 2023-10-18T09:17:42.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.892+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800cd30 con 0x7f1d601057c0 2023-10-18T09:17:42.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.894+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d44005320 con 0x7f1d601057c0 2023-10-18T09:17:42.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.898+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1d58004a90 con 0x7f1d601057c0 2023-10-18T09:17:42.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.900+0000 7f1d567fc700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 0x7f1d4c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:42.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.901+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1d580cccd0 con 0x7f1d601057c0 2023-10-18T09:17:42.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.901+0000 7f1d65335700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 0x7f1d4c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:42.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.901+0000 7f1d65335700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 0x7f1d4c0a7fb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00bc50 tx=0x7f1d5c00b630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:42.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:42.903+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d58095f70 con 0x7f1d601057c0 2023-10-18T09:17:43.092 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2271577147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T09:17:43.092 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1695320704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T09:17:43.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.159+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1d44005190 con 0x7f1d601057c0 2023-10-18T09:17:43.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.159+0000 7f1d567fc700 1 -- 172.21.15.61:0/1386097635 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d58099590 con 0x7f1d601057c0 2023-10-18T09:17:43.162 INFO:teuthology.orchestra.run.smithi061.stdout:146028888122 2023-10-18T09:17:43.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.161+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 msgr2=0x7f1d4c0a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:43.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.161+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 0x7f1d4c0a7fb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00bc50 tx=0x7f1d5c00b630 comp rx=0 tx=0).stop 2023-10-18T09:17:43.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.161+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 msgr2=0x7f1d601969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:43.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.161+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 0x7f1d601969f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800efc0 tx=0x7f1d5800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 shutdown_connections 2023-10-18T09:17:43.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1d4c0a5af0 0x7f1d4c0a7fb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1d601057c0 0x7f1d601969f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1d601008a0 0x7f1d6019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 --2- 172.21.15.61:0/1386097635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d60101260 0x7f1d6019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 >> 172.21.15.61:0/1386097635 conn(0x7f1d600779e0 msgr2=0x7f1d600fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:43.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 shutdown_connections 2023-10-18T09:17:43.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.162+0000 7f1d67599700 1 -- 172.21.15.61:0/1386097635 wait complete. 2023-10-18T09:17:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2271577147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T09:17:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1695320704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T09:17:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2271577147' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-18T09:17:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1695320704' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-18T09:17:43.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299436 for osd.7 2023-10-18T09:17:43.524 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:43.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 -- 172.21.15.61:0/2700379802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd80686d0 msgr2=0x7f4dd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:43.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 --2- 172.21.15.61:0/2700379802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd80686d0 0x7f4dd8068b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009990 tx=0x7f4dc8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 -- 172.21.15.61:0/2700379802 shutdown_connections 2023-10-18T09:17:43.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 --2- 172.21.15.61:0/2700379802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8103250 0x7f4dd8103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 --2- 172.21.15.61:0/2700379802 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd81056f0 0x7f4dd8068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 --2- 172.21.15.61:0/2700379802 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd80686d0 0x7f4dd8068b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 -- 172.21.15.61:0/2700379802 >> 172.21.15.61:0/2700379802 conn(0x7f4dd8075790 msgr2=0x7f4dd8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:43.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 -- 172.21.15.61:0/2700379802 shutdown_connections 2023-10-18T09:17:43.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.759+0000 7f4de0608700 1 -- 172.21.15.61:0/2700379802 wait complete. 2023-10-18T09:17:43.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.760+0000 7f4de0608700 1 Processor -- start 2023-10-18T09:17:43.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.760+0000 7f4de0608700 1 -- start start 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd81056f0 0x7f4dd8106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8107210 0x7f4dd81076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810b790 con 0x7f4dd81056f0 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810b900 con 0x7f4dd8103250 2023-10-18T09:17:43.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4de0608700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd810bac0 con 0x7f4dd8107210 2023-10-18T09:17:43.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4ddeba5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8107210 0x7f4dd81076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:43.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4dde3a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:43.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4dde3a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:34028/0 (socket says 172.21.15.61:34028) 2023-10-18T09:17:43.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.761+0000 7f4dde3a4700 1 -- 172.21.15.61:0/2206564118 learned_addr learned my addr 172.21.15.61:0/2206564118 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:43.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dddba3700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd81056f0 0x7f4dd8106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:43.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 -- 172.21.15.61:0/2206564118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8107210 msgr2=0x7f4dd81076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:43.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8107210 0x7f4dd81076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 -- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd81056f0 msgr2=0x7f4dd8106cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:43.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd81056f0 0x7f4dd8106cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:43.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 -- 172.21.15.61:0/2206564118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80095f0 con 0x7f4dd8103250 2023-10-18T09:17:43.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.762+0000 7f4dde3a4700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9a0 tx=0x7f4dd400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:43.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.763+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4009790 con 0x7f4dd8103250 2023-10-18T09:17:43.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.763+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4dd400cbd0 con 0x7f4dd8103250 2023-10-18T09:17:43.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.763+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd810bda0 con 0x7f4dd8103250 2023-10-18T09:17:43.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.763+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4018680 con 0x7f4dd8103250 2023-10-18T09:17:43.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.763+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd810c270 con 0x7f4dd8103250 2023-10-18T09:17:43.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.764+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd80ffd30 con 0x7f4dd8103250 2023-10-18T09:17:43.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.768+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4dd400cd40 con 0x7f4dd8103250 2023-10-18T09:17:43.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.770+0000 7f4dcf7fe700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 0x7f4dc40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:43.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.770+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4dd4014070 con 0x7f4dd8103250 2023-10-18T09:17:43.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.770+0000 7f4dddba3700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 0x7f4dc40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:43.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.771+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4dd4096850 con 0x7f4dd8103250 2023-10-18T09:17:43.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.773+0000 7f4dddba3700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 0x7f4dc40a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098e0 tx=0x7f4dc8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:43.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888119 got 146028888122 for osd.4 2023-10-18T09:17:43.976 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:43 smithi061 ceph-mon[98573]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-10-18T09:17:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1386097635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T09:17:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.009+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4dd810c550 con 0x7f4dd8103250 2023-10-18T09:17:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.009+0000 7f4dcf7fe700 1 -- 172.21.15.61:0/2206564118 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4dd4099e70 con 0x7f4dd8103250 2023-10-18T09:17:44.012 INFO:teuthology.orchestra.run.smithi061.stdout:77309411398 2023-10-18T09:17:44.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 msgr2=0x7f4dc40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 0x7f4dc40a39d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098e0 tx=0x7f4dc8005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 msgr2=0x7f4dd810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400e9a0 tx=0x7f4dd400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 shutdown_connections 2023-10-18T09:17:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4dc40a1510 0x7f4dc40a39d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd8107210 0x7f4dd81076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dd8103250 0x7f4dd810dbf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 --2- 172.21.15.61:0/2206564118 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4dd81056f0 0x7f4dd8106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.012+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 >> 172.21.15.61:0/2206564118 conn(0x7f4dd8075790 msgr2=0x7f4dd8100220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:44.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.013+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 shutdown_connections 2023-10-18T09:17:44.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:44.013+0000 7f4de0608700 1 -- 172.21.15.61:0/2206564118 wait complete. 2023-10-18T09:17:44.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 -- 172.21.15.61:0/2947594499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 msgr2=0x7ff3d8105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/2947594499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d8105ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009990 tx=0x7ff3d4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 -- 172.21.15.61:0/2947594499 shutdown_connections 2023-10-18T09:17:44.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/2947594499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d8101260 0x7ff3d810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/2947594499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d8105ba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/2947594499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d81008a0 0x7ff3d8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.692+0000 7ff3e02e5700 1 -- 172.21.15.61:0/2947594499 >> 172.21.15.61:0/2947594499 conn(0x7ff3d80779e0 msgr2=0x7ff3d8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:44.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.693+0000 7ff3e02e5700 1 -- 172.21.15.61:0/2947594499 shutdown_connections 2023-10-18T09:17:44.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.693+0000 7ff3e02e5700 1 -- 172.21.15.61:0/2947594499 wait complete. 2023-10-18T09:17:44.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.694+0000 7ff3e02e5700 1 Processor -- start 2023-10-18T09:17:44.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.694+0000 7ff3e02e5700 1 -- start start 2023-10-18T09:17:44.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 0x7ff3d8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:44.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 0x7ff3d8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:44.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:44.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d819dd30 con 0x7ff3d81057c0 2023-10-18T09:17:44.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d819dea0 con 0x7ff3d8101260 2023-10-18T09:17:44.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3e02e5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d819e060 con 0x7ff3d81008a0 2023-10-18T09:17:44.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.695+0000 7ff3de081700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 0x7ff3d8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:44.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:44.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de081700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 0x7ff3d8198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:53754/0 (socket says 172.21.15.61:53754) 2023-10-18T09:17:44.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52262/0 (socket says 172.21.15.61:52262) 2023-10-18T09:17:44.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3dd880700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 0x7ff3d8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:44.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de081700 1 -- 172.21.15.61:0/1826191274 learned_addr learned my addr 172.21.15.61:0/1826191274 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:44.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 -- 172.21.15.61:0/1826191274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 msgr2=0x7ff3d8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 0x7ff3d8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 -- 172.21.15.61:0/1826191274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 msgr2=0x7ff3d8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 0x7ff3d8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 -- 172.21.15.61:0/1826191274 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40095f0 con 0x7ff3d81057c0 2023-10-18T09:17:44.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3dd880700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 0x7ff3d8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:44.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.696+0000 7ff3de882700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:44.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.697+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0018070 con 0x7ff3d81057c0 2023-10-18T09:17:44.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.697+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3d000f040 con 0x7ff3d81057c0 2023-10-18T09:17:44.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.697+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d819e2e0 con 0x7ff3d81057c0 2023-10-18T09:17:44.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.697+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000cc80 con 0x7ff3d81057c0 2023-10-18T09:17:44.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.697+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d81a5ba0 con 0x7ff3d81057c0 2023-10-18T09:17:44.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.698+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d8101f30 con 0x7ff3d81057c0 2023-10-18T09:17:44.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.701+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff3d0003a00 con 0x7ff3d81057c0 2023-10-18T09:17:44.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.703+0000 7ff3cf7fe700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 0x7ff3c40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:44.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.703+0000 7ff3de081700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 0x7ff3c40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:44.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.703+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff3d00cc8b0 con 0x7ff3d81057c0 2023-10-18T09:17:44.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.704+0000 7ff3de081700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 0x7ff3c40a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b6f0 tx=0x7ff3d4005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:44.061 INFO:teuthology.orchestra.run.smithi061.stdout:55834574921 2023-10-18T09:17:44.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.707+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3d0095b30 con 0x7ff3d81057c0 2023-10-18T09:17:44.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.970+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff3d8110710 con 0x7ff3d81057c0 2023-10-18T09:17:44.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.971+0000 7ff3cf7fe700 1 -- 172.21.15.61:0/1826191274 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff3d0099150 con 0x7ff3d81057c0 2023-10-18T09:17:44.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.973+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 msgr2=0x7ff3c40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.973+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 0x7ff3c40a39d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400b6f0 tx=0x7ff3d4005bc0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 msgr2=0x7ff3d819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:44.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff3d000efc0 tx=0x7ff3d000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 shutdown_connections 2023-10-18T09:17:44.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3c40a1510 0x7ff3c40a39d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff3d81057c0 0x7ff3d819d720 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d81008a0 0x7ff3d8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 --2- 172.21.15.61:0/1826191274 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff3d8101260 0x7ff3d8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:44.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 >> 172.21.15.61:0/1826191274 conn(0x7ff3d80779e0 msgr2=0x7ff3d810ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:44.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 shutdown_connections 2023-10-18T09:17:44.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:43.974+0000 7ff3e02e5700 1 -- 172.21.15.61:0/1826191274 wait complete. 2023-10-18T09:17:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:43 smithi110 ceph-mon[134818]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-10-18T09:17:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1386097635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T09:17:44.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:43 smithi170 ceph-mon[135150]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2023-10-18T09:17:44.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1386097635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-18T09:17:44.426 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645341 got 309237645343 for osd.10 2023-10-18T09:17:44.427 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:45.127 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411398 for osd.2 2023-10-18T09:17:45.128 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:45.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:44 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1826191274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T09:17:45.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:44 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2206564118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T09:17:45.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:44 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:44 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:44 smithi110 ceph-mon[134818]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:45.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:44 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1826191274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T09:17:45.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:44 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2206564118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T09:17:45.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:44 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:44 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:44 smithi061 ceph-mon[98573]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:44 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1826191274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-18T09:17:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:44 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2206564118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-18T09:17:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:44 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:44 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:45.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:44 smithi170 ceph-mon[135150]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:45.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574921 for osd.1 2023-10-18T09:17:45.465 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:45.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 -- 172.21.15.61:0/882274512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 msgr2=0x7ff37c10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:45.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 --2- 172.21.15.61:0/882274512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c10cdc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff370009990 tx=0x7ff370009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 -- 172.21.15.61:0/882274512 shutdown_connections 2023-10-18T09:17:45.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 --2- 172.21.15.61:0/882274512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 0x7ff37c10f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 --2- 172.21.15.61:0/882274512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c068570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 --2- 172.21.15.61:0/882274512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c10cdc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 -- 172.21.15.61:0/882274512 >> 172.21.15.61:0/882274512 conn(0x7ff37c075790 msgr2=0x7ff37c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:45.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 -- 172.21.15.61:0/882274512 shutdown_connections 2023-10-18T09:17:45.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.820+0000 7ff380ef4700 1 -- 172.21.15.61:0/882274512 wait complete. 2023-10-18T09:17:45.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.822+0000 7ff380ef4700 1 Processor -- start 2023-10-18T09:17:45.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.822+0000 7ff380ef4700 1 -- start start 2023-10-18T09:17:45.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.822+0000 7ff380ef4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:45.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff380ef4700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:45.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff380ef4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 0x7ff37c19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:45.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff380ef4700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19de30 con 0x7ff37c068ab0 2023-10-18T09:17:45.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff380ef4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19dfa0 con 0x7ff37c10d300 2023-10-18T09:17:45.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff380ef4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19e160 con 0x7ff37c068190 2023-10-18T09:17:45.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff379d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff379d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52284/0 (socket says 172.21.15.61:52284) 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff379d9b700 1 -- 172.21.15.61:0/1293868801 learned_addr learned my addr 172.21.15.61:0/1293868801 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37ad9d700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 0x7ff37c19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 -- 172.21.15.61:0/1293868801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 msgr2=0x7ff37c19d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:45.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 0x7ff37c19d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 -- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 msgr2=0x7ff37c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:45.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:45.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.823+0000 7ff37a59c700 1 -- 172.21.15.61:0/1293868801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3700095f0 con 0x7ff37c068190 2023-10-18T09:17:45.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff37a59c700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c198f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9f0 tx=0x7ff36c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:45.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00c9c0 con 0x7ff37c068190 2023-10-18T09:17:45.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff36c010430 con 0x7ff37c068190 2023-10-18T09:17:45.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37c19e440 con 0x7ff37c068190 2023-10-18T09:17:45.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c018680 con 0x7ff37c068190 2023-10-18T09:17:45.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.824+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c1a5ca0 con 0x7ff37c068190 2023-10-18T09:17:45.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.826+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37c108520 con 0x7ff37c068190 2023-10-18T09:17:45.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.827+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff36c010860 con 0x7ff37c068190 2023-10-18T09:17:45.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.829+0000 7ff36b7fe700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 0x7ff3640a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:45.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.829+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff36c014070 con 0x7ff37c068190 2023-10-18T09:17:45.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.830+0000 7ff379d9b700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 0x7ff3640a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:45.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.830+0000 7ff379d9b700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 0x7ff3640a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff37c19a490 tx=0x7ff37000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:45.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:45.833+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff36c0964b0 con 0x7ff37c068190 2023-10-18T09:17:46.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.056+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ff37c04e4f0 con 0x7ff37c068190 2023-10-18T09:17:46.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.057+0000 7ff36b7fe700 1 -- 172.21.15.61:0/1293868801 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ff36c099ad0 con 0x7ff37c068190 2023-10-18T09:17:46.059 INFO:teuthology.orchestra.run.smithi061.stdout:339302416412 2023-10-18T09:17:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 msgr2=0x7ff3640a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 0x7ff3640a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff37c19a490 tx=0x7ff37000b260 comp rx=0 tx=0).stop 2023-10-18T09:17:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 msgr2=0x7ff37c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:46.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c198f30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00e9f0 tx=0x7ff36c00ed00 comp rx=0 tx=0).stop 2023-10-18T09:17:46.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 shutdown_connections 2023-10-18T09:17:46.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff3640a1510 0x7ff3640a39d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:46.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff37c10d300 0x7ff37c19d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:46.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37c068190 0x7ff37c198f30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:46.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 --2- 172.21.15.61:0/1293868801 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff37c068ab0 0x7ff37c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:46.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.059+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 >> 172.21.15.61:0/1293868801 conn(0x7ff37c075790 msgr2=0x7ff37c10e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:46.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.060+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 shutdown_connections 2023-10-18T09:17:46.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:46.060+0000 7ff380ef4700 1 -- 172.21.15.61:0/1293868801 wait complete. 2023-10-18T09:17:46.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416409 got 339302416412 for osd.11 2023-10-18T09:17:46.532 DEBUG:teuthology.parallel:result is None 2023-10-18T09:17:46.533 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-18T09:17:46.533 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-18T09:17:46.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:17:46.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:17:46.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:46.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:46 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1293868801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T09:17:46.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:17:46.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:17:46.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:46.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1293868801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T09:17:46.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:17:46.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:17:46.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:17:46.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:46 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1293868801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-18T09:17:47.304 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:47.808 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:47 smithi061 ceph-mon[98573]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:47 smithi170 ceph-mon[135150]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:47 smithi110 ceph-mon[134818]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:48.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 -- 172.21.15.61:0/2562300263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 msgr2=0x7f6db8105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:48.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/2562300263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db8105ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009990 tx=0x7f6da8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 -- 172.21.15.61:0/2562300263 shutdown_connections 2023-10-18T09:17:48.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/2562300263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db810d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/2562300263 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db8105ba0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/2562300263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 0x7f6db8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 -- 172.21.15.61:0/2562300263 >> 172.21.15.61:0/2562300263 conn(0x7f6db80779e0 msgr2=0x7f6db8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 -- 172.21.15.61:0/2562300263 shutdown_connections 2023-10-18T09:17:48.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.756+0000 7f6dbcc73700 1 -- 172.21.15.61:0/2562300263 wait complete. 2023-10-18T09:17:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 Processor -- start 2023-10-18T09:17:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 -- start start 2023-10-18T09:17:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 0x7f6db8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db81993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.757+0000 7f6dbcc73700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db819dda0 con 0x7f6db81057c0 2023-10-18T09:17:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6dbcc73700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db819df10 con 0x7f6db81008a0 2023-10-18T09:17:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6dbcc73700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db819e0d0 con 0x7f6db8101260 2023-10-18T09:17:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52310/0 (socket says 172.21.15.61:52310) 2023-10-18T09:17:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 -- 172.21.15.61:0/1457164353 learned_addr learned my addr 172.21.15.61:0/1457164353 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db5d9b700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db81993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 0x7f6db8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 -- 172.21.15.61:0/1457164353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 msgr2=0x7f6db81993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db81993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 -- 172.21.15.61:0/1457164353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 msgr2=0x7f6db8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:48.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 0x7f6db8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:48.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db6d9d700 1 -- 172.21.15.61:0/1457164353 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80095f0 con 0x7f6db81057c0 2023-10-18T09:17:48.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.758+0000 7f6db5d9b700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:48.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6db6d9d700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6db000efc0 tx=0x7f6db000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:48.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0018070 con 0x7f6db81057c0 2023-10-18T09:17:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6db000f040 con 0x7f6db81057c0 2023-10-18T09:17:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db819e350 con 0x7f6db81057c0 2023-10-18T09:17:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db000cc10 con 0x7f6db81057c0 2023-10-18T09:17:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.759+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db81a5c10 con 0x7f6db81057c0 2023-10-18T09:17:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.760+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db8101f30 con 0x7f6db81057c0 2023-10-18T09:17:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.762+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6db000cd70 con 0x7f6db81057c0 2023-10-18T09:17:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.763+0000 7f6da77fe700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 0x7f6da00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.763+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6db00ccb00 con 0x7f6db81057c0 2023-10-18T09:17:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.763+0000 7f6db659c700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 0x7f6da00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.764+0000 7f6db659c700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 0x7f6da00a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6da800b2e0 tx=0x7f6da8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.768+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6db00cee60 con 0x7f6db81057c0 2023-10-18T09:17:48.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.974+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6db8110760 con 0x7f6da00a1510 2023-10-18T09:17:48.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.989+0000 7f6da77fe700 1 -- 172.21.15.61:0/1457164353 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426641 (secure 0 0 0) 0x7f6db8110760 con 0x7f6da00a1510 2023-10-18T09:17:48.999 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:49.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 msgr2=0x7f6da00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:49.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 0x7f6da00a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6da800b2e0 tx=0x7f6da8005fd0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 msgr2=0x7f6db819d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:49.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6db000efc0 tx=0x7f6db000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 shutdown_connections 2023-10-18T09:17:49.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6da00a1510 0x7f6da00a39d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6db81057c0 0x7f6db819d790 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6db81008a0 0x7f6db8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 --2- 172.21.15.61:0/1457164353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db8101260 0x7f6db81993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:49.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.994+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 >> 172.21.15.61:0/1457164353 conn(0x7f6db80779e0 msgr2=0x7f6db810ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:49.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.995+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 shutdown_connections 2023-10-18T09:17:49.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:48.995+0000 7f6dbcc73700 1 -- 172.21.15.61:0/1457164353 wait complete. 2023-10-18T09:17:49.005 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:17:49.512 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2023-10-18T09:17:48.853215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658257,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":642,"num_read_kb":444,"num_write":422,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180927,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338232,"kb_used_data":13976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603592,"statfs":{"total":1151940427776,"available":1151594078208,"internally_reserved":0,"allocated":14311424,"data_stored":7840158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911071+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:17:20.911071+0000","last_peered":"2023-10-18T09:17:20.911071+0000","last_clean":"2023-10-18T09:17:20.911071+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:17:20.911071+0000","last_undegraded":"2023-10-18T09:17:20.911071+0000","last_fullsized":"2023-10-18T09:17:20.911071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908066+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:17:20.908066+0000","last_peered":"2023-10-18T09:17:20.908066+0000","last_clean":"2023-10-18T09:17:20.908066+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:17:20.908066+0000","last_undegraded":"2023-10-18T09:17:20.908066+0000","last_fullsized":"2023-10-18T09:17:20.908066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907956+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:17:20.907956+0000","last_peered":"2023-10-18T09:17:20.907956+0000","last_clean":"2023-10-18T09:17:20.907956+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:17:20.907956+0000","last_undegraded":"2023-10-18T09:17:20.907956+0000","last_fullsized":"2023-10-18T09:17:20.907956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910940+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:17:20.910940+0000","last_peered":"2023-10-18T09:17:20.910940+0000","last_clean":"2023-10-18T09:17:20.910940+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:17:20.910940+0000","last_undegraded":"2023-10-18T09:17:20.910940+0000","last_fullsized":"2023-10-18T09:17:20.910940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912977+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:17:20.912977+0000","last_peered":"2023-10-18T09:17:20.912977+0000","last_clean":"2023-10-18T09:17:20.912977+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:17:20.912977+0000","last_undegraded":"2023-10-18T09:17:20.912977+0000","last_fullsized":"2023-10-18T09:17:20.912977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910640+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:17:20.910640+0000","last_peered":"2023-10-18T09:17:20.910640+0000","last_clean":"2023-10-18T09:17:20.910640+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:17:20.910640+0000","last_undegraded":"2023-10-18T09:17:20.910640+0000","last_fullsized":"2023-10-18T09:17:20.910640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907372+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:17:20.907372+0000","last_peered":"2023-10-18T09:17:20.907372+0000","last_clean":"2023-10-18T09:17:20.907372+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:17:20.907372+0000","last_undegraded":"2023-10-18T09:17:20.907372+0000","last_fullsized":"2023-10-18T09:17:20.907372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915563+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:17:20.915563+0000","last_peered":"2023-10-18T09:17:20.915563+0000","last_clean":"2023-10-18T09:17:20.915563+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:17:20.915563+0000","last_undegraded":"2023-10-18T09:17:20.915563+0000","last_fullsized":"2023-10-18T09:17:20.915563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915035+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:17:20.915035+0000","last_peered":"2023-10-18T09:17:20.915035+0000","last_clean":"2023-10-18T09:17:20.915035+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:17:20.915035+0000","last_undegraded":"2023-10-18T09:17:20.915035+0000","last_fullsized":"2023-10-18T09:17:20.915035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911700+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:17:20.911700+0000","last_peered":"2023-10-18T09:17:20.911700+0000","last_clean":"2023-10-18T09:17:20.911700+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:17:20.911700+0000","last_undegraded":"2023-10-18T09:17:20.911700+0000","last_fullsized":"2023-10-18T09:17:20.911700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907951+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:17:20.907951+0000","last_peered":"2023-10-18T09:17:20.907951+0000","last_clean":"2023-10-18T09:17:20.907951+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:17:20.907951+0000","last_undegraded":"2023-10-18T09:17:20.907951+0000","last_fullsized":"2023-10-18T09:17:20.907951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910831+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:17:20.910831+0000","last_peered":"2023-10-18T09:17:20.910831+0000","last_clean":"2023-10-18T09:17:20.910831+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:17:20.910831+0000","last_undegraded":"2023-10-18T09:17:20.910831+0000","last_fullsized":"2023-10-18T09:17:20.910831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911541+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:17:20.911541+0000","last_peered":"2023-10-18T09:17:20.911541+0000","last_clean":"2023-10-18T09:17:20.911541+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:17:20.911541+0000","last_undegraded":"2023-10-18T09:17:20.911541+0000","last_fullsized":"2023-10-18T09:17:20.911541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907597+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:17:20.907597+0000","last_peered":"2023-10-18T09:17:20.907597+0000","last_clean":"2023-10-18T09:17:20.907597+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:17:20.907597+0000","last_undegraded":"2023-10-18T09:17:20.907597+0000","last_fullsized":"2023-10-18T09:17:20.907597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908148+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:17:20.908148+0000","last_peered":"2023-10-18T09:17:20.908148+0000","last_clean":"2023-10-18T09:17:20.908148+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:17:20.908148+0000","last_undegraded":"2023-10-18T09:17:20.908148+0000","last_fullsized":"2023-10-18T09:17:20.908148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914434+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:17:20.914434+0000","last_peered":"2023-10-18T09:17:20.914434+0000","last_clean":"2023-10-18T09:17:20.914434+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:17:20.914434+0000","last_undegraded":"2023-10-18T09:17:20.914434+0000","last_fullsized":"2023-10-18T09:17:20.914434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911035+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:17:20.911035+0000","last_peered":"2023-10-18T09:17:20.911035+0000","last_clean":"2023-10-18T09:17:20.911035+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:17:20.911035+0000","last_undegraded":"2023-10-18T09:17:20.911035+0000","last_fullsized":"2023-10-18T09:17:20.911035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.927737+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:17:20.927737+0000","last_peered":"2023-10-18T09:17:20.927737+0000","last_clean":"2023-10-18T09:17:20.927737+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:17:20.927737+0000","last_undegraded":"2023-10-18T09:17:20.927737+0000","last_fullsized":"2023-10-18T09:17:20.927737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908526+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:17:20.908526+0000","last_peered":"2023-10-18T09:17:20.908526+0000","last_clean":"2023-10-18T09:17:20.908526+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:17:20.908526+0000","last_undegraded":"2023-10-18T09:17:20.908526+0000","last_fullsized":"2023-10-18T09:17:20.908526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907197+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:17:20.907197+0000","last_peered":"2023-10-18T09:17:20.907197+0000","last_clean":"2023-10-18T09:17:20.907197+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:17:20.907197+0000","last_undegraded":"2023-10-18T09:17:20.907197+0000","last_fullsized":"2023-10-18T09:17:20.907197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:43.612736+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:17:43.612736+0000","last_peered":"2023-10-18T09:17:43.612736+0000","last_clean":"2023-10-18T09:17:43.612736+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:17:43.612736+0000","last_undegraded":"2023-10-18T09:17:43.612736+0000","last_fullsized":"2023-10-18T09:17:43.612736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916529+0000","last_change":"2023-10-18T09:16:26.510858+0000","last_active":"2023-10-18T09:17:20.916529+0000","last_peered":"2023-10-18T09:17:20.916529+0000","last_clean":"2023-10-18T09:17:20.916529+0000","last_became_active":"2023-10-18T09:16:26.509795+0000","last_became_peered":"2023-10-18T09:16:26.509795+0000","last_unstale":"2023-10-18T09:17:20.916529+0000","last_undegraded":"2023-10-18T09:17:20.916529+0000","last_fullsized":"2023-10-18T09:17:20.916529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:01.458412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.909859+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:17:20.909859+0000","last_peered":"2023-10-18T09:17:20.909859+0000","last_clean":"2023-10-18T09:17:20.909859+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:17:20.909859+0000","last_undegraded":"2023-10-18T09:17:20.909859+0000","last_fullsized":"2023-10-18T09:17:20.909859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.917244+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:17:20.917244+0000","last_peered":"2023-10-18T09:17:20.917244+0000","last_clean":"2023-10-18T09:17:20.917244+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:17:20.917244+0000","last_undegraded":"2023-10-18T09:17:20.917244+0000","last_fullsized":"2023-10-18T09:17:20.917244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908198+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:17:20.908198+0000","last_peered":"2023-10-18T09:17:20.908198+0000","last_clean":"2023-10-18T09:17:20.908198+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:17:20.908198+0000","last_undegraded":"2023-10-18T09:17:20.908198+0000","last_fullsized":"2023-10-18T09:17:20.908198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910759+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:17:20.910759+0000","last_peered":"2023-10-18T09:17:20.910759+0000","last_clean":"2023-10-18T09:17:20.910759+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:17:20.910759+0000","last_undegraded":"2023-10-18T09:17:20.910759+0000","last_fullsized":"2023-10-18T09:17:20.910759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912692+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:17:20.912692+0000","last_peered":"2023-10-18T09:17:20.912692+0000","last_clean":"2023-10-18T09:17:20.912692+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:17:20.912692+0000","last_undegraded":"2023-10-18T09:17:20.912692+0000","last_fullsized":"2023-10-18T09:17:20.912692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915269+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:17:20.915269+0000","last_peered":"2023-10-18T09:17:20.915269+0000","last_clean":"2023-10-18T09:17:20.915269+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:17:20.915269+0000","last_undegraded":"2023-10-18T09:17:20.915269+0000","last_fullsized":"2023-10-18T09:17:20.915269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910036+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:17:20.910036+0000","last_peered":"2023-10-18T09:17:20.910036+0000","last_clean":"2023-10-18T09:17:20.910036+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:17:20.910036+0000","last_undegraded":"2023-10-18T09:17:20.910036+0000","last_fullsized":"2023-10-18T09:17:20.910036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912660+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:17:20.912660+0000","last_peered":"2023-10-18T09:17:20.912660+0000","last_clean":"2023-10-18T09:17:20.912660+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:17:20.912660+0000","last_undegraded":"2023-10-18T09:17:20.912660+0000","last_fullsized":"2023-10-18T09:17:20.912660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915375+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:17:20.915375+0000","last_peered":"2023-10-18T09:17:20.915375+0000","last_clean":"2023-10-18T09:17:20.915375+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:17:20.915375+0000","last_undegraded":"2023-10-18T09:17:20.915375+0000","last_fullsized":"2023-10-18T09:17:20.915375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908304+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:17:20.908304+0000","last_peered":"2023-10-18T09:17:20.908304+0000","last_clean":"2023-10-18T09:17:20.908304+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:17:20.908304+0000","last_undegraded":"2023-10-18T09:17:20.908304+0000","last_fullsized":"2023-10-18T09:17:20.908304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:42.590984+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:17:42.590984+0000","last_peered":"2023-10-18T09:17:42.590984+0000","last_clean":"2023-10-18T09:17:42.590984+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:17:42.590984+0000","last_undegraded":"2023-10-18T09:17:42.590984+0000","last_fullsized":"2023-10-18T09:17:42.590984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912856+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:17:20.912856+0000","last_peered":"2023-10-18T09:17:20.912856+0000","last_clean":"2023-10-18T09:17:20.912856+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:17:20.912856+0000","last_undegraded":"2023-10-18T09:17:20.912856+0000","last_fullsized":"2023-10-18T09:17:20.912856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908723+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:17:20.908723+0000","last_peered":"2023-10-18T09:17:20.908723+0000","last_clean":"2023-10-18T09:17:20.908723+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:17:20.908723+0000","last_undegraded":"2023-10-18T09:17:20.908723+0000","last_fullsized":"2023-10-18T09:17:20.908723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910590+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:17:20.910590+0000","last_peered":"2023-10-18T09:17:20.910590+0000","last_clean":"2023-10-18T09:17:20.910590+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:17:20.910590+0000","last_undegraded":"2023-10-18T09:17:20.910590+0000","last_fullsized":"2023-10-18T09:17:20.910590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915220+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:17:20.915220+0000","last_peered":"2023-10-18T09:17:20.915220+0000","last_clean":"2023-10-18T09:17:20.915220+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:17:20.915220+0000","last_undegraded":"2023-10-18T09:17:20.915220+0000","last_fullsized":"2023-10-18T09:17:20.915220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.918712+0000","last_change":"2023-10-18T09:17:21.918712+0000","last_active":"2023-10-18T09:17:21.918712+0000","last_peered":"2023-10-18T09:17:21.918712+0000","last_clean":"2023-10-18T09:17:21.918712+0000","last_became_active":"2023-10-18T09:17:21.918229+0000","last_became_peered":"2023-10-18T09:17:21.918229+0000","last_unstale":"2023-10-18T09:17:21.918712+0000","last_undegraded":"2023-10-18T09:17:21.918712+0000","last_fullsized":"2023-10-18T09:17:21.918712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:31.948581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910075+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:17:20.910075+0000","last_peered":"2023-10-18T09:17:20.910075+0000","last_clean":"2023-10-18T09:17:20.910075+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:17:20.910075+0000","last_undegraded":"2023-10-18T09:17:20.910075+0000","last_fullsized":"2023-10-18T09:17:20.910075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915731+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:17:20.915731+0000","last_peered":"2023-10-18T09:17:20.915731+0000","last_clean":"2023-10-18T09:17:20.915731+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:17:20.915731+0000","last_undegraded":"2023-10-18T09:17:20.915731+0000","last_fullsized":"2023-10-18T09:17:20.915731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907977+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:17:20.907977+0000","last_peered":"2023-10-18T09:17:20.907977+0000","last_clean":"2023-10-18T09:17:20.907977+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:17:20.907977+0000","last_undegraded":"2023-10-18T09:17:20.907977+0000","last_fullsized":"2023-10-18T09:17:20.907977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914251+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:17:20.914251+0000","last_peered":"2023-10-18T09:17:20.914251+0000","last_clean":"2023-10-18T09:17:20.914251+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:17:20.914251+0000","last_undegraded":"2023-10-18T09:17:20.914251+0000","last_fullsized":"2023-10-18T09:17:20.914251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914003+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:17:20.914003+0000","last_peered":"2023-10-18T09:17:20.914003+0000","last_clean":"2023-10-18T09:17:20.914003+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:17:20.914003+0000","last_undegraded":"2023-10-18T09:17:20.914003+0000","last_fullsized":"2023-10-18T09:17:20.914003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916212+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:17:20.916212+0000","last_peered":"2023-10-18T09:17:20.916212+0000","last_clean":"2023-10-18T09:17:20.916212+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:17:20.916212+0000","last_undegraded":"2023-10-18T09:17:20.916212+0000","last_fullsized":"2023-10-18T09:17:20.916212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.595206+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:17:45.595206+0000","last_peered":"2023-10-18T09:17:45.595206+0000","last_clean":"2023-10-18T09:17:45.595206+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:17:45.595206+0000","last_undegraded":"2023-10-18T09:17:45.595206+0000","last_fullsized":"2023-10-18T09:17:45.595206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915035+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:17:20.915035+0000","last_peered":"2023-10-18T09:17:20.915035+0000","last_clean":"2023-10-18T09:17:20.915035+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:17:20.915035+0000","last_undegraded":"2023-10-18T09:17:20.915035+0000","last_fullsized":"2023-10-18T09:17:20.915035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914752+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:17:20.914752+0000","last_peered":"2023-10-18T09:17:20.914752+0000","last_clean":"2023-10-18T09:17:20.914752+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:17:20.914752+0000","last_undegraded":"2023-10-18T09:17:20.914752+0000","last_fullsized":"2023-10-18T09:17:20.914752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907847+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:17:20.907847+0000","last_peered":"2023-10-18T09:17:20.907847+0000","last_clean":"2023-10-18T09:17:20.907847+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:17:20.907847+0000","last_undegraded":"2023-10-18T09:17:20.907847+0000","last_fullsized":"2023-10-18T09:17:20.907847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908279+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:17:20.908279+0000","last_peered":"2023-10-18T09:17:20.908279+0000","last_clean":"2023-10-18T09:17:20.908279+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:17:20.908279+0000","last_undegraded":"2023-10-18T09:17:20.908279+0000","last_fullsized":"2023-10-18T09:17:20.908279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911042+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:17:20.911042+0000","last_peered":"2023-10-18T09:17:20.911042+0000","last_clean":"2023-10-18T09:17:20.911042+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:17:20.911042+0000","last_undegraded":"2023-10-18T09:17:20.911042+0000","last_fullsized":"2023-10-18T09:17:20.911042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915361+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:17:20.915361+0000","last_peered":"2023-10-18T09:17:20.915361+0000","last_clean":"2023-10-18T09:17:20.915361+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:17:20.915361+0000","last_undegraded":"2023-10-18T09:17:20.915361+0000","last_fullsized":"2023-10-18T09:17:20.915361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.921019+0000","last_change":"2023-10-18T09:17:21.921019+0000","last_active":"2023-10-18T09:17:21.921019+0000","last_peered":"2023-10-18T09:17:21.921019+0000","last_clean":"2023-10-18T09:17:21.921019+0000","last_became_active":"2023-10-18T09:17:21.920611+0000","last_became_peered":"2023-10-18T09:17:21.920611+0000","last_unstale":"2023-10-18T09:17:21.921019+0000","last_undegraded":"2023-10-18T09:17:21.921019+0000","last_fullsized":"2023-10-18T09:17:21.921019+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:44.317766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914179+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:17:20.914179+0000","last_peered":"2023-10-18T09:17:20.914179+0000","last_clean":"2023-10-18T09:17:20.914179+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:17:20.914179+0000","last_undegraded":"2023-10-18T09:17:20.914179+0000","last_fullsized":"2023-10-18T09:17:20.914179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.602661+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:17:45.602661+0000","last_peered":"2023-10-18T09:17:45.602661+0000","last_clean":"2023-10-18T09:17:45.602661+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:17:45.602661+0000","last_undegraded":"2023-10-18T09:17:45.602661+0000","last_fullsized":"2023-10-18T09:17:45.602661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":114,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912254+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:17:20.912254+0000","last_peered":"2023-10-18T09:17:20.912254+0000","last_clean":"2023-10-18T09:17:20.912254+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:17:20.912254+0000","last_undegraded":"2023-10-18T09:17:20.912254+0000","last_fullsized":"2023-10-18T09:17:20.912254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915653+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:17:20.915653+0000","last_peered":"2023-10-18T09:17:20.915653+0000","last_clean":"2023-10-18T09:17:20.915653+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:17:20.915653+0000","last_undegraded":"2023-10-18T09:17:20.915653+0000","last_fullsized":"2023-10-18T09:17:20.915653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.956756+0000","last_change":"2023-10-18T09:17:21.956449+0000","last_active":"2023-10-18T09:17:21.956756+0000","last_peered":"2023-10-18T09:17:21.956756+0000","last_clean":"2023-10-18T09:17:21.956756+0000","last_became_active":"2023-10-18T09:17:21.946784+0000","last_became_peered":"2023-10-18T09:17:21.946784+0000","last_unstale":"2023-10-18T09:17:21.956756+0000","last_undegraded":"2023-10-18T09:17:21.956756+0000","last_fullsized":"2023-10-18T09:17:21.956756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:49:21.415006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910722+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:17:20.910722+0000","last_peered":"2023-10-18T09:17:20.910722+0000","last_clean":"2023-10-18T09:17:20.910722+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:17:20.910722+0000","last_undegraded":"2023-10-18T09:17:20.910722+0000","last_fullsized":"2023-10-18T09:17:20.910722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910532+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:17:20.910532+0000","last_peered":"2023-10-18T09:17:20.910532+0000","last_clean":"2023-10-18T09:17:20.910532+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:17:20.910532+0000","last_undegraded":"2023-10-18T09:17:20.910532+0000","last_fullsized":"2023-10-18T09:17:20.910532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914932+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:17:20.914932+0000","last_peered":"2023-10-18T09:17:20.914932+0000","last_clean":"2023-10-18T09:17:20.914932+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:17:20.914932+0000","last_undegraded":"2023-10-18T09:17:20.914932+0000","last_fullsized":"2023-10-18T09:17:20.914932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907657+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:17:20.907657+0000","last_peered":"2023-10-18T09:17:20.907657+0000","last_clean":"2023-10-18T09:17:20.907657+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:17:20.907657+0000","last_undegraded":"2023-10-18T09:17:20.907657+0000","last_fullsized":"2023-10-18T09:17:20.907657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.927162+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:17:20.927162+0000","last_peered":"2023-10-18T09:17:20.927162+0000","last_clean":"2023-10-18T09:17:20.927162+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:17:20.927162+0000","last_undegraded":"2023-10-18T09:17:20.927162+0000","last_fullsized":"2023-10-18T09:17:20.927162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:42.612129+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:17:42.612129+0000","last_peered":"2023-10-18T09:17:42.612129+0000","last_clean":"2023-10-18T09:17:42.612129+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:17:42.612129+0000","last_undegraded":"2023-10-18T09:17:42.612129+0000","last_fullsized":"2023-10-18T09:17:42.612129+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907404+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:17:20.907404+0000","last_peered":"2023-10-18T09:17:20.907404+0000","last_clean":"2023-10-18T09:17:20.907404+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:17:20.907404+0000","last_undegraded":"2023-10-18T09:17:20.907404+0000","last_fullsized":"2023-10-18T09:17:20.907404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.948883+0000","last_change":"2023-10-18T09:17:21.948883+0000","last_active":"2023-10-18T09:17:21.948883+0000","last_peered":"2023-10-18T09:17:21.948883+0000","last_clean":"2023-10-18T09:17:21.948883+0000","last_became_active":"2023-10-18T09:17:21.948451+0000","last_became_peered":"2023-10-18T09:17:21.948451+0000","last_unstale":"2023-10-18T09:17:21.948883+0000","last_undegraded":"2023-10-18T09:17:21.948883+0000","last_fullsized":"2023-10-18T09:17:21.948883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:09:25.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912023+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:17:20.912023+0000","last_peered":"2023-10-18T09:17:20.912023+0000","last_clean":"2023-10-18T09:17:20.912023+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:17:20.912023+0000","last_undegraded":"2023-10-18T09:17:20.912023+0000","last_fullsized":"2023-10-18T09:17:20.912023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914569+0000","last_change":"2023-10-18T09:16:26.516754+0000","last_active":"2023-10-18T09:17:20.914569+0000","last_peered":"2023-10-18T09:17:20.914569+0000","last_clean":"2023-10-18T09:17:20.914569+0000","last_became_active":"2023-10-18T09:16:26.516350+0000","last_became_peered":"2023-10-18T09:16:26.516350+0000","last_unstale":"2023-10-18T09:17:20.914569+0000","last_undegraded":"2023-10-18T09:17:20.914569+0000","last_fullsized":"2023-10-18T09:17:20.914569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:37.396217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914370+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:17:20.914370+0000","last_peered":"2023-10-18T09:17:20.914370+0000","last_clean":"2023-10-18T09:17:20.914370+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:17:20.914370+0000","last_undegraded":"2023-10-18T09:17:20.914370+0000","last_fullsized":"2023-10-18T09:17:20.914370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912990+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:17:20.912990+0000","last_peered":"2023-10-18T09:17:20.912990+0000","last_clean":"2023-10-18T09:17:20.912990+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:17:20.912990+0000","last_undegraded":"2023-10-18T09:17:20.912990+0000","last_fullsized":"2023-10-18T09:17:20.912990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.919795+0000","last_change":"2023-10-18T09:17:21.919795+0000","last_active":"2023-10-18T09:17:21.919795+0000","last_peered":"2023-10-18T09:17:21.919795+0000","last_clean":"2023-10-18T09:17:21.919795+0000","last_became_active":"2023-10-18T09:17:21.919419+0000","last_became_peered":"2023-10-18T09:17:21.919419+0000","last_unstale":"2023-10-18T09:17:21.919795+0000","last_undegraded":"2023-10-18T09:17:21.919795+0000","last_fullsized":"2023-10-18T09:17:21.919795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:45.138506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912659+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:17:20.912659+0000","last_peered":"2023-10-18T09:17:20.912659+0000","last_clean":"2023-10-18T09:17:20.912659+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:17:20.912659+0000","last_undegraded":"2023-10-18T09:17:20.912659+0000","last_fullsized":"2023-10-18T09:17:20.912659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.926889+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:17:20.926889+0000","last_peered":"2023-10-18T09:17:20.926889+0000","last_clean":"2023-10-18T09:17:20.926889+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:17:20.926889+0000","last_undegraded":"2023-10-18T09:17:20.926889+0000","last_fullsized":"2023-10-18T09:17:20.926889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911127+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:17:20.911127+0000","last_peered":"2023-10-18T09:17:20.911127+0000","last_clean":"2023-10-18T09:17:20.911127+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:17:20.911127+0000","last_undegraded":"2023-10-18T09:17:20.911127+0000","last_fullsized":"2023-10-18T09:17:20.911127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911336+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:17:20.911336+0000","last_peered":"2023-10-18T09:17:20.911336+0000","last_clean":"2023-10-18T09:17:20.911336+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:17:20.911336+0000","last_undegraded":"2023-10-18T09:17:20.911336+0000","last_fullsized":"2023-10-18T09:17:20.911336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.616202+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:17:45.616202+0000","last_peered":"2023-10-18T09:17:45.616202+0000","last_clean":"2023-10-18T09:17:45.616202+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:17:45.616202+0000","last_undegraded":"2023-10-18T09:17:45.616202+0000","last_fullsized":"2023-10-18T09:17:45.616202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.920260+0000","last_change":"2023-10-18T09:17:21.920260+0000","last_active":"2023-10-18T09:17:21.920260+0000","last_peered":"2023-10-18T09:17:21.920260+0000","last_clean":"2023-10-18T09:17:21.920260+0000","last_became_active":"2023-10-18T09:17:21.919905+0000","last_became_peered":"2023-10-18T09:17:21.919905+0000","last_unstale":"2023-10-18T09:17:21.920260+0000","last_undegraded":"2023-10-18T09:17:21.920260+0000","last_fullsized":"2023-10-18T09:17:21.920260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:29.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912205+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:17:20.912205+0000","last_peered":"2023-10-18T09:17:20.912205+0000","last_clean":"2023-10-18T09:17:20.912205+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:17:20.912205+0000","last_undegraded":"2023-10-18T09:17:20.912205+0000","last_fullsized":"2023-10-18T09:17:20.912205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914363+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:17:20.914363+0000","last_peered":"2023-10-18T09:17:20.914363+0000","last_clean":"2023-10-18T09:17:20.914363+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:17:20.914363+0000","last_undegraded":"2023-10-18T09:17:20.914363+0000","last_fullsized":"2023-10-18T09:17:20.914363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:44.614259+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:17:44.614259+0000","last_peered":"2023-10-18T09:17:44.614259+0000","last_clean":"2023-10-18T09:17:44.614259+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:17:44.614259+0000","last_undegraded":"2023-10-18T09:17:44.614259+0000","last_fullsized":"2023-10-18T09:17:44.614259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907827+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:17:20.907827+0000","last_peered":"2023-10-18T09:17:20.907827+0000","last_clean":"2023-10-18T09:17:20.907827+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:17:20.907827+0000","last_undegraded":"2023-10-18T09:17:20.907827+0000","last_fullsized":"2023-10-18T09:17:20.907827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915166+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:17:20.915166+0000","last_peered":"2023-10-18T09:17:20.915166+0000","last_clean":"2023-10-18T09:17:20.915166+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:17:20.915166+0000","last_undegraded":"2023-10-18T09:17:20.915166+0000","last_fullsized":"2023-10-18T09:17:20.915166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910943+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:17:20.910943+0000","last_peered":"2023-10-18T09:17:20.910943+0000","last_clean":"2023-10-18T09:17:20.910943+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:17:20.910943+0000","last_undegraded":"2023-10-18T09:17:20.910943+0000","last_fullsized":"2023-10-18T09:17:20.910943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":313,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907261+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:17:20.907261+0000","last_peered":"2023-10-18T09:17:20.907261+0000","last_clean":"2023-10-18T09:17:20.907261+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:17:20.907261+0000","last_undegraded":"2023-10-18T09:17:20.907261+0000","last_fullsized":"2023-10-18T09:17:20.907261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915560+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:17:20.915560+0000","last_peered":"2023-10-18T09:17:20.915560+0000","last_clean":"2023-10-18T09:17:20.915560+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:17:20.915560+0000","last_undegraded":"2023-10-18T09:17:20.915560+0000","last_fullsized":"2023-10-18T09:17:20.915560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.909952+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:17:20.909952+0000","last_peered":"2023-10-18T09:17:20.909952+0000","last_clean":"2023-10-18T09:17:20.909952+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:17:20.909952+0000","last_undegraded":"2023-10-18T09:17:20.909952+0000","last_fullsized":"2023-10-18T09:17:20.909952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907356+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:17:20.907356+0000","last_peered":"2023-10-18T09:17:20.907356+0000","last_clean":"2023-10-18T09:17:20.907356+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:17:20.907356+0000","last_undegraded":"2023-10-18T09:17:20.907356+0000","last_fullsized":"2023-10-18T09:17:20.907356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910440+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:17:20.910440+0000","last_peered":"2023-10-18T09:17:20.910440+0000","last_clean":"2023-10-18T09:17:20.910440+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:17:20.910440+0000","last_undegraded":"2023-10-18T09:17:20.910440+0000","last_fullsized":"2023-10-18T09:17:20.910440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912392+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:17:20.912392+0000","last_peered":"2023-10-18T09:17:20.912392+0000","last_clean":"2023-10-18T09:17:20.912392+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:17:20.912392+0000","last_undegraded":"2023-10-18T09:17:20.912392+0000","last_fullsized":"2023-10-18T09:17:20.912392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911433+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:17:20.911433+0000","last_peered":"2023-10-18T09:17:20.911433+0000","last_clean":"2023-10-18T09:17:20.911433+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:17:20.911433+0000","last_undegraded":"2023-10-18T09:17:20.911433+0000","last_fullsized":"2023-10-18T09:17:20.911433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915649+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:17:20.915649+0000","last_peered":"2023-10-18T09:17:20.915649+0000","last_clean":"2023-10-18T09:17:20.915649+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:17:20.915649+0000","last_undegraded":"2023-10-18T09:17:20.915649+0000","last_fullsized":"2023-10-18T09:17:20.915649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.962634+0000","last_change":"2023-10-18T09:17:21.962634+0000","last_active":"2023-10-18T09:17:21.962634+0000","last_peered":"2023-10-18T09:17:21.962634+0000","last_clean":"2023-10-18T09:17:21.962634+0000","last_became_active":"2023-10-18T09:17:21.948090+0000","last_became_peered":"2023-10-18T09:17:21.948090+0000","last_unstale":"2023-10-18T09:17:21.962634+0000","last_undegraded":"2023-10-18T09:17:21.962634+0000","last_fullsized":"2023-10-18T09:17:21.962634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:12:14.390673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916530+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:17:20.916530+0000","last_peered":"2023-10-18T09:17:20.916530+0000","last_clean":"2023-10-18T09:17:20.916530+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:17:20.916530+0000","last_undegraded":"2023-10-18T09:17:20.916530+0000","last_fullsized":"2023-10-18T09:17:20.916530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907147+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:17:20.907147+0000","last_peered":"2023-10-18T09:17:20.907147+0000","last_clean":"2023-10-18T09:17:20.907147+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:17:20.907147+0000","last_undegraded":"2023-10-18T09:17:20.907147+0000","last_fullsized":"2023-10-18T09:17:20.907147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912586+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:17:20.912586+0000","last_peered":"2023-10-18T09:17:20.912586+0000","last_clean":"2023-10-18T09:17:20.912586+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:17:20.912586+0000","last_undegraded":"2023-10-18T09:17:20.912586+0000","last_fullsized":"2023-10-18T09:17:20.912586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908145+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:17:20.908145+0000","last_peered":"2023-10-18T09:17:20.908145+0000","last_clean":"2023-10-18T09:17:20.908145+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:17:20.908145+0000","last_undegraded":"2023-10-18T09:17:20.908145+0000","last_fullsized":"2023-10-18T09:17:20.908145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911151+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:17:20.911151+0000","last_peered":"2023-10-18T09:17:20.911151+0000","last_clean":"2023-10-18T09:17:20.911151+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:17:20.911151+0000","last_undegraded":"2023-10-18T09:17:20.911151+0000","last_fullsized":"2023-10-18T09:17:20.911151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914513+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:17:20.914513+0000","last_peered":"2023-10-18T09:17:20.914513+0000","last_clean":"2023-10-18T09:17:20.914513+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:17:20.914513+0000","last_undegraded":"2023-10-18T09:17:20.914513+0000","last_fullsized":"2023-10-18T09:17:20.914513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912014+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:17:20.912014+0000","last_peered":"2023-10-18T09:17:20.912014+0000","last_clean":"2023-10-18T09:17:20.912014+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:17:20.912014+0000","last_undegraded":"2023-10-18T09:17:20.912014+0000","last_fullsized":"2023-10-18T09:17:20.912014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.949440+0000","last_change":"2023-10-18T09:17:21.949440+0000","last_active":"2023-10-18T09:17:21.949440+0000","last_peered":"2023-10-18T09:17:21.949440+0000","last_clean":"2023-10-18T09:17:21.949440+0000","last_became_active":"2023-10-18T09:17:21.948562+0000","last_became_peered":"2023-10-18T09:17:21.948562+0000","last_unstale":"2023-10-18T09:17:21.949440+0000","last_undegraded":"2023-10-18T09:17:21.949440+0000","last_fullsized":"2023-10-18T09:17:21.949440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:20.976200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915361+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:17:20.915361+0000","last_peered":"2023-10-18T09:17:20.915361+0000","last_clean":"2023-10-18T09:17:20.915361+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:17:20.915361+0000","last_undegraded":"2023-10-18T09:17:20.915361+0000","last_fullsized":"2023-10-18T09:17:20.915361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910177+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:17:20.910177+0000","last_peered":"2023-10-18T09:17:20.910177+0000","last_clean":"2023-10-18T09:17:20.910177+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:17:20.910177+0000","last_undegraded":"2023-10-18T09:17:20.910177+0000","last_fullsized":"2023-10-18T09:17:20.910177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.957405+0000","last_change":"2023-10-18T09:17:21.957405+0000","last_active":"2023-10-18T09:17:21.957405+0000","last_peered":"2023-10-18T09:17:21.957405+0000","last_clean":"2023-10-18T09:17:21.957405+0000","last_became_active":"2023-10-18T09:17:21.917875+0000","last_became_peered":"2023-10-18T09:17:21.917875+0000","last_unstale":"2023-10-18T09:17:21.957405+0000","last_undegraded":"2023-10-18T09:17:21.957405+0000","last_fullsized":"2023-10-18T09:17:21.957405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:37.429104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907200+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:17:20.907200+0000","last_peered":"2023-10-18T09:17:20.907200+0000","last_clean":"2023-10-18T09:17:20.907200+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:17:20.907200+0000","last_undegraded":"2023-10-18T09:17:20.907200+0000","last_fullsized":"2023-10-18T09:17:20.907200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915131+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:17:20.915131+0000","last_peered":"2023-10-18T09:17:20.915131+0000","last_clean":"2023-10-18T09:17:20.915131+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:17:20.915131+0000","last_undegraded":"2023-10-18T09:17:20.915131+0000","last_fullsized":"2023-10-18T09:17:20.915131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910003+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:17:20.910003+0000","last_peered":"2023-10-18T09:17:20.910003+0000","last_clean":"2023-10-18T09:17:20.910003+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:17:20.910003+0000","last_undegraded":"2023-10-18T09:17:20.910003+0000","last_fullsized":"2023-10-18T09:17:20.910003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.913122+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:17:20.913122+0000","last_peered":"2023-10-18T09:17:20.913122+0000","last_clean":"2023-10-18T09:17:20.913122+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:17:20.913122+0000","last_undegraded":"2023-10-18T09:17:20.913122+0000","last_fullsized":"2023-10-18T09:17:20.913122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907606+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:17:20.907606+0000","last_peered":"2023-10-18T09:17:20.907606+0000","last_clean":"2023-10-18T09:17:20.907606+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:17:20.907606+0000","last_undegraded":"2023-10-18T09:17:20.907606+0000","last_fullsized":"2023-10-18T09:17:20.907606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914602+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:17:20.914602+0000","last_peered":"2023-10-18T09:17:20.914602+0000","last_clean":"2023-10-18T09:17:20.914602+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:17:20.914602+0000","last_undegraded":"2023-10-18T09:17:20.914602+0000","last_fullsized":"2023-10-18T09:17:20.914602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:43.614181+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:17:43.614181+0000","last_peered":"2023-10-18T09:17:43.614181+0000","last_clean":"2023-10-18T09:17:43.614181+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:17:43.614181+0000","last_undegraded":"2023-10-18T09:17:43.614181+0000","last_fullsized":"2023-10-18T09:17:43.614181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908011+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:17:20.908011+0000","last_peered":"2023-10-18T09:17:20.908011+0000","last_clean":"2023-10-18T09:17:20.908011+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:17:20.908011+0000","last_undegraded":"2023-10-18T09:17:20.908011+0000","last_fullsized":"2023-10-18T09:17:20.908011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907534+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:17:20.907534+0000","last_peered":"2023-10-18T09:17:20.907534+0000","last_clean":"2023-10-18T09:17:20.907534+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:17:20.907534+0000","last_undegraded":"2023-10-18T09:17:20.907534+0000","last_fullsized":"2023-10-18T09:17:20.907534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915473+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:17:20.915473+0000","last_peered":"2023-10-18T09:17:20.915473+0000","last_clean":"2023-10-18T09:17:20.915473+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:17:20.915473+0000","last_undegraded":"2023-10-18T09:17:20.915473+0000","last_fullsized":"2023-10-18T09:17:20.915473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907762+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:17:20.907762+0000","last_peered":"2023-10-18T09:17:20.907762+0000","last_clean":"2023-10-18T09:17:20.907762+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:17:20.907762+0000","last_undegraded":"2023-10-18T09:17:20.907762+0000","last_fullsized":"2023-10-18T09:17:20.907762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914050+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:17:20.914050+0000","last_peered":"2023-10-18T09:17:20.914050+0000","last_clean":"2023-10-18T09:17:20.914050+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:17:20.914050+0000","last_undegraded":"2023-10-18T09:17:20.914050+0000","last_fullsized":"2023-10-18T09:17:20.914050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.966321+0000","last_change":"2023-10-18T09:17:21.966321+0000","last_active":"2023-10-18T09:17:21.966321+0000","last_peered":"2023-10-18T09:17:21.966321+0000","last_clean":"2023-10-18T09:17:21.966321+0000","last_became_active":"2023-10-18T09:17:21.949466+0000","last_became_peered":"2023-10-18T09:17:21.949466+0000","last_unstale":"2023-10-18T09:17:21.966321+0000","last_undegraded":"2023-10-18T09:17:21.966321+0000","last_fullsized":"2023-10-18T09:17:21.966321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:05.856643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908122+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:17:20.908122+0000","last_peered":"2023-10-18T09:17:20.908122+0000","last_clean":"2023-10-18T09:17:20.908122+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:17:20.908122+0000","last_undegraded":"2023-10-18T09:17:20.908122+0000","last_fullsized":"2023-10-18T09:17:20.908122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910263+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:17:20.910263+0000","last_peered":"2023-10-18T09:17:20.910263+0000","last_clean":"2023-10-18T09:17:20.910263+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:17:20.910263+0000","last_undegraded":"2023-10-18T09:17:20.910263+0000","last_fullsized":"2023-10-18T09:17:20.910263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907350+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:17:20.907350+0000","last_peered":"2023-10-18T09:17:20.907350+0000","last_clean":"2023-10-18T09:17:20.907350+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:17:20.907350+0000","last_undegraded":"2023-10-18T09:17:20.907350+0000","last_fullsized":"2023-10-18T09:17:20.907350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907292+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:17:20.907292+0000","last_peered":"2023-10-18T09:17:20.907292+0000","last_clean":"2023-10-18T09:17:20.907292+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:17:20.907292+0000","last_undegraded":"2023-10-18T09:17:20.907292+0000","last_fullsized":"2023-10-18T09:17:20.907292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907831+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:17:20.907831+0000","last_peered":"2023-10-18T09:17:20.907831+0000","last_clean":"2023-10-18T09:17:20.907831+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:17:20.907831+0000","last_undegraded":"2023-10-18T09:17:20.907831+0000","last_fullsized":"2023-10-18T09:17:20.907831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911225+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:17:20.911225+0000","last_peered":"2023-10-18T09:17:20.911225+0000","last_clean":"2023-10-18T09:17:20.911225+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:17:20.911225+0000","last_undegraded":"2023-10-18T09:17:20.911225+0000","last_fullsized":"2023-10-18T09:17:20.911225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911060+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:17:20.911060+0000","last_peered":"2023-10-18T09:17:20.911060+0000","last_clean":"2023-10-18T09:17:20.911060+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:17:20.911060+0000","last_undegraded":"2023-10-18T09:17:20.911060+0000","last_fullsized":"2023-10-18T09:17:20.911060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908166+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:17:20.908166+0000","last_peered":"2023-10-18T09:17:20.908166+0000","last_clean":"2023-10-18T09:17:20.908166+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:17:20.908166+0000","last_undegraded":"2023-10-18T09:17:20.908166+0000","last_fullsized":"2023-10-18T09:17:20.908166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916678+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:17:20.916678+0000","last_peered":"2023-10-18T09:17:20.916678+0000","last_clean":"2023-10-18T09:17:20.916678+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:17:20.916678+0000","last_undegraded":"2023-10-18T09:17:20.916678+0000","last_fullsized":"2023-10-18T09:17:20.916678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914252+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:17:20.914252+0000","last_peered":"2023-10-18T09:17:20.914252+0000","last_clean":"2023-10-18T09:17:20.914252+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:17:20.914252+0000","last_undegraded":"2023-10-18T09:17:20.914252+0000","last_fullsized":"2023-10-18T09:17:20.914252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916033+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:17:20.916033+0000","last_peered":"2023-10-18T09:17:20.916033+0000","last_clean":"2023-10-18T09:17:20.916033+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:17:20.916033+0000","last_undegraded":"2023-10-18T09:17:20.916033+0000","last_fullsized":"2023-10-18T09:17:20.916033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914903+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:17:20.914903+0000","last_peered":"2023-10-18T09:17:20.914903+0000","last_clean":"2023-10-18T09:17:20.914903+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:17:20.914903+0000","last_undegraded":"2023-10-18T09:17:20.914903+0000","last_fullsized":"2023-10-18T09:17:20.914903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911608+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:17:20.911608+0000","last_peered":"2023-10-18T09:17:20.911608+0000","last_clean":"2023-10-18T09:17:20.911608+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:17:20.911608+0000","last_undegraded":"2023-10-18T09:17:20.911608+0000","last_fullsized":"2023-10-18T09:17:20.911608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911159+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:17:20.911159+0000","last_peered":"2023-10-18T09:17:20.911159+0000","last_clean":"2023-10-18T09:17:20.911159+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:17:20.911159+0000","last_undegraded":"2023-10-18T09:17:20.911159+0000","last_fullsized":"2023-10-18T09:17:20.911159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":516,"num_read_kb":335,"num_write":239,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.83199999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.625}]},{"osd":2,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.504},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.89500000000000002,"15min":0.81000000000000005},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.055,"5min":13.055,"15min":13.055},"last":0.81299999999999994}]},{"osd":7,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.008,"5min":1.407,"15min":1.407},"last":0.85099999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.133,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.052,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.107,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83799999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1142891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66300000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.434,"15min":0.438},"min":{"1min":0.29899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77000000000000002,"5min":1.196,"15min":1.196},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83399999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81200000000000006,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76200000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95499999999999996,"5min":1.242,"15min":1.242},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83199999999999996,"5min":1.141,"15min":1.141},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.93400000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89900000000000002,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95699999999999996,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86499999999999999,"5min":1.532,"15min":1.532},"last":0.87}]},{"osd":11,"last update":"Wed Oct 18 09:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90700000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":491409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.58099999999999996,"15min":0.55100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.78200000000000003,"15min":0.76500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.84999999999999998,"15min":0.82499999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.65700000000000003}]},{"osd":4,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.56100000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.5190000000000001,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.89000000000000001}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.107,"5min":0.98199999999999998,"15min":0.96099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.76100000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.99199999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.99399999999999999,"15min":0.96599999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.4130000000000003,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.83799999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.98199999999999998,"15min":0.95899999999999996},"min":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":6.3529999999999998,"5min":6.3529999999999998,"15min":6.3529999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.1000000000000001,"5min":0.99299999999999999,"15min":0.97499999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.87}]},{"osd":11,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.92300000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070506,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":491787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80100000000000005}]},{"osd":2,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.38200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.63400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65100000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.755},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.877}]},{"osd":5,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71799999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.77600000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":1.254,"15min":1.254},"last":0.83599999999999997}]},{"osd":9,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":1.071,"15min":1.071},"last":0.86399999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.92400000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.90000000000000002}]}]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1088342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41499999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.438,"15min":0.41999999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72399999999999998,"5min":0.876,"15min":0.876},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.378}]},{"osd":5,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63600000000000001,"15min":0.60299999999999998},"min":{"1min":0.501,"5min":0.433,"15min":0.433},"max":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.999,"5min":1.173,"15min":1.173},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66300000000000003,"15min":0.64600000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.56100000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.871,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98799999999999999,"5min":0.998,"15min":0.998},"last":0.627},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84399999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.753,"15min":0.76400000000000001},"min":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92300000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67400000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.75900000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45800000000000002,"15min":0.434},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0169999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0469999999999999,"5min":1.145,"15min":1.145},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91500000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97399999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.298,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.91500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.88100000000000001,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98499999999999999,"5min":11.112,"15min":11.112},"last":0.879},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.627}]},{"osd":7,"last update":"Wed Oct 18 09:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.68100000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0840000000000001,"5min":1.125,"15min":1.125},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2529999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.85599999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.67300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.7549999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1519999999999999,"5min":1.218,"15min":1.218},"last":0.78300000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.82999999999999996,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.847,"5min":2.847,"15min":2.847},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed Oct 18 09:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":548384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.55900000000000005,"15min":0.51500000000000001},"min":{"1min":0.42799999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.42599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.436}]},{"osd":2,"last update":"Wed Oct 18 09:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.51700000000000002},"min":{"1min":0.42599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.498,"5min":0.379,"15min":0.379},"max":{"1min":0.85099999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.55300000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.42899999999999999,"15min":0.40000000000000002},"min":{"1min":0.34100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.58899999999999997,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65000000000000002,"15min":0.58599999999999997},"min":{"1min":0.46600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.88200000000000001,"5min":5.258,"15min":5.258},"last":0.623}]},{"osd":6,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.069,"15min":1.069},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.083,"15min":1.083},"last":0.66400000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63700000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68000000000000005,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70199999999999996,"15min":0.68200000000000005},"min":{"1min":0.52200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68899999999999995}]},{"osd":9,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.72199999999999998,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.76400000000000001,"15min":0.753},"min":{"1min":0.61599999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.80100000000000005}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55700000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65200000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":488369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.60899999999999999,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.314,"15min":0.314},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.58899999999999997,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66200000000000003,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1,"5min":1.506,"15min":1.506},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.60499999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.88100000000000001,"15min":1.0720000000000001},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91400000000000003,"5min":11.539,"15min":11.539},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.86699999999999999,"15min":1.0489999999999999},"min":{"1min":0.42499999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.93600000000000005,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":1.554,"15min":1.554},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73299999999999998,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0389999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.51000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.96699999999999997}]},{"osd":7,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8540000000000001,"5min":2.8540000000000001,"15min":2.8540000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.028}]},{"osd":8,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.873,"15min":0.874},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.996},{"interface":"front","average":{"1min":1.155,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.94899999999999995}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.01},{"interface":"front","average":{"1min":0.873,"5min":0.92600000000000005,"15min":0.93500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.0569999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96099999999999997,"15min":0.95999999999999996},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.508,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.125},{"interface":"front","average":{"1min":1.077,"5min":0.92400000000000004,"15min":0.89800000000000002},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.98099999999999998}]},{"osd":11,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.95999999999999996,"15min":0.91800000000000004},"min":{"1min":0.68100000000000005,"5min":0.625,"15min":0.625},"max":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.075,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6919999999999999,"5min":1.698,"15min":1.698},"last":1.0800000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":548653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.59899999999999998,"15min":0.61499999999999999},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.56299999999999994}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.58399999999999996,"15min":0.52700000000000002},"min":{"1min":0.47099999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.53700000000000003},"min":{"1min":0.495,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.66100000000000003,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.315,"15min":0.315},"max":{"1min":7.2140000000000004,"5min":7.2140000000000004,"15min":7.2140000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.443,"15min":0.40699999999999997},"min":{"1min":0.36399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.51700000000000002,"15min":0.45400000000000001},"min":{"1min":0.38500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88800000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.91100000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.101,"15min":1.101},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82399999999999995}]},{"osd":7,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.89900000000000002}]},{"osd":8,"last update":"Wed Oct 18 09:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.54500000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.97999999999999998}]},{"osd":9,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.1000000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.006,"5min":1.206,"15min":1.206},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.79500000000000004,"15min":0.77900000000000003},"min":{"1min":0.62,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1180000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.026}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.872},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.05}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":486621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.47499999999999998,"15min":0.40600000000000003},"min":{"1min":0.38200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.498,"15min":0.433},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.73799999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":7.516,"5min":7.516,"15min":7.516},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.65800000000000003,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.68500000000000005,"15min":0.64200000000000002},"min":{"1min":0.61299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.65300000000000002},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64200000000000002,"15min":0.59499999999999997},"min":{"1min":0.45400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82199999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.60199999999999998},"min":{"1min":0.51100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69999999999999996,"15min":0.65200000000000002},"min":{"1min":0.622,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.442,"15min":0.442},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.151,"5min":1.234,"15min":1.234},"last":0.70599999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82099999999999995},"min":{"1min":0.66300000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.96799999999999997,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.877}]},{"osd":10,"last update":"Wed Oct 18 09:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.016,"5min":1.286,"15min":1.286},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.745,"5min":0.626,"15min":0.626},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.85899999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":7.6769999999999996,"5min":7.6769999999999996,"15min":7.6769999999999996},"last":0.91500000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.36599999999999999},"min":{"1min":0.316,"5min":0.186,"15min":0.186},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.39000000000000001},"min":{"1min":0.34200000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.70699999999999996,"5min":0.753,"15min":0.753},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.499,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.627}]},{"osd":3,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.50900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.077,"15min":1.077},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.59799999999999998},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92900000000000005,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.56000000000000005},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.58499999999999996},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.871,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65700000000000003,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92400000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.131,"15min":1.131},"last":0.73999999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92200000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.67200000000000004}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95399999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.89900000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.75800000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.745,"15min":0.745},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.871},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.85599999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1083865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.40000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.375},"min":{"1min":0.317,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.39500000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.54500000000000004},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.74099999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72699999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80900000000000005,"5min":0.88,"15min":0.88},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.51600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.80400000000000005}]},{"osd":8,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.83699999999999997}]},{"osd":9,"last update":"Wed Oct 18 09:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.89900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.63100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.874},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.56100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.85099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:17:49.523 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-18T09:17:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:49 smithi110 ceph-mon[134818]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:50.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:49 smithi061 ceph-mon[98573]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:50.272 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:50.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:49 smithi170 ceph-mon[135150]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:51.123 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:50 smithi061 ceph-mon[98573]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:50 smithi110 ceph-mon[134818]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:50 smithi170 ceph-mon[135150]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:51.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 -- 172.21.15.61:0/3391943629 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1057c0 msgr2=0x7fec8c105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:51.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 --2- 172.21.15.61:0/3391943629 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1057c0 0x7fec8c105ba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009990 tx=0x7fec7c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 -- 172.21.15.61:0/3391943629 shutdown_connections 2023-10-18T09:17:51.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 --2- 172.21.15.61:0/3391943629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c101260 0x7fec8c10d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 --2- 172.21.15.61:0/3391943629 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1057c0 0x7fec8c105ba0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 --2- 172.21.15.61:0/3391943629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c1008a0 0x7fec8c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 -- 172.21.15.61:0/3391943629 >> 172.21.15.61:0/3391943629 conn(0x7fec8c0779e0 msgr2=0x7fec8c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:51.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 -- 172.21.15.61:0/3391943629 shutdown_connections 2023-10-18T09:17:51.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.781+0000 7fec912bf700 1 -- 172.21.15.61:0/3391943629 wait complete. 2023-10-18T09:17:51.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 Processor -- start 2023-10-18T09:17:51.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 -- start start 2023-10-18T09:17:51.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:51.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c101260 0x7fec8c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:51.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c1057c0 0x7fec8c196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:51.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c1970e0 con 0x7fec8c1008a0 2023-10-18T09:17:51.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c197250 con 0x7fec8c101260 2023-10-18T09:17:51.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.782+0000 7fec912bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c197410 con 0x7fec8c1057c0 2023-10-18T09:17:51.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:51.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56180/0 (socket says 172.21.15.61:56180) 2023-10-18T09:17:51.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 -- 172.21.15.61:0/2842957871 learned_addr learned my addr 172.21.15.61:0/2842957871 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:51.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8a7fc700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c101260 0x7fec8c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:51.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8b7fe700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c1057c0 0x7fec8c196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 -- 172.21.15.61:0/2842957871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c1057c0 msgr2=0x7fec8c196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c1057c0 0x7fec8c196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 -- 172.21.15.61:0/2842957871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c101260 msgr2=0x7fec8c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c101260 0x7fec8c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.783+0000 7fec8affd700 1 -- 172.21.15.61:0/2842957871 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0095f0 con 0x7fec8c1008a0 2023-10-18T09:17:51.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec8affd700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fec7c000c00 tx=0x7fec7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:51.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c01d070 con 0x7fec8c1008a0 2023-10-18T09:17:51.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec7c021410 con 0x7fec8c1008a0 2023-10-18T09:17:51.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c197690 con 0x7fec8c1008a0 2023-10-18T09:17:51.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c017650 con 0x7fec8c1008a0 2023-10-18T09:17:51.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.784+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c1a3930 con 0x7fec8c1008a0 2023-10-18T09:17:51.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.785+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c108d80 con 0x7fec8c1008a0 2023-10-18T09:17:51.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.787+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fec7c021580 con 0x7fec8c1008a0 2023-10-18T09:17:51.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.788+0000 7fec73fff700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 0x7fec740a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:51.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.789+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec7c0cea70 con 0x7fec8c1008a0 2023-10-18T09:17:51.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.789+0000 7fec8a7fc700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 0x7fec740a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:51.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.789+0000 7fec8a7fc700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 0x7fec740a3a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec8c197c40 tx=0x7fec80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:51.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.791+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fec7c097c90 con 0x7fec8c1008a0 2023-10-18T09:17:52.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:51.996+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec8c197f40 con 0x7fec740a1590 2023-10-18T09:17:52.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.010+0000 7fec73fff700 1 -- 172.21.15.61:0/2842957871 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426567 (secure 0 0 0) 0x7fec8c197f40 con 0x7fec740a1590 2023-10-18T09:17:52.021 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:52.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.016+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 msgr2=0x7fec740a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:52.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.016+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 0x7fec740a3a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec8c197c40 tx=0x7fec80009260 comp rx=0 tx=0).stop 2023-10-18T09:17:52.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.016+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 msgr2=0x7fec8c19c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:52.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fec7c000c00 tx=0x7fec7c004730 comp rx=0 tx=0).stop 2023-10-18T09:17:52.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 shutdown_connections 2023-10-18T09:17:52.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fec740a1590 0x7fec740a3a50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:52.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c1057c0 0x7fec8c196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:52.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fec8c1008a0 0x7fec8c19c940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:52.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 --2- 172.21.15.61:0/2842957871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec8c101260 0x7fec8c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:52.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 >> 172.21.15.61:0/2842957871 conn(0x7fec8c0779e0 msgr2=0x7fec8c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:52.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 shutdown_connections 2023-10-18T09:17:52.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:52.017+0000 7fec912bf700 1 -- 172.21.15.61:0/2842957871 wait complete. 2023-10-18T09:17:52.027 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:17:52.191 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:51 smithi061 ceph-mon[98573]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:51 smithi110 ceph-mon[134818]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:52.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:51 smithi170 ceph-mon[135150]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:17:52.476 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2023-10-18T09:17:50.853651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658257,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":662,"num_read_kb":449,"num_write":422,"num_write_kb":3078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180927,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338232,"kb_used_data":13976,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603592,"statfs":{"total":1151940427776,"available":1151594078208,"internally_reserved":0,"allocated":14311424,"data_stored":7840158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911071+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:17:20.911071+0000","last_peered":"2023-10-18T09:17:20.911071+0000","last_clean":"2023-10-18T09:17:20.911071+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:17:20.911071+0000","last_undegraded":"2023-10-18T09:17:20.911071+0000","last_fullsized":"2023-10-18T09:17:20.911071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908066+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:17:20.908066+0000","last_peered":"2023-10-18T09:17:20.908066+0000","last_clean":"2023-10-18T09:17:20.908066+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:17:20.908066+0000","last_undegraded":"2023-10-18T09:17:20.908066+0000","last_fullsized":"2023-10-18T09:17:20.908066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907956+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:17:20.907956+0000","last_peered":"2023-10-18T09:17:20.907956+0000","last_clean":"2023-10-18T09:17:20.907956+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:17:20.907956+0000","last_undegraded":"2023-10-18T09:17:20.907956+0000","last_fullsized":"2023-10-18T09:17:20.907956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910940+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:17:20.910940+0000","last_peered":"2023-10-18T09:17:20.910940+0000","last_clean":"2023-10-18T09:17:20.910940+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:17:20.910940+0000","last_undegraded":"2023-10-18T09:17:20.910940+0000","last_fullsized":"2023-10-18T09:17:20.910940+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912977+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:17:20.912977+0000","last_peered":"2023-10-18T09:17:20.912977+0000","last_clean":"2023-10-18T09:17:20.912977+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:17:20.912977+0000","last_undegraded":"2023-10-18T09:17:20.912977+0000","last_fullsized":"2023-10-18T09:17:20.912977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910640+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:17:20.910640+0000","last_peered":"2023-10-18T09:17:20.910640+0000","last_clean":"2023-10-18T09:17:20.910640+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:17:20.910640+0000","last_undegraded":"2023-10-18T09:17:20.910640+0000","last_fullsized":"2023-10-18T09:17:20.910640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907372+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:17:20.907372+0000","last_peered":"2023-10-18T09:17:20.907372+0000","last_clean":"2023-10-18T09:17:20.907372+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:17:20.907372+0000","last_undegraded":"2023-10-18T09:17:20.907372+0000","last_fullsized":"2023-10-18T09:17:20.907372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915563+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:17:20.915563+0000","last_peered":"2023-10-18T09:17:20.915563+0000","last_clean":"2023-10-18T09:17:20.915563+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:17:20.915563+0000","last_undegraded":"2023-10-18T09:17:20.915563+0000","last_fullsized":"2023-10-18T09:17:20.915563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915035+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:17:20.915035+0000","last_peered":"2023-10-18T09:17:20.915035+0000","last_clean":"2023-10-18T09:17:20.915035+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:17:20.915035+0000","last_undegraded":"2023-10-18T09:17:20.915035+0000","last_fullsized":"2023-10-18T09:17:20.915035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911700+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:17:20.911700+0000","last_peered":"2023-10-18T09:17:20.911700+0000","last_clean":"2023-10-18T09:17:20.911700+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:17:20.911700+0000","last_undegraded":"2023-10-18T09:17:20.911700+0000","last_fullsized":"2023-10-18T09:17:20.911700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907951+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:17:20.907951+0000","last_peered":"2023-10-18T09:17:20.907951+0000","last_clean":"2023-10-18T09:17:20.907951+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:17:20.907951+0000","last_undegraded":"2023-10-18T09:17:20.907951+0000","last_fullsized":"2023-10-18T09:17:20.907951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910831+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:17:20.910831+0000","last_peered":"2023-10-18T09:17:20.910831+0000","last_clean":"2023-10-18T09:17:20.910831+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:17:20.910831+0000","last_undegraded":"2023-10-18T09:17:20.910831+0000","last_fullsized":"2023-10-18T09:17:20.910831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911541+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:17:20.911541+0000","last_peered":"2023-10-18T09:17:20.911541+0000","last_clean":"2023-10-18T09:17:20.911541+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:17:20.911541+0000","last_undegraded":"2023-10-18T09:17:20.911541+0000","last_fullsized":"2023-10-18T09:17:20.911541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907597+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:17:20.907597+0000","last_peered":"2023-10-18T09:17:20.907597+0000","last_clean":"2023-10-18T09:17:20.907597+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:17:20.907597+0000","last_undegraded":"2023-10-18T09:17:20.907597+0000","last_fullsized":"2023-10-18T09:17:20.907597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908148+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:17:20.908148+0000","last_peered":"2023-10-18T09:17:20.908148+0000","last_clean":"2023-10-18T09:17:20.908148+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:17:20.908148+0000","last_undegraded":"2023-10-18T09:17:20.908148+0000","last_fullsized":"2023-10-18T09:17:20.908148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914434+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:17:20.914434+0000","last_peered":"2023-10-18T09:17:20.914434+0000","last_clean":"2023-10-18T09:17:20.914434+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:17:20.914434+0000","last_undegraded":"2023-10-18T09:17:20.914434+0000","last_fullsized":"2023-10-18T09:17:20.914434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911035+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:17:20.911035+0000","last_peered":"2023-10-18T09:17:20.911035+0000","last_clean":"2023-10-18T09:17:20.911035+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:17:20.911035+0000","last_undegraded":"2023-10-18T09:17:20.911035+0000","last_fullsized":"2023-10-18T09:17:20.911035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.927737+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:17:20.927737+0000","last_peered":"2023-10-18T09:17:20.927737+0000","last_clean":"2023-10-18T09:17:20.927737+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:17:20.927737+0000","last_undegraded":"2023-10-18T09:17:20.927737+0000","last_fullsized":"2023-10-18T09:17:20.927737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908526+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:17:20.908526+0000","last_peered":"2023-10-18T09:17:20.908526+0000","last_clean":"2023-10-18T09:17:20.908526+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:17:20.908526+0000","last_undegraded":"2023-10-18T09:17:20.908526+0000","last_fullsized":"2023-10-18T09:17:20.908526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907197+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:17:20.907197+0000","last_peered":"2023-10-18T09:17:20.907197+0000","last_clean":"2023-10-18T09:17:20.907197+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:17:20.907197+0000","last_undegraded":"2023-10-18T09:17:20.907197+0000","last_fullsized":"2023-10-18T09:17:20.907197+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:48.619422+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:17:48.619422+0000","last_peered":"2023-10-18T09:17:48.619422+0000","last_clean":"2023-10-18T09:17:48.619422+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:17:48.619422+0000","last_undegraded":"2023-10-18T09:17:48.619422+0000","last_fullsized":"2023-10-18T09:17:48.619422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916529+0000","last_change":"2023-10-18T09:16:26.510858+0000","last_active":"2023-10-18T09:17:20.916529+0000","last_peered":"2023-10-18T09:17:20.916529+0000","last_clean":"2023-10-18T09:17:20.916529+0000","last_became_active":"2023-10-18T09:16:26.509795+0000","last_became_peered":"2023-10-18T09:16:26.509795+0000","last_unstale":"2023-10-18T09:17:20.916529+0000","last_undegraded":"2023-10-18T09:17:20.916529+0000","last_fullsized":"2023-10-18T09:17:20.916529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:56:01.458412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.909859+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:17:20.909859+0000","last_peered":"2023-10-18T09:17:20.909859+0000","last_clean":"2023-10-18T09:17:20.909859+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:17:20.909859+0000","last_undegraded":"2023-10-18T09:17:20.909859+0000","last_fullsized":"2023-10-18T09:17:20.909859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.917244+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:17:20.917244+0000","last_peered":"2023-10-18T09:17:20.917244+0000","last_clean":"2023-10-18T09:17:20.917244+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:17:20.917244+0000","last_undegraded":"2023-10-18T09:17:20.917244+0000","last_fullsized":"2023-10-18T09:17:20.917244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908198+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:17:20.908198+0000","last_peered":"2023-10-18T09:17:20.908198+0000","last_clean":"2023-10-18T09:17:20.908198+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:17:20.908198+0000","last_undegraded":"2023-10-18T09:17:20.908198+0000","last_fullsized":"2023-10-18T09:17:20.908198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910759+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:17:20.910759+0000","last_peered":"2023-10-18T09:17:20.910759+0000","last_clean":"2023-10-18T09:17:20.910759+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:17:20.910759+0000","last_undegraded":"2023-10-18T09:17:20.910759+0000","last_fullsized":"2023-10-18T09:17:20.910759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912692+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:17:20.912692+0000","last_peered":"2023-10-18T09:17:20.912692+0000","last_clean":"2023-10-18T09:17:20.912692+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:17:20.912692+0000","last_undegraded":"2023-10-18T09:17:20.912692+0000","last_fullsized":"2023-10-18T09:17:20.912692+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915269+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:17:20.915269+0000","last_peered":"2023-10-18T09:17:20.915269+0000","last_clean":"2023-10-18T09:17:20.915269+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:17:20.915269+0000","last_undegraded":"2023-10-18T09:17:20.915269+0000","last_fullsized":"2023-10-18T09:17:20.915269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910036+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:17:20.910036+0000","last_peered":"2023-10-18T09:17:20.910036+0000","last_clean":"2023-10-18T09:17:20.910036+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:17:20.910036+0000","last_undegraded":"2023-10-18T09:17:20.910036+0000","last_fullsized":"2023-10-18T09:17:20.910036+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912660+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:17:20.912660+0000","last_peered":"2023-10-18T09:17:20.912660+0000","last_clean":"2023-10-18T09:17:20.912660+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:17:20.912660+0000","last_undegraded":"2023-10-18T09:17:20.912660+0000","last_fullsized":"2023-10-18T09:17:20.912660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915375+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:17:20.915375+0000","last_peered":"2023-10-18T09:17:20.915375+0000","last_clean":"2023-10-18T09:17:20.915375+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:17:20.915375+0000","last_undegraded":"2023-10-18T09:17:20.915375+0000","last_fullsized":"2023-10-18T09:17:20.915375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908304+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:17:20.908304+0000","last_peered":"2023-10-18T09:17:20.908304+0000","last_clean":"2023-10-18T09:17:20.908304+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:17:20.908304+0000","last_undegraded":"2023-10-18T09:17:20.908304+0000","last_fullsized":"2023-10-18T09:17:20.908304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:47.599106+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:17:47.599106+0000","last_peered":"2023-10-18T09:17:47.599106+0000","last_clean":"2023-10-18T09:17:47.599106+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:17:47.599106+0000","last_undegraded":"2023-10-18T09:17:47.599106+0000","last_fullsized":"2023-10-18T09:17:47.599106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912856+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:17:20.912856+0000","last_peered":"2023-10-18T09:17:20.912856+0000","last_clean":"2023-10-18T09:17:20.912856+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:17:20.912856+0000","last_undegraded":"2023-10-18T09:17:20.912856+0000","last_fullsized":"2023-10-18T09:17:20.912856+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908723+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:17:20.908723+0000","last_peered":"2023-10-18T09:17:20.908723+0000","last_clean":"2023-10-18T09:17:20.908723+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:17:20.908723+0000","last_undegraded":"2023-10-18T09:17:20.908723+0000","last_fullsized":"2023-10-18T09:17:20.908723+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910590+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:17:20.910590+0000","last_peered":"2023-10-18T09:17:20.910590+0000","last_clean":"2023-10-18T09:17:20.910590+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:17:20.910590+0000","last_undegraded":"2023-10-18T09:17:20.910590+0000","last_fullsized":"2023-10-18T09:17:20.910590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915220+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:17:20.915220+0000","last_peered":"2023-10-18T09:17:20.915220+0000","last_clean":"2023-10-18T09:17:20.915220+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:17:20.915220+0000","last_undegraded":"2023-10-18T09:17:20.915220+0000","last_fullsized":"2023-10-18T09:17:20.915220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.918712+0000","last_change":"2023-10-18T09:17:21.918712+0000","last_active":"2023-10-18T09:17:21.918712+0000","last_peered":"2023-10-18T09:17:21.918712+0000","last_clean":"2023-10-18T09:17:21.918712+0000","last_became_active":"2023-10-18T09:17:21.918229+0000","last_became_peered":"2023-10-18T09:17:21.918229+0000","last_unstale":"2023-10-18T09:17:21.918712+0000","last_undegraded":"2023-10-18T09:17:21.918712+0000","last_fullsized":"2023-10-18T09:17:21.918712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:31.948581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910075+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:17:20.910075+0000","last_peered":"2023-10-18T09:17:20.910075+0000","last_clean":"2023-10-18T09:17:20.910075+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:17:20.910075+0000","last_undegraded":"2023-10-18T09:17:20.910075+0000","last_fullsized":"2023-10-18T09:17:20.910075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915731+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:17:20.915731+0000","last_peered":"2023-10-18T09:17:20.915731+0000","last_clean":"2023-10-18T09:17:20.915731+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:17:20.915731+0000","last_undegraded":"2023-10-18T09:17:20.915731+0000","last_fullsized":"2023-10-18T09:17:20.915731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907977+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:17:20.907977+0000","last_peered":"2023-10-18T09:17:20.907977+0000","last_clean":"2023-10-18T09:17:20.907977+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:17:20.907977+0000","last_undegraded":"2023-10-18T09:17:20.907977+0000","last_fullsized":"2023-10-18T09:17:20.907977+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914251+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:17:20.914251+0000","last_peered":"2023-10-18T09:17:20.914251+0000","last_clean":"2023-10-18T09:17:20.914251+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:17:20.914251+0000","last_undegraded":"2023-10-18T09:17:20.914251+0000","last_fullsized":"2023-10-18T09:17:20.914251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914003+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:17:20.914003+0000","last_peered":"2023-10-18T09:17:20.914003+0000","last_clean":"2023-10-18T09:17:20.914003+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:17:20.914003+0000","last_undegraded":"2023-10-18T09:17:20.914003+0000","last_fullsized":"2023-10-18T09:17:20.914003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916212+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:17:20.916212+0000","last_peered":"2023-10-18T09:17:20.916212+0000","last_clean":"2023-10-18T09:17:20.916212+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:17:20.916212+0000","last_undegraded":"2023-10-18T09:17:20.916212+0000","last_fullsized":"2023-10-18T09:17:20.916212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.595206+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:17:45.595206+0000","last_peered":"2023-10-18T09:17:45.595206+0000","last_clean":"2023-10-18T09:17:45.595206+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:17:45.595206+0000","last_undegraded":"2023-10-18T09:17:45.595206+0000","last_fullsized":"2023-10-18T09:17:45.595206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915035+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:17:20.915035+0000","last_peered":"2023-10-18T09:17:20.915035+0000","last_clean":"2023-10-18T09:17:20.915035+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:17:20.915035+0000","last_undegraded":"2023-10-18T09:17:20.915035+0000","last_fullsized":"2023-10-18T09:17:20.915035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914752+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:17:20.914752+0000","last_peered":"2023-10-18T09:17:20.914752+0000","last_clean":"2023-10-18T09:17:20.914752+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:17:20.914752+0000","last_undegraded":"2023-10-18T09:17:20.914752+0000","last_fullsized":"2023-10-18T09:17:20.914752+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907847+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:17:20.907847+0000","last_peered":"2023-10-18T09:17:20.907847+0000","last_clean":"2023-10-18T09:17:20.907847+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:17:20.907847+0000","last_undegraded":"2023-10-18T09:17:20.907847+0000","last_fullsized":"2023-10-18T09:17:20.907847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908279+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:17:20.908279+0000","last_peered":"2023-10-18T09:17:20.908279+0000","last_clean":"2023-10-18T09:17:20.908279+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:17:20.908279+0000","last_undegraded":"2023-10-18T09:17:20.908279+0000","last_fullsized":"2023-10-18T09:17:20.908279+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911042+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:17:20.911042+0000","last_peered":"2023-10-18T09:17:20.911042+0000","last_clean":"2023-10-18T09:17:20.911042+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:17:20.911042+0000","last_undegraded":"2023-10-18T09:17:20.911042+0000","last_fullsized":"2023-10-18T09:17:20.911042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915361+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:17:20.915361+0000","last_peered":"2023-10-18T09:17:20.915361+0000","last_clean":"2023-10-18T09:17:20.915361+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:17:20.915361+0000","last_undegraded":"2023-10-18T09:17:20.915361+0000","last_fullsized":"2023-10-18T09:17:20.915361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.921019+0000","last_change":"2023-10-18T09:17:21.921019+0000","last_active":"2023-10-18T09:17:21.921019+0000","last_peered":"2023-10-18T09:17:21.921019+0000","last_clean":"2023-10-18T09:17:21.921019+0000","last_became_active":"2023-10-18T09:17:21.920611+0000","last_became_peered":"2023-10-18T09:17:21.920611+0000","last_unstale":"2023-10-18T09:17:21.921019+0000","last_undegraded":"2023-10-18T09:17:21.921019+0000","last_fullsized":"2023-10-18T09:17:21.921019+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:44.317766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914179+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:17:20.914179+0000","last_peered":"2023-10-18T09:17:20.914179+0000","last_clean":"2023-10-18T09:17:20.914179+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:17:20.914179+0000","last_undegraded":"2023-10-18T09:17:20.914179+0000","last_fullsized":"2023-10-18T09:17:20.914179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'35","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.602661+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:17:45.602661+0000","last_peered":"2023-10-18T09:17:45.602661+0000","last_clean":"2023-10-18T09:17:45.602661+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:17:45.602661+0000","last_undegraded":"2023-10-18T09:17:45.602661+0000","last_fullsized":"2023-10-18T09:17:45.602661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60441,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":114,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912254+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:17:20.912254+0000","last_peered":"2023-10-18T09:17:20.912254+0000","last_clean":"2023-10-18T09:17:20.912254+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:17:20.912254+0000","last_undegraded":"2023-10-18T09:17:20.912254+0000","last_fullsized":"2023-10-18T09:17:20.912254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915653+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:17:20.915653+0000","last_peered":"2023-10-18T09:17:20.915653+0000","last_clean":"2023-10-18T09:17:20.915653+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:17:20.915653+0000","last_undegraded":"2023-10-18T09:17:20.915653+0000","last_fullsized":"2023-10-18T09:17:20.915653+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.956756+0000","last_change":"2023-10-18T09:17:21.956449+0000","last_active":"2023-10-18T09:17:21.956756+0000","last_peered":"2023-10-18T09:17:21.956756+0000","last_clean":"2023-10-18T09:17:21.956756+0000","last_became_active":"2023-10-18T09:17:21.946784+0000","last_became_peered":"2023-10-18T09:17:21.946784+0000","last_unstale":"2023-10-18T09:17:21.956756+0000","last_undegraded":"2023-10-18T09:17:21.956756+0000","last_fullsized":"2023-10-18T09:17:21.956756+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:49:21.415006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910722+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:17:20.910722+0000","last_peered":"2023-10-18T09:17:20.910722+0000","last_clean":"2023-10-18T09:17:20.910722+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:17:20.910722+0000","last_undegraded":"2023-10-18T09:17:20.910722+0000","last_fullsized":"2023-10-18T09:17:20.910722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910532+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:17:20.910532+0000","last_peered":"2023-10-18T09:17:20.910532+0000","last_clean":"2023-10-18T09:17:20.910532+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:17:20.910532+0000","last_undegraded":"2023-10-18T09:17:20.910532+0000","last_fullsized":"2023-10-18T09:17:20.910532+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914932+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:17:20.914932+0000","last_peered":"2023-10-18T09:17:20.914932+0000","last_clean":"2023-10-18T09:17:20.914932+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:17:20.914932+0000","last_undegraded":"2023-10-18T09:17:20.914932+0000","last_fullsized":"2023-10-18T09:17:20.914932+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907657+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:17:20.907657+0000","last_peered":"2023-10-18T09:17:20.907657+0000","last_clean":"2023-10-18T09:17:20.907657+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:17:20.907657+0000","last_undegraded":"2023-10-18T09:17:20.907657+0000","last_fullsized":"2023-10-18T09:17:20.907657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.927162+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:17:20.927162+0000","last_peered":"2023-10-18T09:17:20.927162+0000","last_clean":"2023-10-18T09:17:20.927162+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:17:20.927162+0000","last_undegraded":"2023-10-18T09:17:20.927162+0000","last_fullsized":"2023-10-18T09:17:20.927162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:47.618997+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:17:47.618997+0000","last_peered":"2023-10-18T09:17:47.618997+0000","last_clean":"2023-10-18T09:17:47.618997+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:17:47.618997+0000","last_undegraded":"2023-10-18T09:17:47.618997+0000","last_fullsized":"2023-10-18T09:17:47.618997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907404+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:17:20.907404+0000","last_peered":"2023-10-18T09:17:20.907404+0000","last_clean":"2023-10-18T09:17:20.907404+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:17:20.907404+0000","last_undegraded":"2023-10-18T09:17:20.907404+0000","last_fullsized":"2023-10-18T09:17:20.907404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.948883+0000","last_change":"2023-10-18T09:17:21.948883+0000","last_active":"2023-10-18T09:17:21.948883+0000","last_peered":"2023-10-18T09:17:21.948883+0000","last_clean":"2023-10-18T09:17:21.948883+0000","last_became_active":"2023-10-18T09:17:21.948451+0000","last_became_peered":"2023-10-18T09:17:21.948451+0000","last_unstale":"2023-10-18T09:17:21.948883+0000","last_undegraded":"2023-10-18T09:17:21.948883+0000","last_fullsized":"2023-10-18T09:17:21.948883+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:09:25.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912023+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:17:20.912023+0000","last_peered":"2023-10-18T09:17:20.912023+0000","last_clean":"2023-10-18T09:17:20.912023+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:17:20.912023+0000","last_undegraded":"2023-10-18T09:17:20.912023+0000","last_fullsized":"2023-10-18T09:17:20.912023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914569+0000","last_change":"2023-10-18T09:16:26.516754+0000","last_active":"2023-10-18T09:17:20.914569+0000","last_peered":"2023-10-18T09:17:20.914569+0000","last_clean":"2023-10-18T09:17:20.914569+0000","last_became_active":"2023-10-18T09:16:26.516350+0000","last_became_peered":"2023-10-18T09:16:26.516350+0000","last_unstale":"2023-10-18T09:17:20.914569+0000","last_undegraded":"2023-10-18T09:17:20.914569+0000","last_fullsized":"2023-10-18T09:17:20.914569+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:16:37.396217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914370+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:17:20.914370+0000","last_peered":"2023-10-18T09:17:20.914370+0000","last_clean":"2023-10-18T09:17:20.914370+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:17:20.914370+0000","last_undegraded":"2023-10-18T09:17:20.914370+0000","last_fullsized":"2023-10-18T09:17:20.914370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912990+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:17:20.912990+0000","last_peered":"2023-10-18T09:17:20.912990+0000","last_clean":"2023-10-18T09:17:20.912990+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:17:20.912990+0000","last_undegraded":"2023-10-18T09:17:20.912990+0000","last_fullsized":"2023-10-18T09:17:20.912990+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.919795+0000","last_change":"2023-10-18T09:17:21.919795+0000","last_active":"2023-10-18T09:17:21.919795+0000","last_peered":"2023-10-18T09:17:21.919795+0000","last_clean":"2023-10-18T09:17:21.919795+0000","last_became_active":"2023-10-18T09:17:21.919419+0000","last_became_peered":"2023-10-18T09:17:21.919419+0000","last_unstale":"2023-10-18T09:17:21.919795+0000","last_undegraded":"2023-10-18T09:17:21.919795+0000","last_fullsized":"2023-10-18T09:17:21.919795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:45.138506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912659+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:17:20.912659+0000","last_peered":"2023-10-18T09:17:20.912659+0000","last_clean":"2023-10-18T09:17:20.912659+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:17:20.912659+0000","last_undegraded":"2023-10-18T09:17:20.912659+0000","last_fullsized":"2023-10-18T09:17:20.912659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.926889+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:17:20.926889+0000","last_peered":"2023-10-18T09:17:20.926889+0000","last_clean":"2023-10-18T09:17:20.926889+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:17:20.926889+0000","last_undegraded":"2023-10-18T09:17:20.926889+0000","last_fullsized":"2023-10-18T09:17:20.926889+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911127+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:17:20.911127+0000","last_peered":"2023-10-18T09:17:20.911127+0000","last_clean":"2023-10-18T09:17:20.911127+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:17:20.911127+0000","last_undegraded":"2023-10-18T09:17:20.911127+0000","last_fullsized":"2023-10-18T09:17:20.911127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911336+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:17:20.911336+0000","last_peered":"2023-10-18T09:17:20.911336+0000","last_clean":"2023-10-18T09:17:20.911336+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:17:20.911336+0000","last_undegraded":"2023-10-18T09:17:20.911336+0000","last_fullsized":"2023-10-18T09:17:20.911336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:45.616202+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:17:45.616202+0000","last_peered":"2023-10-18T09:17:45.616202+0000","last_clean":"2023-10-18T09:17:45.616202+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:17:45.616202+0000","last_undegraded":"2023-10-18T09:17:45.616202+0000","last_fullsized":"2023-10-18T09:17:45.616202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.920260+0000","last_change":"2023-10-18T09:17:21.920260+0000","last_active":"2023-10-18T09:17:21.920260+0000","last_peered":"2023-10-18T09:17:21.920260+0000","last_clean":"2023-10-18T09:17:21.920260+0000","last_became_active":"2023-10-18T09:17:21.919905+0000","last_became_peered":"2023-10-18T09:17:21.919905+0000","last_unstale":"2023-10-18T09:17:21.920260+0000","last_undegraded":"2023-10-18T09:17:21.920260+0000","last_fullsized":"2023-10-18T09:17:21.920260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:29.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912205+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:17:20.912205+0000","last_peered":"2023-10-18T09:17:20.912205+0000","last_clean":"2023-10-18T09:17:20.912205+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:17:20.912205+0000","last_undegraded":"2023-10-18T09:17:20.912205+0000","last_fullsized":"2023-10-18T09:17:20.912205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914363+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:17:20.914363+0000","last_peered":"2023-10-18T09:17:20.914363+0000","last_clean":"2023-10-18T09:17:20.914363+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:17:20.914363+0000","last_undegraded":"2023-10-18T09:17:20.914363+0000","last_fullsized":"2023-10-18T09:17:20.914363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:44.614259+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:17:44.614259+0000","last_peered":"2023-10-18T09:17:44.614259+0000","last_clean":"2023-10-18T09:17:44.614259+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:17:44.614259+0000","last_undegraded":"2023-10-18T09:17:44.614259+0000","last_fullsized":"2023-10-18T09:17:44.614259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907827+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:17:20.907827+0000","last_peered":"2023-10-18T09:17:20.907827+0000","last_clean":"2023-10-18T09:17:20.907827+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:17:20.907827+0000","last_undegraded":"2023-10-18T09:17:20.907827+0000","last_fullsized":"2023-10-18T09:17:20.907827+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915166+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:17:20.915166+0000","last_peered":"2023-10-18T09:17:20.915166+0000","last_clean":"2023-10-18T09:17:20.915166+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:17:20.915166+0000","last_undegraded":"2023-10-18T09:17:20.915166+0000","last_fullsized":"2023-10-18T09:17:20.915166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910943+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:17:20.910943+0000","last_peered":"2023-10-18T09:17:20.910943+0000","last_clean":"2023-10-18T09:17:20.910943+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:17:20.910943+0000","last_undegraded":"2023-10-18T09:17:20.910943+0000","last_fullsized":"2023-10-18T09:17:20.910943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":313,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907261+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:17:20.907261+0000","last_peered":"2023-10-18T09:17:20.907261+0000","last_clean":"2023-10-18T09:17:20.907261+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:17:20.907261+0000","last_undegraded":"2023-10-18T09:17:20.907261+0000","last_fullsized":"2023-10-18T09:17:20.907261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915560+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:17:20.915560+0000","last_peered":"2023-10-18T09:17:20.915560+0000","last_clean":"2023-10-18T09:17:20.915560+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:17:20.915560+0000","last_undegraded":"2023-10-18T09:17:20.915560+0000","last_fullsized":"2023-10-18T09:17:20.915560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.909952+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:17:20.909952+0000","last_peered":"2023-10-18T09:17:20.909952+0000","last_clean":"2023-10-18T09:17:20.909952+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:17:20.909952+0000","last_undegraded":"2023-10-18T09:17:20.909952+0000","last_fullsized":"2023-10-18T09:17:20.909952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907356+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:17:20.907356+0000","last_peered":"2023-10-18T09:17:20.907356+0000","last_clean":"2023-10-18T09:17:20.907356+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:17:20.907356+0000","last_undegraded":"2023-10-18T09:17:20.907356+0000","last_fullsized":"2023-10-18T09:17:20.907356+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910440+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:17:20.910440+0000","last_peered":"2023-10-18T09:17:20.910440+0000","last_clean":"2023-10-18T09:17:20.910440+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:17:20.910440+0000","last_undegraded":"2023-10-18T09:17:20.910440+0000","last_fullsized":"2023-10-18T09:17:20.910440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912392+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:17:20.912392+0000","last_peered":"2023-10-18T09:17:20.912392+0000","last_clean":"2023-10-18T09:17:20.912392+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:17:20.912392+0000","last_undegraded":"2023-10-18T09:17:20.912392+0000","last_fullsized":"2023-10-18T09:17:20.912392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911433+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:17:20.911433+0000","last_peered":"2023-10-18T09:17:20.911433+0000","last_clean":"2023-10-18T09:17:20.911433+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:17:20.911433+0000","last_undegraded":"2023-10-18T09:17:20.911433+0000","last_fullsized":"2023-10-18T09:17:20.911433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915649+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:17:20.915649+0000","last_peered":"2023-10-18T09:17:20.915649+0000","last_clean":"2023-10-18T09:17:20.915649+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:17:20.915649+0000","last_undegraded":"2023-10-18T09:17:20.915649+0000","last_fullsized":"2023-10-18T09:17:20.915649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.962634+0000","last_change":"2023-10-18T09:17:21.962634+0000","last_active":"2023-10-18T09:17:21.962634+0000","last_peered":"2023-10-18T09:17:21.962634+0000","last_clean":"2023-10-18T09:17:21.962634+0000","last_became_active":"2023-10-18T09:17:21.948090+0000","last_became_peered":"2023-10-18T09:17:21.948090+0000","last_unstale":"2023-10-18T09:17:21.962634+0000","last_undegraded":"2023-10-18T09:17:21.962634+0000","last_fullsized":"2023-10-18T09:17:21.962634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:12:14.390673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916530+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:17:20.916530+0000","last_peered":"2023-10-18T09:17:20.916530+0000","last_clean":"2023-10-18T09:17:20.916530+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:17:20.916530+0000","last_undegraded":"2023-10-18T09:17:20.916530+0000","last_fullsized":"2023-10-18T09:17:20.916530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907147+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:17:20.907147+0000","last_peered":"2023-10-18T09:17:20.907147+0000","last_clean":"2023-10-18T09:17:20.907147+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:17:20.907147+0000","last_undegraded":"2023-10-18T09:17:20.907147+0000","last_fullsized":"2023-10-18T09:17:20.907147+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912586+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:17:20.912586+0000","last_peered":"2023-10-18T09:17:20.912586+0000","last_clean":"2023-10-18T09:17:20.912586+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:17:20.912586+0000","last_undegraded":"2023-10-18T09:17:20.912586+0000","last_fullsized":"2023-10-18T09:17:20.912586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908145+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:17:20.908145+0000","last_peered":"2023-10-18T09:17:20.908145+0000","last_clean":"2023-10-18T09:17:20.908145+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:17:20.908145+0000","last_undegraded":"2023-10-18T09:17:20.908145+0000","last_fullsized":"2023-10-18T09:17:20.908145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911151+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:17:20.911151+0000","last_peered":"2023-10-18T09:17:20.911151+0000","last_clean":"2023-10-18T09:17:20.911151+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:17:20.911151+0000","last_undegraded":"2023-10-18T09:17:20.911151+0000","last_fullsized":"2023-10-18T09:17:20.911151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914513+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:17:20.914513+0000","last_peered":"2023-10-18T09:17:20.914513+0000","last_clean":"2023-10-18T09:17:20.914513+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:17:20.914513+0000","last_undegraded":"2023-10-18T09:17:20.914513+0000","last_fullsized":"2023-10-18T09:17:20.914513+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.912014+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:17:20.912014+0000","last_peered":"2023-10-18T09:17:20.912014+0000","last_clean":"2023-10-18T09:17:20.912014+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:17:20.912014+0000","last_undegraded":"2023-10-18T09:17:20.912014+0000","last_fullsized":"2023-10-18T09:17:20.912014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.949440+0000","last_change":"2023-10-18T09:17:21.949440+0000","last_active":"2023-10-18T09:17:21.949440+0000","last_peered":"2023-10-18T09:17:21.949440+0000","last_clean":"2023-10-18T09:17:21.949440+0000","last_became_active":"2023-10-18T09:17:21.948562+0000","last_became_peered":"2023-10-18T09:17:21.948562+0000","last_unstale":"2023-10-18T09:17:21.949440+0000","last_undegraded":"2023-10-18T09:17:21.949440+0000","last_fullsized":"2023-10-18T09:17:21.949440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:20.976200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915361+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:17:20.915361+0000","last_peered":"2023-10-18T09:17:20.915361+0000","last_clean":"2023-10-18T09:17:20.915361+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:17:20.915361+0000","last_undegraded":"2023-10-18T09:17:20.915361+0000","last_fullsized":"2023-10-18T09:17:20.915361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910177+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:17:20.910177+0000","last_peered":"2023-10-18T09:17:20.910177+0000","last_clean":"2023-10-18T09:17:20.910177+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:17:20.910177+0000","last_undegraded":"2023-10-18T09:17:20.910177+0000","last_fullsized":"2023-10-18T09:17:20.910177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.957405+0000","last_change":"2023-10-18T09:17:21.957405+0000","last_active":"2023-10-18T09:17:21.957405+0000","last_peered":"2023-10-18T09:17:21.957405+0000","last_clean":"2023-10-18T09:17:21.957405+0000","last_became_active":"2023-10-18T09:17:21.917875+0000","last_became_peered":"2023-10-18T09:17:21.917875+0000","last_unstale":"2023-10-18T09:17:21.957405+0000","last_undegraded":"2023-10-18T09:17:21.957405+0000","last_fullsized":"2023-10-18T09:17:21.957405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:37.429104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907200+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:17:20.907200+0000","last_peered":"2023-10-18T09:17:20.907200+0000","last_clean":"2023-10-18T09:17:20.907200+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:17:20.907200+0000","last_undegraded":"2023-10-18T09:17:20.907200+0000","last_fullsized":"2023-10-18T09:17:20.907200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915131+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:17:20.915131+0000","last_peered":"2023-10-18T09:17:20.915131+0000","last_clean":"2023-10-18T09:17:20.915131+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:17:20.915131+0000","last_undegraded":"2023-10-18T09:17:20.915131+0000","last_fullsized":"2023-10-18T09:17:20.915131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910003+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:17:20.910003+0000","last_peered":"2023-10-18T09:17:20.910003+0000","last_clean":"2023-10-18T09:17:20.910003+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:17:20.910003+0000","last_undegraded":"2023-10-18T09:17:20.910003+0000","last_fullsized":"2023-10-18T09:17:20.910003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.913122+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:17:20.913122+0000","last_peered":"2023-10-18T09:17:20.913122+0000","last_clean":"2023-10-18T09:17:20.913122+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:17:20.913122+0000","last_undegraded":"2023-10-18T09:17:20.913122+0000","last_fullsized":"2023-10-18T09:17:20.913122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907606+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:17:20.907606+0000","last_peered":"2023-10-18T09:17:20.907606+0000","last_clean":"2023-10-18T09:17:20.907606+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:17:20.907606+0000","last_undegraded":"2023-10-18T09:17:20.907606+0000","last_fullsized":"2023-10-18T09:17:20.907606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914602+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:17:20.914602+0000","last_peered":"2023-10-18T09:17:20.914602+0000","last_clean":"2023-10-18T09:17:20.914602+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:17:20.914602+0000","last_undegraded":"2023-10-18T09:17:20.914602+0000","last_fullsized":"2023-10-18T09:17:20.914602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:48.621080+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:17:48.621080+0000","last_peered":"2023-10-18T09:17:48.621080+0000","last_clean":"2023-10-18T09:17:48.621080+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:17:48.621080+0000","last_undegraded":"2023-10-18T09:17:48.621080+0000","last_fullsized":"2023-10-18T09:17:48.621080+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908011+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:17:20.908011+0000","last_peered":"2023-10-18T09:17:20.908011+0000","last_clean":"2023-10-18T09:17:20.908011+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:17:20.908011+0000","last_undegraded":"2023-10-18T09:17:20.908011+0000","last_fullsized":"2023-10-18T09:17:20.908011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907534+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:17:20.907534+0000","last_peered":"2023-10-18T09:17:20.907534+0000","last_clean":"2023-10-18T09:17:20.907534+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:17:20.907534+0000","last_undegraded":"2023-10-18T09:17:20.907534+0000","last_fullsized":"2023-10-18T09:17:20.907534+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.915473+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:17:20.915473+0000","last_peered":"2023-10-18T09:17:20.915473+0000","last_clean":"2023-10-18T09:17:20.915473+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:17:20.915473+0000","last_undegraded":"2023-10-18T09:17:20.915473+0000","last_fullsized":"2023-10-18T09:17:20.915473+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907762+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:17:20.907762+0000","last_peered":"2023-10-18T09:17:20.907762+0000","last_clean":"2023-10-18T09:17:20.907762+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:17:20.907762+0000","last_undegraded":"2023-10-18T09:17:20.907762+0000","last_fullsized":"2023-10-18T09:17:20.907762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914050+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:17:20.914050+0000","last_peered":"2023-10-18T09:17:20.914050+0000","last_clean":"2023-10-18T09:17:20.914050+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:17:20.914050+0000","last_undegraded":"2023-10-18T09:17:20.914050+0000","last_fullsized":"2023-10-18T09:17:20.914050+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-18T09:17:21.966321+0000","last_change":"2023-10-18T09:17:21.966321+0000","last_active":"2023-10-18T09:17:21.966321+0000","last_peered":"2023-10-18T09:17:21.966321+0000","last_clean":"2023-10-18T09:17:21.966321+0000","last_became_active":"2023-10-18T09:17:21.949466+0000","last_became_peered":"2023-10-18T09:17:21.949466+0000","last_unstale":"2023-10-18T09:17:21.966321+0000","last_undegraded":"2023-10-18T09:17:21.966321+0000","last_fullsized":"2023-10-18T09:17:21.966321+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:05.856643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908122+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:17:20.908122+0000","last_peered":"2023-10-18T09:17:20.908122+0000","last_clean":"2023-10-18T09:17:20.908122+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:17:20.908122+0000","last_undegraded":"2023-10-18T09:17:20.908122+0000","last_fullsized":"2023-10-18T09:17:20.908122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.910263+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:17:20.910263+0000","last_peered":"2023-10-18T09:17:20.910263+0000","last_clean":"2023-10-18T09:17:20.910263+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:17:20.910263+0000","last_undegraded":"2023-10-18T09:17:20.910263+0000","last_fullsized":"2023-10-18T09:17:20.910263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907350+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:17:20.907350+0000","last_peered":"2023-10-18T09:17:20.907350+0000","last_clean":"2023-10-18T09:17:20.907350+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:17:20.907350+0000","last_undegraded":"2023-10-18T09:17:20.907350+0000","last_fullsized":"2023-10-18T09:17:20.907350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907292+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:17:20.907292+0000","last_peered":"2023-10-18T09:17:20.907292+0000","last_clean":"2023-10-18T09:17:20.907292+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:17:20.907292+0000","last_undegraded":"2023-10-18T09:17:20.907292+0000","last_fullsized":"2023-10-18T09:17:20.907292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.907831+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:17:20.907831+0000","last_peered":"2023-10-18T09:17:20.907831+0000","last_clean":"2023-10-18T09:17:20.907831+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:17:20.907831+0000","last_undegraded":"2023-10-18T09:17:20.907831+0000","last_fullsized":"2023-10-18T09:17:20.907831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911225+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:17:20.911225+0000","last_peered":"2023-10-18T09:17:20.911225+0000","last_clean":"2023-10-18T09:17:20.911225+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:17:20.911225+0000","last_undegraded":"2023-10-18T09:17:20.911225+0000","last_fullsized":"2023-10-18T09:17:20.911225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911060+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:17:20.911060+0000","last_peered":"2023-10-18T09:17:20.911060+0000","last_clean":"2023-10-18T09:17:20.911060+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:17:20.911060+0000","last_undegraded":"2023-10-18T09:17:20.911060+0000","last_fullsized":"2023-10-18T09:17:20.911060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.908166+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:17:20.908166+0000","last_peered":"2023-10-18T09:17:20.908166+0000","last_clean":"2023-10-18T09:17:20.908166+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:17:20.908166+0000","last_undegraded":"2023-10-18T09:17:20.908166+0000","last_fullsized":"2023-10-18T09:17:20.908166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916678+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:17:20.916678+0000","last_peered":"2023-10-18T09:17:20.916678+0000","last_clean":"2023-10-18T09:17:20.916678+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:17:20.916678+0000","last_undegraded":"2023-10-18T09:17:20.916678+0000","last_fullsized":"2023-10-18T09:17:20.916678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914252+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:17:20.914252+0000","last_peered":"2023-10-18T09:17:20.914252+0000","last_clean":"2023-10-18T09:17:20.914252+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:17:20.914252+0000","last_undegraded":"2023-10-18T09:17:20.914252+0000","last_fullsized":"2023-10-18T09:17:20.914252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.916033+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:17:20.916033+0000","last_peered":"2023-10-18T09:17:20.916033+0000","last_clean":"2023-10-18T09:17:20.916033+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:17:20.916033+0000","last_undegraded":"2023-10-18T09:17:20.916033+0000","last_fullsized":"2023-10-18T09:17:20.916033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.914903+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:17:20.914903+0000","last_peered":"2023-10-18T09:17:20.914903+0000","last_clean":"2023-10-18T09:17:20.914903+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:17:20.914903+0000","last_undegraded":"2023-10-18T09:17:20.914903+0000","last_fullsized":"2023-10-18T09:17:20.914903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911608+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:17:20.911608+0000","last_peered":"2023-10-18T09:17:20.911608+0000","last_clean":"2023-10-18T09:17:20.911608+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:17:20.911608+0000","last_undegraded":"2023-10-18T09:17:20.911608+0000","last_fullsized":"2023-10-18T09:17:20.911608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-18T09:17:20.911159+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:17:20.911159+0000","last_peered":"2023-10-18T09:17:20.911159+0000","last_clean":"2023-10-18T09:17:20.911159+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:17:20.911159+0000","last_undegraded":"2023-10-18T09:17:20.911159+0000","last_fullsized":"2023-10-18T09:17:20.911159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":536,"num_read_kb":340,"num_write":239,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.59899999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.83199999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.625}]},{"osd":2,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.504},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.89500000000000002,"15min":0.81000000000000005},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.055,"5min":13.055,"15min":13.055},"last":0.81299999999999994}]},{"osd":7,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.008,"5min":1.407,"15min":1.407},"last":0.85099999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.78600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.133,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.871},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.052,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.107,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83799999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1703936,"data_stored":1142891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66300000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.434,"15min":0.438},"min":{"1min":0.29899999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.55100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.41499999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77000000000000002,"5min":1.196,"15min":1.196},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.59399999999999997},"min":{"1min":0.42699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.83399999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81200000000000006,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63500000000000001,"15min":0.63100000000000001},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83299999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74299999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52600000000000002,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76200000000000001,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95499999999999996,"5min":1.242,"15min":1.242},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.83199999999999996,"5min":1.141,"15min":1.141},"last":0.755}]},{"osd":6,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.495,"5min":0.441,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.93000000000000005,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.85399999999999998}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.93400000000000005,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.51000000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89900000000000002,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.89400000000000002}]},{"osd":9,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.76900000000000002,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95699999999999996,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.86499999999999999,"5min":1.532,"15min":1.532},"last":0.87}]},{"osd":11,"last update":"Wed Oct 18 09:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90700000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.88400000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":491409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.58099999999999996,"15min":0.55100000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.1280000000000001,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.623},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.59699999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.68500000000000005,"15min":0.67100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":6.2610000000000001,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.78200000000000003,"15min":0.76500000000000001},"min":{"1min":0.47299999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.7269999999999999,"5min":3.7269999999999999,"15min":3.7269999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.495,"15min":0.495},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.69499999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.64600000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.84999999999999998,"15min":0.82499999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":6.5679999999999996,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.7719999999999998,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80900000000000005,"15min":0.80500000000000005},"min":{"1min":0.56100000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.7650000000000001,"5min":2.7650000000000001,"15min":2.7650000000000001},"last":0.51200000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.878,"15min":0.86199999999999999},"min":{"1min":0.56799999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":4.5190000000000001,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58499999999999996,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91300000000000003,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.755},{"interface":"front","average":{"1min":1.107,"5min":0.98199999999999998,"15min":0.96099999999999997},"min":{"1min":0.63800000000000001,"5min":0.63,"15min":0.63},"max":{"1min":6.3090000000000002,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.82399999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":0.99199999999999999,"15min":0.97799999999999998},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.1639999999999999,"5min":0.99399999999999999,"15min":0.96599999999999997},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":6.4130000000000003,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.874}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":0.98199999999999998,"15min":0.95899999999999996},"min":{"1min":0.71699999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":6.3529999999999998,"5min":6.3529999999999998,"15min":6.3529999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1000000000000001,"5min":0.99299999999999999,"15min":0.97499999999999998},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":4.8819999999999997,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.93200000000000005}]},{"osd":11,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":4.8339999999999996,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":1},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.81299999999999994}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":491787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.51900000000000002}]},{"osd":1,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48999999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.38200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.63400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65100000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.71599999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.80100000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71799999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.871},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.88}]},{"osd":7,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":1.254,"15min":1.254},"last":0.78100000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76300000000000001,"15min":0.76400000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95299999999999996,"5min":1.071,"15min":1.071},"last":0.93600000000000005}]},{"osd":10,"last update":"Wed Oct 18 09:17:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.76400000000000001,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.89800000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.92200000000000004}]}]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1626112,"data_stored":1088342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41499999999999998,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.438,"15min":0.41999999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.72399999999999998,"5min":0.876,"15min":0.876},"last":0.40400000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79200000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.46400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.40999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.57799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.30099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.378}]},{"osd":5,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.83799999999999997,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.63600000000000001,"15min":0.60299999999999998},"min":{"1min":0.501,"5min":0.433,"15min":0.433},"max":{"1min":0.94699999999999995,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.69399999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.999,"5min":1.173,"15min":1.173},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66300000000000003,"15min":0.64600000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60899999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:17:33 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.56100000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.871,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98799999999999999,"5min":0.998,"15min":0.998},"last":0.627},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.84399999999999997,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.753,"15min":0.76400000000000001},"min":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.92300000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.67400000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.75900000000000001}]}]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45800000000000002,"15min":0.434},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0169999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64500000000000002,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.2,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.64700000000000002,"15min":0.621},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0469999999999999,"5min":1.145,"15min":1.145},"last":0.65300000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.91500000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79900000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.86699999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61299999999999999,"15min":0.59199999999999997},"min":{"1min":0.44900000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97399999999999998,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.40899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.52000000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70899999999999996,"15min":0.68899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.298,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":1.0009999999999999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.91500000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:16:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.88100000000000001,"15min":0.79900000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98499999999999999,"5min":11.112,"15min":11.112},"last":0.879},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.627}]},{"osd":7,"last update":"Wed Oct 18 09:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.68100000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0840000000000001,"5min":1.125,"15min":1.125},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80200000000000005,"15min":0.79200000000000004},"min":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2529999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.85599999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.87,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.81899999999999995,"15min":0.80700000000000005},"min":{"1min":0.67300000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.7549999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.82099999999999995}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84899999999999998,"15min":0.85699999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.88200000000000001,"15min":0.88800000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1519999999999999,"5min":1.218,"15min":1.218},"last":0.78300000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.82999999999999996,"15min":0.81599999999999995},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.847,"5min":2.847,"15min":2.847},"last":0.64100000000000001}]},{"osd":11,"last update":"Wed Oct 18 09:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.77500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":548384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.55900000000000005,"15min":0.51500000000000001},"min":{"1min":0.42799999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.2779999999999996,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.42599999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.61899999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.56699999999999995,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.51700000000000002},"min":{"1min":0.42599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.56299999999999994,"15min":0.54100000000000004},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59999999999999998,"15min":0.57399999999999995},"min":{"1min":0.498,"5min":0.379,"15min":0.379},"max":{"1min":0.85099999999999998,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.86899999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.42899999999999999,"15min":0.40000000000000002},"min":{"1min":0.34100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.58899999999999997,"15min":0.54500000000000004},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.65000000000000002,"15min":0.58599999999999997},"min":{"1min":0.46600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.88200000000000001,"5min":5.258,"15min":5.258},"last":0.56699999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.60699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95399999999999996,"5min":1.069,"15min":1.069},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.083,"15min":1.083},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57299999999999995,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.625,"15min":0.61499999999999999},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.51500000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:17:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68000000000000005,"15min":0.64700000000000002},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70199999999999996,"15min":0.68200000000000005},"min":{"1min":0.52200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.74099999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.72199999999999998,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.76400000000000001,"15min":0.753},"min":{"1min":0.61599999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.76100000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.55700000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63400000000000001}]},{"osd":11,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1,"5min":1,"15min":1},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.78100000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":488369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.60899999999999999,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.314,"15min":0.314},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.58899999999999997,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Oct 18 09:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66200000000000003,"15min":0.60199999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1,"5min":1.506,"15min":1.506},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65200000000000002,"15min":0.60499999999999998},"min":{"1min":0.46200000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Oct 18 09:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.88100000000000001,"15min":1.0720000000000001},"min":{"1min":0.46500000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91400000000000003,"5min":11.539,"15min":11.539},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.86699999999999999,"15min":1.0489999999999999},"min":{"1min":0.42499999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.93600000000000005,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Wed Oct 18 09:17:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0569999999999999,"5min":1.554,"15min":1.554},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73299999999999998,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:16:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.48199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0389999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.51000000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.96699999999999997}]},{"osd":7,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.8540000000000001,"5min":2.8540000000000001,"15min":2.8540000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":1.028}]},{"osd":8,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.873,"15min":0.874},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.996},{"interface":"front","average":{"1min":1.155,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.94899999999999995}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.01},{"interface":"front","average":{"1min":0.873,"5min":0.92600000000000005,"15min":0.93500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.0569999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:17:23 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96099999999999997,"15min":0.95999999999999996},"min":{"1min":0.73899999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.508,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.125},{"interface":"front","average":{"1min":1.077,"5min":0.92400000000000004,"15min":0.89800000000000002},"min":{"1min":0.77500000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.8889999999999998,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":0.98099999999999998}]},{"osd":11,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.95999999999999996,"15min":0.91800000000000004},"min":{"1min":0.68100000000000005,"5min":0.625,"15min":0.625},"max":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.075,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6919999999999999,"5min":1.698,"15min":1.698},"last":1.0800000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":548653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.59899999999999998,"15min":0.61499999999999999},"min":{"1min":0.36799999999999999,"5min":0.316,"15min":0.316},"max":{"1min":3.016,"5min":3.016,"15min":3.016},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.58399999999999996,"15min":0.52700000000000002},"min":{"1min":0.47099999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.98399999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57599999999999996,"15min":0.53700000000000003},"min":{"1min":0.495,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.58299999999999996}]},{"osd":3,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.66100000000000003,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.315,"15min":0.315},"max":{"1min":7.2140000000000004,"5min":7.2140000000000004,"15min":7.2140000000000004},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.77700000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.443,"15min":0.40699999999999997},"min":{"1min":0.36399999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.874},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.51700000000000002,"15min":0.45400000000000001},"min":{"1min":0.38500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.88800000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:16:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65100000000000002,"15min":0.622},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.101,"15min":1.101},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.60699999999999998,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80700000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.98599999999999999}]},{"osd":7,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.89300000000000002}]},{"osd":8,"last update":"Wed Oct 18 09:17:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70699999999999996,"15min":0.70499999999999996},"min":{"1min":0.54500000000000004,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.999}]},{"osd":9,"last update":"Wed Oct 18 09:16:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.754},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":1.024}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.006,"5min":1.206,"15min":1.206},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.79500000000000004,"15min":0.77900000000000003},"min":{"1min":0.62,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1180000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.038}]},{"osd":11,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":1.012},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":1.0629999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":486621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.47499999999999998,"15min":0.40600000000000003},"min":{"1min":0.38200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.498,"15min":0.433},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.73799999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":7.516,"5min":7.516,"15min":7.516},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.65800000000000003,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.68500000000000005,"15min":0.64200000000000002},"min":{"1min":0.61299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.65300000000000002},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64200000000000002,"15min":0.59499999999999997},"min":{"1min":0.45400000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82199999999999995,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.497},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65000000000000002,"15min":0.60199999999999998},"min":{"1min":0.51100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52100000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:17:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.69999999999999996,"15min":0.65200000000000002},"min":{"1min":0.622,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.67800000000000005},"min":{"1min":0.501,"5min":0.442,"15min":0.442},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Oct 18 09:16:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.61499999999999999,"15min":0.60199999999999998},"min":{"1min":0.40400000000000003,"5min":0.373,"15min":0.373},"max":{"1min":0.95399999999999996,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.39800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98299999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.89300000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.53300000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.151,"5min":1.234,"15min":1.234},"last":0.70599999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:16:57 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82299999999999995,"15min":0.82099999999999995},"min":{"1min":0.66300000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.89100000000000001,"15min":0.90800000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.96799999999999997,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.877}]},{"osd":10,"last update":"Wed Oct 18 09:17:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.88300000000000001,"15min":0.89100000000000001},"min":{"1min":0.623,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.016,"5min":1.286,"15min":1.286},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.745,"5min":0.626,"15min":0.626},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.85899999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:17:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":7.6449999999999996,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":7.6769999999999996,"5min":7.6769999999999996,"15min":7.6769999999999996},"last":0.91500000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.39600000000000002,"15min":0.36599999999999999},"min":{"1min":0.316,"5min":0.186,"15min":0.186},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.504},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.41599999999999998,"15min":0.39000000000000001},"min":{"1min":0.34200000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.70699999999999996,"5min":0.753,"15min":0.753},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.873,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.499,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.73399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65000000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.50900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.077,"15min":1.077},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59899999999999998,"15min":0.59799999999999998},"min":{"1min":0.39700000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.92900000000000005,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.81399999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60199999999999998,"15min":0.56000000000000005},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.751,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.626,"15min":0.58499999999999996},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.871,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.65700000000000003,"15min":0.63600000000000001},"min":{"1min":0.52800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.92400000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.90700000000000003,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59599999999999997,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.81299999999999994,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.59599999999999997},"min":{"1min":0.46200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.92500000000000004,"5min":1.131,"15min":1.131},"last":0.92300000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.86199999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.59599999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92200000000000004,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78500000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.77400000000000002,"15min":0.78000000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95399999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.89900000000000002,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.878}]},{"osd":10,"last update":"Wed Oct 18 09:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.96899999999999997,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.745,"15min":0.745},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.93799999999999994}]},{"osd":11,"last update":"Wed Oct 18 09:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.94999999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1083865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53500000000000003,"15min":0.51300000000000001},"min":{"1min":0.40000000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63500000000000001}]},{"osd":1,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.499},{"interface":"front","average":{"1min":0.628,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85299999999999998,"5min":0.88,"15min":0.88},"last":0.52200000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.375},"min":{"1min":0.317,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68200000000000005,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.42899999999999999,"15min":0.39600000000000002},"min":{"1min":0.32000000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56699999999999995,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.54500000000000004},"min":{"1min":0.41599999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.76200000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61399999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.59199999999999997,"15min":0.56599999999999995},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.59499999999999997,"15min":0.56100000000000005},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:16:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.60299999999999998},"min":{"1min":0.50600000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.80900000000000005,"5min":0.88,"15min":0.88},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.51600000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57899999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.82599999999999996,"5min":0.88,"15min":0.88},"last":0.79400000000000004}]},{"osd":9,"last update":"Wed Oct 18 09:16:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71399999999999997,"15min":0.70999999999999996},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.89900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.58599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.75}]},{"osd":10,"last update":"Wed Oct 18 09:17:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.63100000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.56100000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:17:52.485 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-18T09:17:52.485 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-18T09:17:52.485 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-18T09:17:52.486 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph health --format=json 2023-10-18T09:17:53.244 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:17:54.181 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:53 smithi061 ceph-mon[98573]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:54.181 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:53 smithi061 ceph-mon[98573]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:17:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:53 smithi110 ceph-mon[134818]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:53 smithi110 ceph-mon[134818]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:17:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:53 smithi170 ceph-mon[135150]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:17:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:53 smithi170 ceph-mon[135150]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:17:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 -- 172.21.15.61:0/1024830387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701008a0 msgr2=0x7f9170100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 --2- 172.21.15.61:0/1024830387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701008a0 0x7f9170100d20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91600098f0 tx=0x7f9160009c00 comp rx=0 tx=0).stop 2023-10-18T09:17:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 -- 172.21.15.61:0/1024830387 shutdown_connections 2023-10-18T09:17:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 --2- 172.21.15.61:0/1024830387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9170101260 0x7f917010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 --2- 172.21.15.61:0/1024830387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91701057c0 0x7f9170105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 --2- 172.21.15.61:0/1024830387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701008a0 0x7f9170100d20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 -- 172.21.15.61:0/1024830387 >> 172.21.15.61:0/1024830387 conn(0x7f91700779e0 msgr2=0x7f9170077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 -- 172.21.15.61:0/1024830387 shutdown_connections 2023-10-18T09:17:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.871+0000 7f91782c6700 1 -- 172.21.15.61:0/1024830387 wait complete. 2023-10-18T09:17:54.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 Processor -- start 2023-10-18T09:17:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 -- start start 2023-10-18T09:17:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91701008a0 0x7f917019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701057c0 0x7f9170196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170197160 con 0x7f9170101260 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91701972d0 con 0x7f91701057c0 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f91782c6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9170197490 con 0x7f91701008a0 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.873+0000 7f9175861700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9176062700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91701008a0 0x7f917019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56194/0 (socket says 172.21.15.61:56194) 2023-10-18T09:17:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 -- 172.21.15.61:0/495561185 learned_addr learned my addr 172.21.15.61:0/495561185 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9176863700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701057c0 0x7f9170196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 -- 172.21.15.61:0/495561185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91701008a0 msgr2=0x7f917019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91701008a0 0x7f917019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 -- 172.21.15.61:0/495561185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701057c0 msgr2=0x7f9170196b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701057c0 0x7f9170196b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:54.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 -- 172.21.15.61:0/495561185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0095f0 con 0x7f9170101260 2023-10-18T09:17:54.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f9175861700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9160005950 tx=0x7f916000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:54.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.874+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916001c070 con 0x7f9170101260 2023-10-18T09:17:54.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.875+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916000baa0 con 0x7f9170101260 2023-10-18T09:17:54.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.875+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9160009520 con 0x7f9170101260 2023-10-18T09:17:54.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.875+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91600038e0 con 0x7f9170101260 2023-10-18T09:17:54.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.875+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9170072a20 con 0x7f9170101260 2023-10-18T09:17:54.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.876+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9170108da0 con 0x7f9170101260 2023-10-18T09:17:54.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.878+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9160020030 con 0x7f9170101260 2023-10-18T09:17:54.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.879+0000 7f91677fe700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 0x7f915c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:54.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.879+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f91600cdeb0 con 0x7f9170101260 2023-10-18T09:17:54.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.879+0000 7f9176062700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 0x7f915c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:54.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.880+0000 7f9176062700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 0x7f915c0a3a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9170101930 tx=0x7f916c012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:54.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:54.883+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91600970d0 con 0x7f9170101260 2023-10-18T09:17:55.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.142+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9170197f70 con 0x7f9170101260 2023-10-18T09:17:55.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.142+0000 7f91677fe700 1 -- 172.21.15.61:0/495561185 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f916009a6f0 con 0x7f9170101260 2023-10-18T09:17:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:55.146 INFO:teuthology.orchestra.run.smithi061.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-18T09:17:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 msgr2=0x7f915c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:55.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 0x7f915c0a3a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9170101930 tx=0x7f916c012040 comp rx=0 tx=0).stop 2023-10-18T09:17:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 msgr2=0x7f917019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9160005950 tx=0x7f916000fa00 comp rx=0 tx=0).stop 2023-10-18T09:17:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 shutdown_connections 2023-10-18T09:17:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f915c0a1590 0x7f915c0a3a50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91701057c0 0x7f9170196b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91701008a0 0x7f917019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 --2- 172.21.15.61:0/495561185 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9170101260 0x7f917019ceb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 >> 172.21.15.61:0/495561185 conn(0x7f91700779e0 msgr2=0x7f91700fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 shutdown_connections 2023-10-18T09:17:55.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.145+0000 7f91782c6700 1 -- 172.21.15.61:0/495561185 wait complete. 2023-10-18T09:17:55.649 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-18T09:17:55.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:17:55.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 -- 172.21.15.61:0/3162672253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c101930 msgr2=0x7f8c9c101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:55.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/3162672253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c101930 0x7f8c9c101d90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8c840099c0 tx=0x7f8c84009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 -- 172.21.15.61:0/3162672253 shutdown_connections 2023-10-18T09:17:55.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/3162672253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/3162672253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/3162672253 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c101930 0x7f8c9c101d90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 -- 172.21.15.61:0/3162672253 >> 172.21.15.61:0/3162672253 conn(0x7f8c9c0fbc90 msgr2=0x7f8c9c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:55.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 -- 172.21.15.61:0/3162672253 shutdown_connections 2023-10-18T09:17:55.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.855+0000 7f8ca29ff700 1 -- 172.21.15.61:0/3162672253 wait complete. 2023-10-18T09:17:55.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.857+0000 7f8ca29ff700 1 Processor -- start 2023-10-18T09:17:55.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.857+0000 7f8ca29ff700 1 -- start start 2023-10-18T09:17:55.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:55.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 0x7f8c9c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c0738c0 con 0x7f8c9c1022d0 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c073a30 con 0x7f8c9c100740 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca29ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9c1a0760 con 0x7f8c9c101930 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8ca0f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:55.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8c9bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:55.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8c93fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 0x7f8c9c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:55.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8c9bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45228/0 (socket says 172.21.15.61:45228) 2023-10-18T09:17:55.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.858+0000 7f8c9bfff700 1 -- 172.21.15.61:0/2987744207 learned_addr learned my addr 172.21.15.61:0/2987744207 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:55.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 -- 172.21.15.61:0/2987744207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 msgr2=0x7f8c9c072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:55.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 0x7f8c9c072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 -- 172.21.15.61:0/2987744207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 msgr2=0x7f8c9c074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:55.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:55.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 -- 172.21.15.61:0/2987744207 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840095f0 con 0x7f8c9c1022d0 2023-10-18T09:17:55.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8c93fff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 0x7f8c9c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca0f9c700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c073120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007f10 tx=0x7f8c9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94009ea0 con 0x7f8c9c1022d0 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c9400f040 con 0x7f8c9c1022d0 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c1a09f0 con 0x7f8c9c1022d0 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94004350 con 0x7f8c9c1022d0 2023-10-18T09:17:55.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.859+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9c1a0f00 con 0x7f8c9c1022d0 2023-10-18T09:17:55.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.860+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9c107c30 con 0x7f8c9c1022d0 2023-10-18T09:17:55.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.862+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f8c9400dac0 con 0x7f8c9c1022d0 2023-10-18T09:17:55.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.864+0000 7f8c99ffb700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 0x7f8c880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:55.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.864+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c940cd380 con 0x7f8c9c1022d0 2023-10-18T09:17:55.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.864+0000 7f8c9bfff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 0x7f8c880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:55.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.865+0000 7f8c9bfff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 0x7f8c880a3a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005950 tx=0x7f8c8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:55.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:55.867+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8c94096570 con 0x7f8c9c1022d0 2023-10-18T09:17:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.081+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c9c067530 con 0x7f8c9c1022d0 2023-10-18T09:17:56.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.083+0000 7f8c99ffb700 1 -- 172.21.15.61:0/2987744207 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24394 (secure 0 0 0) 0x7f8c94019070 con 0x7f8c9c1022d0 2023-10-18T09:17:56.087 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:56.087 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":90,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:17:21.904505+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:17:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 msgr2=0x7f8c880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 0x7f8c880a3a50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005950 tx=0x7f8c8c009260 comp rx=0 tx=0).stop 2023-10-18T09:17:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 msgr2=0x7f8c9c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c073120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007f10 tx=0x7f8c9400d3b0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 shutdown_connections 2023-10-18T09:17:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8c880a1590 0x7f8c880a3a50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8c9c1022d0 0x7f8c9c073120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8c9c100740 0x7f8c9c074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.085+0000 7f8ca29ff700 1 --2- 172.21.15.61:0/2987744207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c9c101930 0x7f8c9c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.086+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 >> 172.21.15.61:0/2987744207 conn(0x7f8c9c0fbc90 msgr2=0x7f8c9c104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.086+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 shutdown_connections 2023-10-18T09:17:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.086+0000 7f8ca29ff700 1 -- 172.21.15.61:0/2987744207 wait complete. 2023-10-18T09:17:56.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:17:56.109 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:17:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:55 smithi110 ceph-mon[134818]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:55 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/495561185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T09:17:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:55 smithi061 ceph-mon[98573]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:55 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/495561185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T09:17:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3450795683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 msgr2=0x7fc484108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3450795683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484108be0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00b0d0 tx=0x7fc47c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3450795683 shutdown_connections 2023-10-18T09:17:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3450795683 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484108be0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3450795683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4841007a0 0x7fc484100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3450795683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc484101990 0x7fc484101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3450795683 >> 172.21.15.61:0/3450795683 conn(0x7fc4840fbcf0 msgr2=0x7fc4840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3450795683 shutdown_connections 2023-10-18T09:17:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.314+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3450795683 wait complete. 2023-10-18T09:17:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.315+0000 7fc48b9ff700 1 Processor -- start 2023-10-18T09:17:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 -- start start 2023-10-18T09:17:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4841007a0 0x7fc484074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc484101990 0x7fc484072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4840738c0 con 0x7fc484102330 2023-10-18T09:17:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484073a30 con 0x7fc4841007a0 2023-10-18T09:17:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48b9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4841a07c0 con 0x7fc484101990 2023-10-18T09:17:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc489f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc488f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc484101990 0x7fc484072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc489f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56236/0 (socket says 172.21.15.61:56236) 2023-10-18T09:17:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc489f9c700 1 -- 172.21.15.61:0/3373237686 learned_addr learned my addr 172.21.15.61:0/3373237686 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc48979b700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4841007a0 0x7fc484074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc489f9c700 1 -- 172.21.15.61:0/3373237686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc484101990 msgr2=0x7fc484072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.316+0000 7fc489f9c700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc484101990 0x7fc484072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc489f9c700 1 -- 172.21.15.61:0/3373237686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4841007a0 msgr2=0x7fc484074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc489f9c700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4841007a0 0x7fc484074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc489f9c700 1 -- 172.21.15.61:0/3373237686 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c009d00 con 0x7fc484102330 2023-10-18T09:17:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc489f9c700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc47c007b60 tx=0x7fc47c008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00e040 con 0x7fc484102330 2023-10-18T09:17:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc47c01d930 con 0x7fc484102330 2023-10-18T09:17:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4841a09f0 con 0x7fc484102330 2023-10-18T09:17:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c01ba00 con 0x7fc484102330 2023-10-18T09:17:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.317+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4841a0e10 con 0x7fc484102330 2023-10-18T09:17:56.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.318+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484107b50 con 0x7fc484102330 2023-10-18T09:17:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.322+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fc47c01daa0 con 0x7fc484102330 2023-10-18T09:17:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.323+0000 7fc47a7fc700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 0x7fc4700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.323+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc47c0cebb0 con 0x7fc484102330 2023-10-18T09:17:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.323+0000 7fc48979b700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 0x7fc4700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.324+0000 7fc48979b700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 0x7fc4700a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc4841017f0 tx=0x7fc480009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.325+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc47c097dd0 con 0x7fc484102330 2023-10-18T09:17:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:55 smithi170 ceph-mon[135150]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:17:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:55 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/495561185' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-18T09:17:56.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.535+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc48410c0d0 con 0x7fc484102330 2023-10-18T09:17:56.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.535+0000 7fc47a7fc700 1 -- 172.21.15.61:0/3373237686 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc47c09b3f0 con 0x7fc484102330 2023-10-18T09:17:56.539 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:17:56.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 msgr2=0x7fc4700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 0x7fc4700a39d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc4841017f0 tx=0x7fc480009260 comp rx=0 tx=0).stop 2023-10-18T09:17:56.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 msgr2=0x7fc484073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc47c007b60 tx=0x7fc47c008ec0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 shutdown_connections 2023-10-18T09:17:56.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4700a1510 0x7fc4700a39d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc484102330 0x7fc484073120 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4841007a0 0x7fc484074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 --2- 172.21.15.61:0/3373237686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc484101990 0x7fc484072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 >> 172.21.15.61:0/3373237686 conn(0x7fc4840fbcf0 msgr2=0x7fc484104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 shutdown_connections 2023-10-18T09:17:56.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.538+0000 7fc48b9ff700 1 -- 172.21.15.61:0/3373237686 wait complete. 2023-10-18T09:17:56.560 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:17:56.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 -- 172.21.15.61:0/569621745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0102330 msgr2=0x7f69b0108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 --2- 172.21.15.61:0/569621745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0102330 0x7f69b0108be0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b0d0 tx=0x7f69a800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 -- 172.21.15.61:0/569621745 shutdown_connections 2023-10-18T09:17:56.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 --2- 172.21.15.61:0/569621745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b0102330 0x7f69b0108be0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 --2- 172.21.15.61:0/569621745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b01007a0 0x7f69b0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 --2- 172.21.15.61:0/569621745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 0x7f69b0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.764+0000 7f69b65ff700 1 -- 172.21.15.61:0/569621745 >> 172.21.15.61:0/569621745 conn(0x7f69b00fbcf0 msgr2=0x7f69b00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:56.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.765+0000 7f69b65ff700 1 -- 172.21.15.61:0/569621745 shutdown_connections 2023-10-18T09:17:56.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.765+0000 7f69b65ff700 1 -- 172.21.15.61:0/569621745 wait complete. 2023-10-18T09:17:56.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.765+0000 7f69b65ff700 1 Processor -- start 2023-10-18T09:17:56.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 -- start start 2023-10-18T09:17:56.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 0x7f69b0196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b0102330 0x7f69b019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019e370 con 0x7f69b0102330 2023-10-18T09:17:56.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019e4b0 con 0x7f69b01007a0 2023-10-18T09:17:56.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b65ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b019e640 con 0x7f69b0101990 2023-10-18T09:17:56.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69b4b9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b0102330 0x7f69b019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45272/0 (socket says 172.21.15.61:45272) 2023-10-18T09:17:56.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69affff700 1 -- 172.21.15.61:0/3843379258 learned_addr learned my addr 172.21.15.61:0/3843379258 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:56.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.766+0000 7f69af7fe700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 0x7f69b0196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 -- 172.21.15.61:0/3843379258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 msgr2=0x7f69b0196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 0x7f69b0196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 -- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b0102330 msgr2=0x7f69b019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b0102330 0x7f69b019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 -- 172.21.15.61:0/3843379258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8009d00 con 0x7f69b01007a0 2023-10-18T09:17:56.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69affff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69a000d980 tx=0x7f69a000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:56.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0004020 con 0x7f69b01007a0 2023-10-18T09:17:56.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69a000f690 con 0x7f69b01007a0 2023-10-18T09:17:56.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b019e920 con 0x7f69b01007a0 2023-10-18T09:17:56.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0003a60 con 0x7f69b01007a0 2023-10-18T09:17:56.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.767+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b019ee30 con 0x7f69b01007a0 2023-10-18T09:17:56.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.768+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b0107b50 con 0x7f69b01007a0 2023-10-18T09:17:56.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.770+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f69a000fb00 con 0x7f69b01007a0 2023-10-18T09:17:56.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.772+0000 7f69ad7fa700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 0x7f69980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:56.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.772+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f69a00cd130 con 0x7f69b01007a0 2023-10-18T09:17:56.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.772+0000 7f69af7fe700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 0x7f69980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:56.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.773+0000 7f69af7fe700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 0x7f69980a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009c60 tx=0x7f69a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:56.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.774+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69a00962c0 con 0x7f69b01007a0 2023-10-18T09:17:56.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.990+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69b010c010 con 0x7f69b01007a0 2023-10-18T09:17:56.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.991+0000 7f69ad7fa700 1 -- 172.21.15.61:0/3843379258 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69a00998e0 con 0x7f69b01007a0 2023-10-18T09:17:56.995 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:17:56.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 msgr2=0x7f69980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 0x7f69980a39d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009c60 tx=0x7f69a4009260 comp rx=0 tx=0).stop 2023-10-18T09:17:56.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 msgr2=0x7f69b0196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:56.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69a000d980 tx=0x7f69a000dc90 comp rx=0 tx=0).stop 2023-10-18T09:17:56.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 shutdown_connections 2023-10-18T09:17:56.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69980a1510 0x7f69980a39d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69b0102330 0x7f69b019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69b01007a0 0x7f69b0196840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 --2- 172.21.15.61:0/3843379258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69b0101990 0x7f69b0196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:56.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.993+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 >> 172.21.15.61:0/3843379258 conn(0x7f69b00fbcf0 msgr2=0x7f69b0104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:56.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.994+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 shutdown_connections 2023-10-18T09:17:56.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:56.994+0000 7f69b65ff700 1 -- 172.21.15.61:0/3843379258 wait complete. 2023-10-18T09:17:57.016 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:17:57.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2987744207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:57.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3373237686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:17:57.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 -- 172.21.15.61:0/166895463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 msgr2=0x7f118c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 --2- 172.21.15.61:0/166895463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c101df0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f11800099c0 tx=0x7f1180009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 -- 172.21.15.61:0/166895463 shutdown_connections 2023-10-18T09:17:57.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 --2- 172.21.15.61:0/166895463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 --2- 172.21.15.61:0/166895463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 --2- 172.21.15.61:0/166895463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c101df0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 -- 172.21.15.61:0/166895463 >> 172.21.15.61:0/166895463 conn(0x7f118c0fbcf0 msgr2=0x7f118c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:57.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 -- 172.21.15.61:0/166895463 shutdown_connections 2023-10-18T09:17:57.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.220+0000 7f11925ff700 1 -- 172.21.15.61:0/166895463 wait complete. 2023-10-18T09:17:57.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 Processor -- start 2023-10-18T09:17:57.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 -- start start 2023-10-18T09:17:57.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19e270 con 0x7f118c101990 2023-10-18T09:17:57.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19e3b0 con 0x7f118c1007a0 2023-10-18T09:17:57.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.222+0000 7f11925ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19e540 con 0x7f118c102330 2023-10-18T09:17:57.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45282/0 (socket says 172.21.15.61:45282) 2023-10-18T09:17:57.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 -- 172.21.15.61:0/3300577194 learned_addr learned my addr 172.21.15.61:0/3300577194 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:57.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f1190b9c700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 -- 172.21.15.61:0/3300577194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 msgr2=0x7f118c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 -- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 msgr2=0x7f118c196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 -- 172.21.15.61:0/3300577194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800095f0 con 0x7f118c1007a0 2023-10-18T09:17:57.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f1190b9c700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.223+0000 7f118bfff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e970 tx=0x7f117c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.224+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c009790 con 0x7f118c1007a0 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.224+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f117c00eec0 con 0x7f118c1007a0 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.224+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c19e820 con 0x7f118c1007a0 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.224+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c003cf0 con 0x7f118c1007a0 2023-10-18T09:17:57.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.224+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c19ed80 con 0x7f118c1007a0 2023-10-18T09:17:57.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.225+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c107c20 con 0x7f118c1007a0 2023-10-18T09:17:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.227+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f117c00ccd0 con 0x7f118c1007a0 2023-10-18T09:17:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.228+0000 7f11897fa700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 0x7f11740a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.228+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f117c014070 con 0x7f118c1007a0 2023-10-18T09:17:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.229+0000 7f118b7fe700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 0x7f11740a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.229+0000 7f118b7fe700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 0x7f11740a3a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1180006010 tx=0x7f118000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:57.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.232+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f117c096580 con 0x7f118c1007a0 2023-10-18T09:17:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2987744207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:57.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3373237686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:17:57.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2987744207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:57.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3373237686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:17:57.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.444+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f118c10c110 con 0x7f118c1007a0 2023-10-18T09:17:57.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.445+0000 7f11897fa700 1 -- 172.21.15.61:0/3300577194 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f117c099ba0 con 0x7f118c1007a0 2023-10-18T09:17:57.448 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 msgr2=0x7f11740a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 0x7f11740a3a50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1180006010 tx=0x7f118000b540 comp rx=0 tx=0).stop 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 msgr2=0x7f118c196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f117c00e970 tx=0x7f117c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 shutdown_connections 2023-10-18T09:17:57.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11740a1590 0x7f11740a3a50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f118c102330 0x7f118c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f118c1007a0 0x7f118c196740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 --2- 172.21.15.61:0/3300577194 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f118c101990 0x7f118c196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 >> 172.21.15.61:0/3300577194 conn(0x7f118c0fbcf0 msgr2=0x7f118c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:57.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 shutdown_connections 2023-10-18T09:17:57.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.447+0000 7f11925ff700 1 -- 172.21.15.61:0/3300577194 wait complete. 2023-10-18T09:17:57.471 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:17:57.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.676+0000 7f9a20fff700 1 -- 172.21.15.61:0/3049498751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c101990 msgr2=0x7f9a1c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.676+0000 7f9a20fff700 1 --2- 172.21.15.61:0/3049498751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c101990 0x7f9a1c101df0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9a100099c0 tx=0x7f9a10009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 -- 172.21.15.61:0/3049498751 shutdown_connections 2023-10-18T09:17:57.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 --2- 172.21.15.61:0/3049498751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c102330 0x7f9a1c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 --2- 172.21.15.61:0/3049498751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 0x7f9a1c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 --2- 172.21.15.61:0/3049498751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c101990 0x7f9a1c101df0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 -- 172.21.15.61:0/3049498751 >> 172.21.15.61:0/3049498751 conn(0x7f9a1c0fbcf0 msgr2=0x7f9a1c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:57.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 -- 172.21.15.61:0/3049498751 shutdown_connections 2023-10-18T09:17:57.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.677+0000 7f9a20fff700 1 -- 172.21.15.61:0/3049498751 wait complete. 2023-10-18T09:17:57.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 Processor -- start 2023-10-18T09:17:57.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 -- start start 2023-10-18T09:17:57.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 0x7f9a1c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 0x7f9a1c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1a0420 con 0x7f9a1c102330 2023-10-18T09:17:57.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1a0560 con 0x7f9a1c101990 2023-10-18T09:17:57.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a20fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1a06f0 con 0x7f9a1c1007a0 2023-10-18T09:17:57.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a1ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 0x7f9a1c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.678+0000 7f9a1ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56276/0 (socket says 172.21.15.61:56276) 2023-10-18T09:17:57.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 -- 172.21.15.61:0/462659334 learned_addr learned my addr 172.21.15.61:0/462659334 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:57.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1a59c700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 0x7f9a1c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 -- 172.21.15.61:0/462659334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 msgr2=0x7f9a1c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 0x7f9a1c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 -- 172.21.15.61:0/462659334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 msgr2=0x7f9a1c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 0x7f9a1c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 -- 172.21.15.61:0/462659334 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100095f0 con 0x7f9a1c102330 2023-10-18T09:17:57.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a19d9b700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 0x7f9a1c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:57.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a1ad9d700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ef30 tx=0x7f9a1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:57.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14018070 con 0x7f9a1c102330 2023-10-18T09:17:57.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1400f040 con 0x7f9a1c102330 2023-10-18T09:17:57.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c1a0970 con 0x7f9a1c102330 2023-10-18T09:17:57.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.679+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400cd00 con 0x7f9a1c102330 2023-10-18T09:17:57.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.680+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c1a0e30 con 0x7f9a1c102330 2023-10-18T09:17:57.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.681+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c0675a0 con 0x7f9a1c102330 2023-10-18T09:17:57.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.683+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f9a14003a00 con 0x7f9a1c102330 2023-10-18T09:17:57.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.684+0000 7f9a0b7fe700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 0x7f9a040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:57.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.684+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a140ccd70 con 0x7f9a1c102330 2023-10-18T09:17:57.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.684+0000 7f9a1a59c700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 0x7f9a040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:57.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.685+0000 7f9a1a59c700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 0x7f9a040a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c1017f0 tx=0x7f9a0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:57.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.688+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9a14095f90 con 0x7f9a1c102330 2023-10-18T09:17:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.901+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a1c1a1120 con 0x7f9a1c102330 2023-10-18T09:17:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.901+0000 7f9a0b7fe700 1 -- 172.21.15.61:0/462659334 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a140995b0 con 0x7f9a1c102330 2023-10-18T09:17:57.905 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:17:57.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.903+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 msgr2=0x7f9a040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.903+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 0x7f9a040a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c1017f0 tx=0x7f9a0c009260 comp rx=0 tx=0).stop 2023-10-18T09:17:57.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.903+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 msgr2=0x7f9a1c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:57.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.903+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400ef30 tx=0x7f9a1400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 shutdown_connections 2023-10-18T09:17:57.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9a040a1510 0x7f9a040a39d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9a1c102330 0x7f9a1c19fe50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a1c1007a0 0x7f9a1c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 --2- 172.21.15.61:0/462659334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a1c101990 0x7f9a1c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:57.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 >> 172.21.15.61:0/462659334 conn(0x7f9a1c0fbcf0 msgr2=0x7f9a1c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:57.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 shutdown_connections 2023-10-18T09:17:57.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:57.904+0000 7f9a20fff700 1 -- 172.21.15.61:0/462659334 wait complete. 2023-10-18T09:17:57.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:17:58.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 -- 172.21.15.61:0/1189197606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 msgr2=0x7fb520100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 --2- 172.21.15.61:0/1189197606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520100c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0098f0 tx=0x7fb50c009c00 comp rx=0 tx=0).stop 2023-10-18T09:17:58.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 -- 172.21.15.61:0/1189197606 shutdown_connections 2023-10-18T09:17:58.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 --2- 172.21.15.61:0/1189197606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5201011a0 0x7fb520107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 --2- 172.21.15.61:0/1189197606 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5200ff610 0x7fb5200ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 --2- 172.21.15.61:0/1189197606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520100c60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 -- 172.21.15.61:0/1189197606 >> 172.21.15.61:0/1189197606 conn(0x7fb5200faba0 msgr2=0x7fb5200fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:58.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 -- 172.21.15.61:0/1189197606 shutdown_connections 2023-10-18T09:17:58.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.129+0000 7fb526fff700 1 -- 172.21.15.61:0/1189197606 wait complete. 2023-10-18T09:17:58.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.130+0000 7fb526fff700 1 Processor -- start 2023-10-18T09:17:58.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.130+0000 7fb526fff700 1 -- start start 2023-10-18T09:17:58.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5200ff610 0x7fb520198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5201011a0 0x7fb52019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201a0440 con 0x7fb5201011a0 2023-10-18T09:17:58.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201a0580 con 0x7fb5200ff610 2023-10-18T09:17:58.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb526fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201a0710 con 0x7fb520100800 2023-10-18T09:17:58.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:40784/0 (socket says 172.21.15.61:40784) 2023-10-18T09:17:58.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb52559c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5201011a0 0x7fb52019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 -- 172.21.15.61:0/3032940541 learned_addr learned my addr 172.21.15.61:0/3032940541 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:58.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb524d9b700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5200ff610 0x7fb520198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 -- 172.21.15.61:0/3032940541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5200ff610 msgr2=0x7fb520198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5200ff610 0x7fb520198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 -- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5201011a0 msgr2=0x7fb52019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5201011a0 0x7fb52019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.131+0000 7fb51ffff700 1 -- 172.21.15.61:0/3032940541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009520 con 0x7fb520100800 2023-10-18T09:17:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb51ffff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009810 tx=0x7fb50c00b560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c01c070 con 0x7fb520100800 2023-10-18T09:17:58.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201a0990 con 0x7fb520100800 2023-10-18T09:17:58.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb50c005040 con 0x7fb520100800 2023-10-18T09:17:58.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0038c0 con 0x7fb520100800 2023-10-18T09:17:58.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.132+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201a0e10 con 0x7fb520100800 2023-10-18T09:17:58.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.133+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520067530 con 0x7fb520100800 2023-10-18T09:17:58.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.135+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fb50c020030 con 0x7fb520100800 2023-10-18T09:17:58.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.137+0000 7fb51dffb700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 0x7fb5100a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.137+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb50c0ce300 con 0x7fb520100800 2023-10-18T09:17:58.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.137+0000 7fb524d9b700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 0x7fb5100a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.138+0000 7fb524d9b700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 0x7fb5100a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb520100660 tx=0x7fb514009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:58.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.139+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb50c097520 con 0x7fb520100800 2023-10-18T09:17:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:57 smithi110 ceph-mon[134818]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3843379258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3300577194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:57 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/462659334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:57 smithi061 ceph-mon[98573]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3843379258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3300577194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:57 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/462659334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:57 smithi170 ceph-mon[135150]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:17:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3843379258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3300577194' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:57 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/462659334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:17:58.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.403+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb52010afd0 con 0x7fb520100800 2023-10-18T09:17:58.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.404+0000 7fb51dffb700 1 -- 172.21.15.61:0/3032940541 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fb50c09ab40 con 0x7fb520100800 2023-10-18T09:17:58.407 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:58.408 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:17:58.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 msgr2=0x7fb5100a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 0x7fb5100a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb520100660 tx=0x7fb514009310 comp rx=0 tx=0).stop 2023-10-18T09:17:58.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 msgr2=0x7fb520198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009810 tx=0x7fb50c00b560 comp rx=0 tx=0).stop 2023-10-18T09:17:58.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 shutdown_connections 2023-10-18T09:17:58.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb5100a1510 0x7fb5100a39d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb5201011a0 0x7fb52019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5200ff610 0x7fb520198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 --2- 172.21.15.61:0/3032940541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb520100800 0x7fb520198e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 >> 172.21.15.61:0/3032940541 conn(0x7fb5200faba0 msgr2=0x7fb520067880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:58.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.406+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 shutdown_connections 2023-10-18T09:17:58.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.407+0000 7fb526fff700 1 -- 172.21.15.61:0/3032940541 wait complete. 2023-10-18T09:17:58.412 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:17:58.429 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:17:58.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 -- 172.21.15.61:0/852941371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 msgr2=0x7f3c3c108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 --2- 172.21.15.61:0/852941371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c108b80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400b0d0 tx=0x7f3c3400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 -- 172.21.15.61:0/852941371 shutdown_connections 2023-10-18T09:17:58.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 --2- 172.21.15.61:0/852941371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c108b80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 --2- 172.21.15.61:0/852941371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 --2- 172.21.15.61:0/852941371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 -- 172.21.15.61:0/852941371 >> 172.21.15.61:0/852941371 conn(0x7f3c3c0fbc90 msgr2=0x7f3c3c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:58.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 -- 172.21.15.61:0/852941371 shutdown_connections 2023-10-18T09:17:58.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.631+0000 7f3c421ff700 1 -- 172.21.15.61:0/852941371 wait complete. 2023-10-18T09:17:58.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.632+0000 7f3c421ff700 1 Processor -- start 2023-10-18T09:17:58.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.632+0000 7f3c421ff700 1 -- start start 2023-10-18T09:17:58.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c19e2f0 con 0x7f3c3c1022d0 2023-10-18T09:17:58.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c19e430 con 0x7f3c3c100740 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c421ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c19e5c0 con 0x7f3c3c101930 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c33fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45322/0 (socket says 172.21.15.61:45322) 2023-10-18T09:17:58.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 -- 172.21.15.61:0/2391971529 learned_addr learned my addr 172.21.15.61:0/2391971529 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:58.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c33fff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:40806/0 (socket says 172.21.15.61:40806) 2023-10-18T09:17:58.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 -- 172.21.15.61:0/2391971529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 msgr2=0x7f3c3c196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 -- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 msgr2=0x7f3c3c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3b7fe700 1 -- 172.21.15.61:0/2391971529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c34009d00 con 0x7f3c3c100740 2023-10-18T09:17:58.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.633+0000 7f3c3bfff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:17:58.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c3b7fe700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e970 tx=0x7f3c2c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:58.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c009790 con 0x7f3c3c100740 2023-10-18T09:17:58.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c2c00c7e0 con 0x7f3c3c100740 2023-10-18T09:17:58.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c003c70 con 0x7f3c3c100740 2023-10-18T09:17:58.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c19e8a0 con 0x7f3c3c100740 2023-10-18T09:17:58.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.634+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c19ed60 con 0x7f3c3c100740 2023-10-18T09:17:58.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.635+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c107af0 con 0x7f3c3c100740 2023-10-18T09:17:58.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.637+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3c2c00cc50 con 0x7f3c3c100740 2023-10-18T09:17:58.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.638+0000 7f3c397fa700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 0x7f3c280a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:58.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.639+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c2c014070 con 0x7f3c3c100740 2023-10-18T09:17:58.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.639+0000 7f3c33fff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 0x7f3c280a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:58.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.639+0000 7f3c33fff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 0x7f3c280a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009c60 tx=0x7f3c24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:58.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.641+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3c2c096560 con 0x7f3c3c100740 2023-10-18T09:17:58.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.858+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c3c067530 con 0x7f3c3c100740 2023-10-18T09:17:58.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.860+0000 7f3c397fa700 1 -- 172.21.15.61:0/2391971529 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24394 (secure 0 0 0) 0x7f3c2c017e90 con 0x7f3c3c100740 2023-10-18T09:17:58.863 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:17:58.864 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":90,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:17:21.904505+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:17:58.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 msgr2=0x7f3c280a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 0x7f3c280a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009c60 tx=0x7f3c24009260 comp rx=0 tx=0).stop 2023-10-18T09:17:58.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 msgr2=0x7f3c3c1967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:58.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e970 tx=0x7f3c2c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:58.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 shutdown_connections 2023-10-18T09:17:58.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3c280a1510 0x7f3c280a39d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3c3c1022d0 0x7f3c3c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c3c100740 0x7f3c3c1967c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 --2- 172.21.15.61:0/2391971529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c3c101930 0x7f3c3c196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:58.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.862+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 >> 172.21.15.61:0/2391971529 conn(0x7f3c3c0fbc90 msgr2=0x7f3c3c104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:58.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.863+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 shutdown_connections 2023-10-18T09:17:58.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:58.863+0000 7f3c421ff700 1 -- 172.21.15.61:0/2391971529 wait complete. 2023-10-18T09:17:58.885 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:17:58.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:17:59.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/1654918397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 msgr2=0x7f3e98101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/1654918397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98101df0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0099c0 tx=0x7f3e8c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/1654918397 shutdown_connections 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/1654918397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e98102330 0x7f3e98108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/1654918397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e981007a0 0x7f3e98100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/1654918397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98101df0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/1654918397 >> 172.21.15.61:0/1654918397 conn(0x7f3e980fbcf0 msgr2=0x7f3e980fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:59.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/1654918397 shutdown_connections 2023-10-18T09:17:59.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.094+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/1654918397 wait complete. 2023-10-18T09:17:59.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.095+0000 7f3e9e9ff700 1 Processor -- start 2023-10-18T09:17:59.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 -- start start 2023-10-18T09:17:59.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e981007a0 0x7f3e981988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e98102330 0x7f3e9819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e981a0410 con 0x7f3e98101990 2023-10-18T09:17:59.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e981a0550 con 0x7f3e98102330 2023-10-18T09:17:59.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9e9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e981a06e0 con 0x7f3e981007a0 2023-10-18T09:17:59.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e977fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e977fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42338/0 (socket says 172.21.15.61:42338) 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e977fe700 1 -- 172.21.15.61:0/2277325388 learned_addr learned my addr 172.21.15.61:0/2277325388 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e97fff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e981007a0 0x7f3e981988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.096+0000 7f3e9cf9c700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e98102330 0x7f3e9819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 -- 172.21.15.61:0/2277325388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e981007a0 msgr2=0x7f3e981988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e981007a0 0x7f3e981988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 -- 172.21.15.61:0/2277325388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e98102330 msgr2=0x7f3e9819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e98102330 0x7f3e9819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 -- 172.21.15.61:0/2277325388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0095f0 con 0x7f3e98101990 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e977fe700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c006010 tx=0x7f3e8c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c01d070 con 0x7f3e98101990 2023-10-18T09:17:59.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e8c005030 con 0x7f3e98101990 2023-10-18T09:17:59.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c004b10 con 0x7f3e98101990 2023-10-18T09:17:59.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e981a0960 con 0x7f3e98101990 2023-10-18T09:17:59.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.097+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e981a0d30 con 0x7f3e98101990 2023-10-18T09:17:59.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.098+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980675a0 con 0x7f3e98101990 2023-10-18T09:17:59.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.100+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3e8c0051a0 con 0x7f3e98101990 2023-10-18T09:17:59.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.102+0000 7f3e957fa700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 0x7f3e800a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.102+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3e8c0cdd60 con 0x7f3e98101990 2023-10-18T09:17:59.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.102+0000 7f3e97fff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 0x7f3e800a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.103+0000 7f3e97fff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 0x7f3e800a3a50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e981017f0 tx=0x7f3e88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:59.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.105+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e8c096fe0 con 0x7f3e98101990 2023-10-18T09:17:59.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:58 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3032940541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:58 smithi110 ceph-mon[134818]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:59.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:58 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2391971529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:58 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3032940541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:58 smithi061 ceph-mon[98573]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:59.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:58 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2391971529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.319+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e9810c250 con 0x7f3e98101990 2023-10-18T09:17:59.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.319+0000 7f3e957fa700 1 -- 172.21.15.61:0/2277325388 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e8c09a600 con 0x7f3e98101990 2023-10-18T09:17:59.323 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:17:59.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 msgr2=0x7f3e800a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 0x7f3e800a3a50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e981017f0 tx=0x7f3e88009260 comp rx=0 tx=0).stop 2023-10-18T09:17:59.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 msgr2=0x7f3e98198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c006010 tx=0x7f3e8c00f690 comp rx=0 tx=0).stop 2023-10-18T09:17:59.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 shutdown_connections 2023-10-18T09:17:59.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e800a1590 0x7f3e800a3a50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e98102330 0x7f3e9819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e981007a0 0x7f3e981988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 --2- 172.21.15.61:0/2277325388 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3e98101990 0x7f3e98198e20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 >> 172.21.15.61:0/2277325388 conn(0x7f3e980fbcf0 msgr2=0x7f3e98072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:59.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 shutdown_connections 2023-10-18T09:17:59.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.322+0000 7f3e9e9ff700 1 -- 172.21.15.61:0/2277325388 wait complete. 2023-10-18T09:17:59.345 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:17:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:58 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3032940541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:58 smithi170 ceph-mon[135150]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:17:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:58 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2391971529' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:17:59.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 -- 172.21.15.61:0/412604966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac101990 msgr2=0x7f3eac101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/412604966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac101990 0x7f3eac101df0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3e940099c0 tx=0x7f3e94009cd0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 -- 172.21.15.61:0/412604966 shutdown_connections 2023-10-18T09:17:59.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/412604966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac102330 0x7f3eac108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/412604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac1007a0 0x7f3eac100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/412604966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac101990 0x7f3eac101df0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 -- 172.21.15.61:0/412604966 >> 172.21.15.61:0/412604966 conn(0x7f3eac0fbcf0 msgr2=0x7f3eac0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:59.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 -- 172.21.15.61:0/412604966 shutdown_connections 2023-10-18T09:17:59.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.548+0000 7f3eb1fff700 1 -- 172.21.15.61:0/412604966 wait complete. 2023-10-18T09:17:59.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.549+0000 7f3eb1fff700 1 Processor -- start 2023-10-18T09:17:59.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 -- start start 2023-10-18T09:17:59.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac101990 0x7f3eac072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac102330 0x7f3eac073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0738c0 con 0x7f3eac1007a0 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac073a30 con 0x7f3eac102330 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eb1fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1a0730 con 0x7f3eac101990 2023-10-18T09:17:59.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eab7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42356/0 (socket says 172.21.15.61:42356) 2023-10-18T09:17:59.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3eab7fe700 1 -- 172.21.15.61:0/1637435005 learned_addr learned my addr 172.21.15.61:0/1637435005 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:17:59.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.550+0000 7f3ea3fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac101990 0x7f3eac072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eabfff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac102330 0x7f3eac073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 -- 172.21.15.61:0/1637435005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac101990 msgr2=0x7f3eac072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac101990 0x7f3eac072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 -- 172.21.15.61:0/1637435005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac102330 msgr2=0x7f3eac073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac102330 0x7f3eac073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 -- 172.21.15.61:0/1637435005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940095f0 con 0x7f3eac1007a0 2023-10-18T09:17:59.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3eab7fe700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e970 tx=0x7f3e9c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:59.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c009790 con 0x7f3eac1007a0 2023-10-18T09:17:59.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.551+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e9c00c780 con 0x7f3eac1007a0 2023-10-18T09:17:59.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.552+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1a09c0 con 0x7f3eac1007a0 2023-10-18T09:17:59.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.552+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c003be0 con 0x7f3eac1007a0 2023-10-18T09:17:59.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.552+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1a0f20 con 0x7f3eac1007a0 2023-10-18T09:17:59.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.553+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac107bc0 con 0x7f3eac1007a0 2023-10-18T09:17:59.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.555+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f3e9c00cbf0 con 0x7f3eac1007a0 2023-10-18T09:17:59.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.556+0000 7f3ea97fa700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 0x7f3e980a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:17:59.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.556+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3e9c014070 con 0x7f3eac1007a0 2023-10-18T09:17:59.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.556+0000 7f3ea3fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 0x7f3e980a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:17:59.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.557+0000 7f3ea3fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 0x7f3e980a3a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e9400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:17:59.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.560+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e9c096440 con 0x7f3eac1007a0 2023-10-18T09:17:59.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.771+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3eac10bfc0 con 0x7f3eac1007a0 2023-10-18T09:17:59.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.772+0000 7f3ea97fa700 1 -- 172.21.15.61:0/1637435005 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e9c099a60 con 0x7f3eac1007a0 2023-10-18T09:17:59.776 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:17:59.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 msgr2=0x7f3e980a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 0x7f3e980a3a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e9400b540 comp rx=0 tx=0).stop 2023-10-18T09:17:59.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 msgr2=0x7f3eac074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:17:59.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e970 tx=0x7f3e9c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:17:59.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 shutdown_connections 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f3e980a1590 0x7f3e980a3a50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3eac102330 0x7f3eac073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3eac1007a0 0x7f3eac074550 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 --2- 172.21.15.61:0/1637435005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3eac101990 0x7f3eac072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 >> 172.21.15.61:0/1637435005 conn(0x7f3eac0fbcf0 msgr2=0x7f3eac104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:17:59.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 shutdown_connections 2023-10-18T09:17:59.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:17:59.774+0000 7f3eb1fff700 1 -- 172.21.15.61:0/1637435005 wait complete. 2023-10-18T09:17:59.797 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:00.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 -- 172.21.15.61:0/1821142904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a81007a0 msgr2=0x7fe2a8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/1821142904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a81007a0 0x7fe2a8100bc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40099c0 tx=0x7fe2a4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 -- 172.21.15.61:0/1821142904 shutdown_connections 2023-10-18T09:18:00.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/1821142904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 0x7fe2a8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/1821142904 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a81007a0 0x7fe2a8100bc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/1821142904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a8101990 0x7fe2a8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 -- 172.21.15.61:0/1821142904 >> 172.21.15.61:0/1821142904 conn(0x7fe2a80fbcf0 msgr2=0x7fe2a80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:00.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 -- 172.21.15.61:0/1821142904 shutdown_connections 2023-10-18T09:18:00.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.002+0000 7fe2b03ff700 1 -- 172.21.15.61:0/1821142904 wait complete. 2023-10-18T09:18:00.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.003+0000 7fe2b03ff700 1 Processor -- start 2023-10-18T09:18:00.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.003+0000 7fe2b03ff700 1 -- start start 2023-10-18T09:18:00.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 0x7fe2a8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 0x7fe2a819fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81a0490 con 0x7fe2a8101990 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81a05d0 con 0x7fe2a8102330 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2b03ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81a0760 con 0x7fe2a81007a0 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42388/0 (socket says 172.21.15.61:42388) 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 -- 172.21.15.61:0/395277110 learned_addr learned my addr 172.21.15.61:0/395277110 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:00.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ae19b700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 0x7fe2a8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ae99c700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 0x7fe2a819fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 -- 172.21.15.61:0/395277110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 msgr2=0x7fe2a8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 0x7fe2a8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.004+0000 7fe2ad99a700 1 -- 172.21.15.61:0/395277110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 msgr2=0x7fe2a819fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2ad99a700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 0x7fe2a819fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2ad99a700 1 -- 172.21.15.61:0/395277110 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a40095f0 con 0x7fe2a8101990 2023-10-18T09:18:00.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2ae19b700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 0x7fe2a8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:00.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2ad99a700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9a0 tx=0x7fe29800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298009790 con 0x7fe2a8101990 2023-10-18T09:18:00.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe298010430 con 0x7fe2a8101990 2023-10-18T09:18:00.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a81a0a40 con 0x7fe2a8101990 2023-10-18T09:18:00.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298018680 con 0x7fe2a8101990 2023-10-18T09:18:00.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.005+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a81a0fa0 con 0x7fe2a8101990 2023-10-18T09:18:00.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.006+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a80675a0 con 0x7fe2a8101990 2023-10-18T09:18:00.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.008+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fe2980105a0 con 0x7fe2a8101990 2023-10-18T09:18:00.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.010+0000 7fe29f7fe700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 0x7fe2940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.010+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe298014070 con 0x7fe2a8101990 2023-10-18T09:18:00.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.010+0000 7fe2ae19b700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 0x7fe2940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.010+0000 7fe2ae19b700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 0x7fe2940a3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4003730 tx=0x7fe2a400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.013+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe2980964e0 con 0x7fe2a8101990 2023-10-18T09:18:00.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:59 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2277325388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:17:59 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1637435005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.230+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2a81a1230 con 0x7fe2a8101990 2023-10-18T09:18:00.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.230+0000 7fe29f7fe700 1 -- 172.21.15.61:0/395277110 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe298099b00 con 0x7fe2a8101990 2023-10-18T09:18:00.233 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:00.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.232+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 msgr2=0x7fe2940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.232+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 0x7fe2940a3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4003730 tx=0x7fe2a400b540 comp rx=0 tx=0).stop 2023-10-18T09:18:00.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 msgr2=0x7fe2a8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9a0 tx=0x7fe29800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 shutdown_connections 2023-10-18T09:18:00.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2940a1590 0x7fe2940a3a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2a8102330 0x7fe2a819fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2a81007a0 0x7fe2a8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 --2- 172.21.15.61:0/395277110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2a8101990 0x7fe2a8198ea0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 >> 172.21.15.61:0/395277110 conn(0x7fe2a80fbcf0 msgr2=0x7fe2a8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:00.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 shutdown_connections 2023-10-18T09:18:00.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.233+0000 7fe2b03ff700 1 -- 172.21.15.61:0/395277110 wait complete. 2023-10-18T09:18:00.252 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:59 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2277325388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.252 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:17:59 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1637435005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.254 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:59 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2277325388' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:17:59 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1637435005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:00.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 -- 172.21.15.61:0/49442752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36241012d0 msgr2=0x7f36241036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 --2- 172.21.15.61:0/49442752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36241012d0 0x7f36241036c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3620009990 tx=0x7f3620009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 -- 172.21.15.61:0/49442752 shutdown_connections 2023-10-18T09:18:00.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 --2- 172.21.15.61:0/49442752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3624106530 0x7f3624108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 --2- 172.21.15.61:0/49442752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36241012d0 0x7f36241036c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 --2- 172.21.15.61:0/49442752 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624103c00 0x7f3624105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 -- 172.21.15.61:0/49442752 >> 172.21.15.61:0/49442752 conn(0x7f36240fac00 msgr2=0x7f36240fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:00.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 -- 172.21.15.61:0/49442752 shutdown_connections 2023-10-18T09:18:00.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.460+0000 7f362b7ff700 1 -- 172.21.15.61:0/49442752 wait complete. 2023-10-18T09:18:00.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 Processor -- start 2023-10-18T09:18:00.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 -- start start 2023-10-18T09:18:00.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36241012d0 0x7f3624100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624103c00 0x7f36240ff280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36240fff60 con 0x7f3624106530 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241000d0 con 0x7f36241012d0 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.461+0000 7f362b7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36241a06c0 con 0x7f3624103c00 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42410/0 (socket says 172.21.15.61:42410) 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3628d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624103c00 0x7f36240ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 -- 172.21.15.61:0/1717050195 learned_addr learned my addr 172.21.15.61:0/1717050195 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f362959b700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36241012d0 0x7f3624100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 -- 172.21.15.61:0/1717050195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624103c00 msgr2=0x7f36240ff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624103c00 0x7f36240ff280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 -- 172.21.15.61:0/1717050195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36241012d0 msgr2=0x7f3624100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36241012d0 0x7f3624100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 -- 172.21.15.61:0/1717050195 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200095f0 con 0x7f3624106530 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.462+0000 7f3629d9c700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f361c00ef30 tx=0x7f361c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.463+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c018070 con 0x7f3624106530 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.463+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f361c00f040 con 0x7f3624106530 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.463+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c00cca0 con 0x7f3624106530 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.463+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36241a08f0 con 0x7f3624106530 2023-10-18T09:18:00.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.463+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36241a0db0 con 0x7f3624106530 2023-10-18T09:18:00.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.464+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36240fc780 con 0x7f3624106530 2023-10-18T09:18:00.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.466+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f361c003a00 con 0x7f3624106530 2023-10-18T09:18:00.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.468+0000 7f361a7fc700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 0x7f36100a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.468+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f361c0ccc60 con 0x7f3624106530 2023-10-18T09:18:00.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.469+0000 7f362959b700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 0x7f36100a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.469+0000 7f362959b700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 0x7f36100a3a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f362000b2e0 tx=0x7f362000b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.473+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f361c095e80 con 0x7f3624106530 2023-10-18T09:18:00.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.685+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f362410bf70 con 0x7f3624106530 2023-10-18T09:18:00.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.686+0000 7f361a7fc700 1 -- 172.21.15.61:0/1717050195 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f361c084020 con 0x7f3624106530 2023-10-18T09:18:00.688 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:00.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 msgr2=0x7f36100a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 0x7f36100a3a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f362000b2e0 tx=0x7f362000b750 comp rx=0 tx=0).stop 2023-10-18T09:18:00.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 msgr2=0x7f36240ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f361c00ef30 tx=0x7f361c00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 shutdown_connections 2023-10-18T09:18:00.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f36100a1590 0x7f36100a3a50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3624106530 0x7f36240ff7c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36241012d0 0x7f3624100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 --2- 172.21.15.61:0/1717050195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3624103c00 0x7f36240ff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 >> 172.21.15.61:0/1717050195 conn(0x7f36240fac00 msgr2=0x7f3624101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:00.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 shutdown_connections 2023-10-18T09:18:00.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.688+0000 7f362b7ff700 1 -- 172.21.15.61:0/1717050195 wait complete. 2023-10-18T09:18:00.712 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:00.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.913+0000 7f42c43ff700 1 -- 172.21.15.61:0/2588798338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc100670 msgr2=0x7f42bc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.913+0000 7f42c43ff700 1 --2- 172.21.15.61:0/2588798338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc100670 0x7f42bc100a90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42b80099c0 tx=0x7f42b8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 -- 172.21.15.61:0/2588798338 shutdown_connections 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 --2- 172.21.15.61:0/2588798338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 --2- 172.21.15.61:0/2588798338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc100670 0x7f42bc100a90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 --2- 172.21.15.61:0/2588798338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc101860 0x7f42bc101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 -- 172.21.15.61:0/2588798338 >> 172.21.15.61:0/2588798338 conn(0x7f42bc0fbba0 msgr2=0x7f42bc0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 -- 172.21.15.61:0/2588798338 shutdown_connections 2023-10-18T09:18:00.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.914+0000 7f42c43ff700 1 -- 172.21.15.61:0/2588798338 wait complete. 2023-10-18T09:18:00.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 Processor -- start 2023-10-18T09:18:00.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 -- start start 2023-10-18T09:18:00.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc100670 0x7f42bc196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 0x7f42bc196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc19e120 con 0x7f42bc101860 2023-10-18T09:18:00.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc19e260 con 0x7f42bc100670 2023-10-18T09:18:00.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c43ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc19e3f0 con 0x7f42bc102200 2023-10-18T09:18:00.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c199a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 0x7f42bc196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c199a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 0x7f42bc196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42422/0 (socket says 172.21.15.61:42422) 2023-10-18T09:18:00.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c199a700 1 -- 172.21.15.61:0/4280405814 learned_addr learned my addr 172.21.15.61:0/4280405814 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:00.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c219b700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc100670 0x7f42bc196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.915+0000 7f42c299c700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 -- 172.21.15.61:0/4280405814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc100670 msgr2=0x7f42bc196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc100670 0x7f42bc196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 -- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 msgr2=0x7f42bc196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 0x7f42bc196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 -- 172.21.15.61:0/4280405814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80095f0 con 0x7f42bc102200 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c299c700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc19db50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b330 tx=0x7f42b400b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4015070 con 0x7f42bc102200 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42b400bdd0 con 0x7f42bc102200 2023-10-18T09:18:00.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc072a20 con 0x7f42bc102200 2023-10-18T09:18:00.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.916+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4007c20 con 0x7f42bc102200 2023-10-18T09:18:00.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.917+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc072ee0 con 0x7f42bc102200 2023-10-18T09:18:00.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.917+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc107b90 con 0x7f42bc102200 2023-10-18T09:18:00.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.920+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f42b4007500 con 0x7f42bc102200 2023-10-18T09:18:00.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.921+0000 7f42b37fe700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 0x7f42a80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:00.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.921+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42b40d1170 con 0x7f42bc102200 2023-10-18T09:18:00.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.921+0000 7f42c219b700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 0x7f42a80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:00.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.922+0000 7f42c219b700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 0x7f42a80a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009990 tx=0x7f42b8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:00.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:00.923+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f42b409a300 con 0x7f42bc102200 2023-10-18T09:18:01.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:00 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/395277110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:00 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1717050195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:00 smithi061 ceph-mon[98573]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:01.191 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.189+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42bc0673e0 con 0x7f42bc102200 2023-10-18T09:18:01.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.190+0000 7f42b37fe700 1 -- 172.21.15.61:0/4280405814 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f42b409d920 con 0x7f42bc102200 2023-10-18T09:18:01.193 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:01.193 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:01.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 msgr2=0x7f42a80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 0x7f42a80a39d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009990 tx=0x7f42b8000bc0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 msgr2=0x7f42bc19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc19db50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b330 tx=0x7f42b400b640 comp rx=0 tx=0).stop 2023-10-18T09:18:01.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 shutdown_connections 2023-10-18T09:18:01.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f42a80a1510 0x7f42a80a39d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42bc102200 0x7f42bc19db50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42bc100670 0x7f42bc196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 --2- 172.21.15.61:0/4280405814 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f42bc101860 0x7f42bc196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 >> 172.21.15.61:0/4280405814 conn(0x7f42bc0fbba0 msgr2=0x7f42bc104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:01.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 shutdown_connections 2023-10-18T09:18:01.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.193+0000 7f42c43ff700 1 -- 172.21.15.61:0/4280405814 wait complete. 2023-10-18T09:18:01.198 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:01.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:00 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/395277110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:00 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1717050195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:00 smithi110 ceph-mon[134818]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:01.215 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:00 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/395277110' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:00 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1717050195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:00 smithi170 ceph-mon[135150]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:01.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 -- 172.21.15.61:0/2490945213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 msgr2=0x7f5fec108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/2490945213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec108b80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400b0d0 tx=0x7f5fe400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 -- 172.21.15.61:0/2490945213 shutdown_connections 2023-10-18T09:18:01.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/2490945213 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec108b80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/2490945213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec100740 0x7f5fec100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/2490945213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec101930 0x7f5fec101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 -- 172.21.15.61:0/2490945213 >> 172.21.15.61:0/2490945213 conn(0x7f5fec0fbc90 msgr2=0x7f5fec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:01.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 -- 172.21.15.61:0/2490945213 shutdown_connections 2023-10-18T09:18:01.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.417+0000 7f5ff15ff700 1 -- 172.21.15.61:0/2490945213 wait complete. 2023-10-18T09:18:01.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.418+0000 7f5ff15ff700 1 Processor -- start 2023-10-18T09:18:01.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.418+0000 7f5ff15ff700 1 -- start start 2023-10-18T09:18:01.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec100740 0x7f5fec1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101930 0x7f5fec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec19e2f0 con 0x7f5fec1022d0 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec19e430 con 0x7f5fec101930 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5ff15ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec19e5c0 con 0x7f5fec100740 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feaffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec100740 0x7f5fec1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42436/0 (socket says 172.21.15.61:42436) 2023-10-18T09:18:01.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 -- 172.21.15.61:0/3174998751 learned_addr learned my addr 172.21.15.61:0/3174998751 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:01.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5fea7fc700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101930 0x7f5fec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 -- 172.21.15.61:0/3174998751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec100740 msgr2=0x7f5fec1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec100740 0x7f5fec1967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 -- 172.21.15.61:0/3174998751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101930 msgr2=0x7f5fec196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101930 0x7f5fec196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.419+0000 7f5feb7fe700 1 -- 172.21.15.61:0/3174998751 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009d00 con 0x7f5fec1022d0 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5feb7fe700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007b60 tx=0x7f5fe4008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400e040 con 0x7f5fec1022d0 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fe401d930 con 0x7f5fec1022d0 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec19e840 con 0x7f5fec1022d0 2023-10-18T09:18:01.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe401b9f0 con 0x7f5fec1022d0 2023-10-18T09:18:01.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.420+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec19ecc0 con 0x7f5fec1022d0 2023-10-18T09:18:01.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.421+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec107af0 con 0x7f5fec1022d0 2023-10-18T09:18:01.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.423+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f5fe401daa0 con 0x7f5fec1022d0 2023-10-18T09:18:01.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.424+0000 7f5fd3fff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 0x7f5fd40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.424+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5fe40ce5e0 con 0x7f5fec1022d0 2023-10-18T09:18:01.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.425+0000 7f5feaffd700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 0x7f5fd40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.425+0000 7f5feaffd700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 0x7f5fd40a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5fec101790 tx=0x7f5fdc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:01.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.428+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5fe4097800 con 0x7f5fec1022d0 2023-10-18T09:18:01.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.647+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fec10bfb0 con 0x7f5fec1022d0 2023-10-18T09:18:01.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.649+0000 7f5fd3fff700 1 -- 172.21.15.61:0/3174998751 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24394 (secure 0 0 0) 0x7f5fe409ae20 con 0x7f5fec1022d0 2023-10-18T09:18:01.651 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:01.652 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":90,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:17:21.904505+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:01.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 msgr2=0x7f5fd40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 0x7f5fd40a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5fec101790 tx=0x7f5fdc009260 comp rx=0 tx=0).stop 2023-10-18T09:18:01.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 msgr2=0x7f5fec19dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007b60 tx=0x7f5fe4008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:01.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 shutdown_connections 2023-10-18T09:18:01.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5fd40a1510 0x7f5fd40a39d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5fec1022d0 0x7f5fec19dd20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fec100740 0x7f5fec1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 --2- 172.21.15.61:0/3174998751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5fec101930 0x7f5fec196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.651+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 >> 172.21.15.61:0/3174998751 conn(0x7f5fec0fbc90 msgr2=0x7f5fec104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:01.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.652+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 shutdown_connections 2023-10-18T09:18:01.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.652+0000 7f5ff15ff700 1 -- 172.21.15.61:0/3174998751 wait complete. 2023-10-18T09:18:01.673 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:01.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 -- 172.21.15.61:0/3835526077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 msgr2=0x7fbda0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 --2- 172.21.15.61:0/3835526077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0105eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009990 tx=0x7fbd94009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 -- 172.21.15.61:0/3835526077 shutdown_connections 2023-10-18T09:18:01.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 --2- 172.21.15.61:0/3835526077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda01063f0 0x7fbda0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 --2- 172.21.15.61:0/3835526077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda0101190 0x7fbda0103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 --2- 172.21.15.61:0/3835526077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0105eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 -- 172.21.15.61:0/3835526077 >> 172.21.15.61:0/3835526077 conn(0x7fbda00faba0 msgr2=0x7fbda00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:01.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.879+0000 7fbda61ff700 1 -- 172.21.15.61:0/3835526077 shutdown_connections 2023-10-18T09:18:01.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.880+0000 7fbda61ff700 1 -- 172.21.15.61:0/3835526077 wait complete. 2023-10-18T09:18:01.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.880+0000 7fbda61ff700 1 Processor -- start 2023-10-18T09:18:01.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 -- start start 2023-10-18T09:18:01.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 0x7fbda019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019e180 con 0x7fbda0101190 2023-10-18T09:18:01.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019e2c0 con 0x7fbda01063f0 2023-10-18T09:18:01.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbda61ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda019e450 con 0x7fbda0103ac0 2023-10-18T09:18:01.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbd9f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbd9f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42452/0 (socket says 172.21.15.61:42452) 2023-10-18T09:18:01.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbd9f7fe700 1 -- 172.21.15.61:0/2306647594 learned_addr learned my addr 172.21.15.61:0/2306647594 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:01.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbd9effd700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.881+0000 7fbd9ffff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 0x7fbda019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 -- 172.21.15.61:0/2306647594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 msgr2=0x7fbda0196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 -- 172.21.15.61:0/2306647594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 msgr2=0x7fbda019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:01.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 0x7fbda019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:01.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 -- 172.21.15.61:0/2306647594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98009c40 con 0x7fbda0101190 2023-10-18T09:18:01.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9ffff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 0x7fbda019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:01.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9f7fe700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9a0 tx=0x7fbd9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:01.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90009790 con 0x7fbda0101190 2023-10-18T09:18:01.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd940095f0 con 0x7fbda0101190 2023-10-18T09:18:01.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9000cbd0 con 0x7fbda0101190 2023-10-18T09:18:01.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90018680 con 0x7fbda0101190 2023-10-18T09:18:01.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.882+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda019eab0 con 0x7fbda0101190 2023-10-18T09:18:01.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.883+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda00fc720 con 0x7fbda0101190 2023-10-18T09:18:01.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.885+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fbd9000cd40 con 0x7fbda0101190 2023-10-18T09:18:01.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.887+0000 7fbd9cff9700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 0x7fbd880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:01.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.887+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbd90014070 con 0x7fbda0101190 2023-10-18T09:18:01.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.887+0000 7fbd9effd700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 0x7fbd880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:01.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.888+0000 7fbd9effd700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 0x7fbd880a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400b2e0 tx=0x7fbd94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:01.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:01.889+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbd900968e0 con 0x7fbda0101190 2023-10-18T09:18:02.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.152+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbda0067530 con 0x7fbda0101190 2023-10-18T09:18:02.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.153+0000 7fbd9cff9700 1 -- 172.21.15.61:0/2306647594 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fbd90099f00 con 0x7fbda0101190 2023-10-18T09:18:02.155 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:02.155 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:02.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 msgr2=0x7fbd880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 0x7fbd880a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400b2e0 tx=0x7fbd94005fd0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 msgr2=0x7fbda0196650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000e9a0 tx=0x7fbd9000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 shutdown_connections 2023-10-18T09:18:02.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbd880a1510 0x7fbd880a39d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbda01063f0 0x7fbda019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbda0101190 0x7fbda0196650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 --2- 172.21.15.61:0/2306647594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbda0103ac0 0x7fbda0196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 >> 172.21.15.61:0/2306647594 conn(0x7fbda00faba0 msgr2=0x7fbda0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 shutdown_connections 2023-10-18T09:18:02.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.155+0000 7fbda61ff700 1 -- 172.21.15.61:0/2306647594 wait complete. 2023-10-18T09:18:02.160 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:02.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-18T09:16:26.396655+0000', 'modified': '2023-10-18T09:16:59.542245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24692, 'mds_1': 24676, 'mds_2': 24653, 'mds_3': 24646, 'mds_4': 24629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14688': {'gid': 14688, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.61:6841/2114349313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6840', 'nonce': 2114349313}, {'type': 'v1', 'addr': '172.21.15.61:6841', 'nonce': 2114349313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/2686391817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 2686391817}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 2686391817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/3590204423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 3590204423}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 3590204423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24629': {'gid': 24629, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.110:6835/145516333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 145516333}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 145516333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24646': {'gid': 24646, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.170:6835/1203136193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1203136193}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1203136193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24653': {'gid': 24653, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.110:6837/2750873982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2750873982}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2750873982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24658': {'gid': 24658, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/3025282224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 3025282224}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3025282224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24668': {'gid': 24668, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.61:6839/2550944546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6838', 'nonce': 2550944546}, {'type': 'v1', 'addr': '172.21.15.61:6839', 'nonce': 2550944546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24676': {'gid': 24676, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6839/2453939791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2453939791}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2453939791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24692': {'gid': 24692, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6839/1483417844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 1483417844}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 1483417844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:18:02.178 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-18T09:18:02.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-18T09:18:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:01 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4280405814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:01 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3174998751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.239 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:01 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4280405814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.239 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:01 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3174998751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 -- 172.21.15.61:0/3006643491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01006c0 msgr2=0x7f6de0100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 --2- 172.21.15.61:0/3006643491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01006c0 0x7f6de0100ae0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009920 tx=0x7f6ddc009c30 comp rx=0 tx=0).stop 2023-10-18T09:18:02.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 -- 172.21.15.61:0/3006643491 shutdown_connections 2023-10-18T09:18:02.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 --2- 172.21.15.61:0/3006643491 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de0102250 0x7f6de0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 --2- 172.21.15.61:0/3006643491 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01006c0 0x7f6de0100ae0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 --2- 172.21.15.61:0/3006643491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de01018b0 0x7f6de0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 -- 172.21.15.61:0/3006643491 >> 172.21.15.61:0/3006643491 conn(0x7f6de00fbc30 msgr2=0x7f6de00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 -- 172.21.15.61:0/3006643491 shutdown_connections 2023-10-18T09:18:02.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.381+0000 7f6de77ff700 1 -- 172.21.15.61:0/3006643491 wait complete. 2023-10-18T09:18:02.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.382+0000 7f6de77ff700 1 Processor -- start 2023-10-18T09:18:02.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 -- start start 2023-10-18T09:18:02.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de01006c0 0x7f6de01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 0x7f6de019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01a0480 con 0x7f6de01018b0 2023-10-18T09:18:02.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01a05c0 con 0x7f6de01006c0 2023-10-18T09:18:02.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de77ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01a0750 con 0x7f6de0102250 2023-10-18T09:18:02.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42464/0 (socket says 172.21.15.61:42464) 2023-10-18T09:18:02.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de4d9a700 1 -- 172.21.15.61:0/400723167 learned_addr learned my addr 172.21.15.61:0/400723167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 0x7f6de019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.383+0000 7f6de559b700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de01006c0 0x7f6de01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 -- 172.21.15.61:0/400723167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 msgr2=0x7f6de019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 0x7f6de019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 -- 172.21.15.61:0/400723167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de01006c0 msgr2=0x7f6de01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de01006c0 0x7f6de01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 -- 172.21.15.61:0/400723167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009520 con 0x7f6de01018b0 2023-10-18T09:18:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 0x7f6de019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9f0 tx=0x7f6dd000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6dd67fc700 1 -- 172.21.15.61:0/400723167 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0004a30 con 0x7f6de01018b0 2023-10-18T09:18:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6dd67fc700 1 -- 172.21.15.61:0/400723167 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd000cc80 con 0x7f6de01018b0 2023-10-18T09:18:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de01a0a30 con 0x7f6de01018b0 2023-10-18T09:18:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.384+0000 7f6dd67fc700 1 -- 172.21.15.61:0/400723167 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0005530 con 0x7f6de01018b0 2023-10-18T09:18:02.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.385+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de01a0f40 con 0x7f6de01018b0 2023-10-18T09:18:02.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:01 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4280405814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:01 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3174998751' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.388+0000 7f6dd67fc700 1 -- 172.21.15.61:0/400723167 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6dd0004b90 con 0x7f6de01018b0 2023-10-18T09:18:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.389+0000 7f6dfb027b80 1 Processor -- start 2023-10-18T09:18:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.389+0000 7f6dd67fc700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 0x7f6dcc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.389+0000 7f6dfb027b80 1 -- start start 2023-10-18T09:18:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.389+0000 7f6dd67fc700 1 -- 172.21.15.61:0/400723167 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6dd0014070 con 0x7f6de01018b0 2023-10-18T09:18:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.389+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 0x7f6dcc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf359690 con 0x55dbaf3e7be0 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf33fc10 con 0x55dbaf3e7280 2023-10-18T09:18:02.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf33d530 con 0x55dbaf3e6580 2023-10-18T09:18:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de559b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de559b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54128/0 (socket says 172.21.15.61:54128) 2023-10-18T09:18:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de559b700 1 -- 172.21.15.61:0/669025147 learned_addr learned my addr 172.21.15.61:0/669025147 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 0x7f6dcc0a3a50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8003e80 tx=0x7f6dd800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 -- 172.21.15.61:0/669025147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 msgr2=0x55dbaf3e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 -- 172.21.15.61:0/669025147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 msgr2=0x55dbaf3e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de4d9a700 1 -- 172.21.15.61:0/669025147 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbaf2f63a0 con 0x55dbaf3e7be0 2023-10-18T09:18:02.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.390+0000 7f6de559b700 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:02.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3e6040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ca930 tx=0x7f6dd00965b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4780437ccd9d876d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6dcb7fe700 1 -- 172.21.15.61:0/669025147 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd009e290 con 0x55dbaf3e7be0 2023-10-18T09:18:02.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6dcb7fe700 1 -- 172.21.15.61:0/669025147 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd00a24d0 con 0x55dbaf3e7be0 2023-10-18T09:18:02.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6dcb7fe700 1 -- 172.21.15.61:0/669025147 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0099630 con 0x55dbaf3e7be0 2023-10-18T09:18:02.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6dfb027b80 1 -- 172.21.15.61:0/669025147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 msgr2=0x55dbaf3e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.391+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3e6040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ca930 tx=0x7f6dd00965b0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 -- 172.21.15.61:0/669025147 shutdown_connections 2023-10-18T09:18:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/669025147 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3e6040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 -- 172.21.15.61:0/669025147 >> 172.21.15.61:0/669025147 conn(0x55dbaf35a6d0 msgr2=0x55dbaf3e0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 -- 172.21.15.61:0/669025147 shutdown_connections 2023-10-18T09:18:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 -- 172.21.15.61:0/669025147 wait complete. 2023-10-18T09:18:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 -- start start 2023-10-18T09:18:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.392+0000 7f6dfb027b80 1 Processor -- start 2023-10-18T09:18:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e6580 0x55dbaf3efb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf338e20 con 0x55dbaf3e7280 2023-10-18T09:18:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf2d56c0 con 0x55dbaf3e6580 2023-10-18T09:18:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf2d5360 con 0x55dbaf3e7be0 2023-10-18T09:18:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6dfb027b80 10 client.0 _ll_register_callbacks cb 0x55dbaf21fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42468/0 (socket says 172.21.15.61:42468) 2023-10-18T09:18:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6de4d9a700 1 -- 172.21.15.61:0/4294772197 learned_addr learned my addr 172.21.15.61:0/4294772197 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.393+0000 7f6de559b700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e6580 0x55dbaf3efb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 -- 172.21.15.61:0/4294772197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 msgr2=0x55dbaf3f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 -- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e6580 msgr2=0x55dbaf3efb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e6580 0x55dbaf3efb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 -- 172.21.15.61:0/4294772197 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de0109450 con 0x55dbaf3e7280 2023-10-18T09:18:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ca960 tx=0x7f6dd009e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355086979e2b011 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd009e290 con 0x55dbaf3e7280 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6dd0099630 con 0x55dbaf3e7280 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbaf33dea0 con 0x55dbaf3e7280 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd00997f0 con 0x55dbaf3e7280 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dfb027b80 10 client.15273 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.394+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f6de01a0f40 con 0x55dbaf3e7280 2023-10-18T09:18:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dbaf33dea0 con 0x55dbaf3e7280 2023-10-18T09:18:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6dd0095f30 con 0x55dbaf3e7280 2023-10-18T09:18:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 10 client.15273 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 10 client.15273 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-18T09:18:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 20 client.15273 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf3fb290 0x55dbaf3fa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 4 client.15273 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbaf3b5860 con 0x55dbaf3fb290 2023-10-18T09:18:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.395+0000 7f6de559b700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf3fb290 0x55dbaf3fa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.396+0000 7f6de559b700 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf3fb290 0x55dbaf3fa060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f923e3a4e1e10e63 server_cookie=11cb267b196a6475 in_seq=0 out_seq=0 2023-10-18T09:18:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.396+0000 7f6dc8ff9700 10 client.15273 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/4294772197 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dbaf3b5860 con 0x55dbaf3fb290 2023-10-18T09:18:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dc8ff9700 10 client.15273 handle_command_reply: tid=0 2023-10-18T09:18:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 client.15273 shutdown 2023-10-18T09:18:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 msgr2=0x55dbaf3f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ca960 tx=0x7f6dd009e170 comp rx=0 tx=0).stop 2023-10-18T09:18:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 shutdown_connections 2023-10-18T09:18:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf3fb290 0x55dbaf3fa060 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dc8ff9700 0 client.15273 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e6580 0x55dbaf3efb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/4294772197 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f0050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 >> 172.21.15.61:0/4294772197 conn(0x55dbaf35a6d0 msgr2=0x55dbaf3e5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 shutdown_connections 2023-10-18T09:18:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 1 -- 172.21.15.61:0/4294772197 wait complete. 2023-10-18T09:18:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.397+0000 7f6dfb027b80 20 client.15273 trim_cache size 0 max 16384 2023-10-18T09:18:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 Processor -- start 2023-10-18T09:18:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 -- start start 2023-10-18T09:18:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3db580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf1b98d0 con 0x55dbaf3e6580 2023-10-18T09:18:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf3b5860 con 0x55dbaf3e7280 2023-10-18T09:18:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf2f0720 con 0x55dbaf3e7be0 2023-10-18T09:18:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6de559b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6de559b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42478/0 (socket says 172.21.15.61:42478) 2023-10-18T09:18:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6de559b700 1 -- 172.21.15.61:0/2828398636 learned_addr learned my addr 172.21.15.61:0/2828398636 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.416+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 -- 172.21.15.61:0/2828398636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 msgr2=0x55dbaf3f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 -- 172.21.15.61:0/2828398636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 msgr2=0x55dbaf3db580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3db580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 -- 172.21.15.61:0/2828398636 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbaf163d90 con 0x55dbaf3e6580 2023-10-18T09:18:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6de559b700 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66753f839d6657da server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6dcbfff700 1 -- 172.21.15.61:0/2828398636 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc028cf0 con 0x55dbaf3e6580 2023-10-18T09:18:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6dcbfff700 1 -- 172.21.15.61:0/2828398636 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ddc01d040 con 0x55dbaf3e6580 2023-10-18T09:18:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.417+0000 7f6dcbfff700 1 -- 172.21.15.61:0/2828398636 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc005040 con 0x55dbaf3e6580 2023-10-18T09:18:02.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2828398636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 msgr2=0x55dbaf3e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc009f80 comp rx=0 tx=0).stop 2023-10-18T09:18:02.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2828398636 shutdown_connections 2023-10-18T09:18:02.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e6580 0x55dbaf3e6040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2828398636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7280 0x55dbaf3db580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2828398636 >> 172.21.15.61:0/2828398636 conn(0x55dbaf35a6d0 msgr2=0x55dbaf3e6960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2828398636 shutdown_connections 2023-10-18T09:18:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2828398636 wait complete. 2023-10-18T09:18:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 -- start start 2023-10-18T09:18:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.418+0000 7f6dfb027b80 1 Processor -- start 2023-10-18T09:18:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf3f4650 con 0x55dbaf3e7280 2023-10-18T09:18:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf44ec80 con 0x55dbaf3e7be0 2023-10-18T09:18:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbaf44edc0 con 0x55dbaf3e6580 2023-10-18T09:18:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6dfb027b80 10 client.0 _ll_register_callbacks cb 0x55dbaf21fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6de4d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42482/0 (socket says 172.21.15.61:42482) 2023-10-18T09:18:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6de4d9a700 1 -- 172.21.15.61:0/2972024043 learned_addr learned my addr 172.21.15.61:0/2972024043 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6de559b700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.419+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 -- 172.21.15.61:0/2972024043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 msgr2=0x55dbaf3f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 -- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 msgr2=0x55dbaf3f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 -- 172.21.15.61:0/2972024043 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbaf14cc10 con 0x55dbaf3e7280 2023-10-18T09:18:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de5d9c700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6de4d9a700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0096310 tx=0x7f6dd0004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf9e550609946a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dc8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00b8180 con 0x55dbaf3e7280 2023-10-18T09:18:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6dd00a24d0 con 0x55dbaf3e7280 2023-10-18T09:18:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6dd00b83e0 con 0x55dbaf3e7280 2023-10-18T09:18:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6dd009e290 con 0x55dbaf3e7280 2023-10-18T09:18:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dfb027b80 10 client.15279 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55dbaf18a200 con 0x55dbaf3e7280 2023-10-18T09:18:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.420+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd00a24d0 con 0x55dbaf3e7280 2023-10-18T09:18:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6dd000c760 con 0x55dbaf3e7280 2023-10-18T09:18:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 10 client.15279 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-18T09:18:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 20 client.15279 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf453ad0 0x55dbaf450020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 4 client.15279 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dbaf331e00 con 0x55dbaf453ad0 2023-10-18T09:18:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.421+0000 7f6de559b700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf453ad0 0x55dbaf450020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6de559b700 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf453ad0 0x55dbaf450020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f44df3148204121 server_cookie=f2c8fdfd0815ad4e in_seq=0 out_seq=0 2023-10-18T09:18:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dc8ff9700 10 client.15279 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dc8ff9700 1 -- 172.21.15.61:0/2972024043 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55dbaf331e00 con 0x55dbaf453ad0 2023-10-18T09:18:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dc8ff9700 10 client.15279 handle_command_reply: tid=0 2023-10-18T09:18:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 client.15279 shutdown 2023-10-18T09:18:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 msgr2=0x55dbaf3f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0096310 tx=0x7f6dd0004e40 comp rx=0 tx=0).stop 2023-10-18T09:18:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 shutdown_connections 2023-10-18T09:18:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55dbaf453ad0 0x55dbaf450020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dbaf3e7be0 0x55dbaf3f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaf3e6580 0x55dbaf3f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 --2- 172.21.15.61:0/2972024043 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55dbaf3e7280 0x55dbaf3f3af0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 >> 172.21.15.61:0/2972024043 conn(0x55dbaf35a6d0 msgr2=0x55dbaf3e0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dc8ff9700 0 client.15279 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 shutdown_connections 2023-10-18T09:18:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.422+0000 7f6dfb027b80 1 -- 172.21.15.61:0/2972024043 wait complete. 2023-10-18T09:18:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6dfb027b80 20 client.15279 trim_cache size 0 max 16384 2023-10-18T09:18:02.441 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 4, 2023-10-18T09:18:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24629, 2023-10-18T09:18:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 63.898155066000001, 2023-10-18T09:18:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 90, 2023-10-18T09:18:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 119.766253988 2023-10-18T09:18:02.443 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 msgr2=0x7f6dcc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 0x7f6dcc0a3a50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8003e80 tx=0x7f6dd800b040 comp rx=0 tx=0).stop 2023-10-18T09:18:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 msgr2=0x7f6de0198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9f0 tx=0x7f6dd000ed00 comp rx=0 tx=0).stop 2023-10-18T09:18:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 shutdown_connections 2023-10-18T09:18:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6dcc0a1590 0x7f6dcc0a3a50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6de0102250 0x7f6de019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6de01006c0 0x7f6de01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 --2- 172.21.15.61:0/400723167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6de01018b0 0x7f6de0198ee0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 >> 172.21.15.61:0/400723167 conn(0x7f6de00fbc30 msgr2=0x7f6de01070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 shutdown_connections 2023-10-18T09:18:02.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.423+0000 7f6de77ff700 1 -- 172.21.15.61:0/400723167 wait complete. 2023-10-18T09:18:02.447 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-18T09:18:02.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 -- 172.21.15.61:0/1691340534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 msgr2=0x7f1e9c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1691340534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c1009e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0099c0 tx=0x7f1e8c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 -- 172.21.15.61:0/1691340534 shutdown_connections 2023-10-18T09:18:02.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1691340534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c102150 0x7f1e9c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1691340534 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c1009e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1691340534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.654+0000 7f1ea099d700 1 -- 172.21.15.61:0/1691340534 >> 172.21.15.61:0/1691340534 conn(0x7f1e9c0fbaf0 msgr2=0x7f1e9c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.655+0000 7f1ea099d700 1 -- 172.21.15.61:0/1691340534 shutdown_connections 2023-10-18T09:18:02.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.655+0000 7f1ea099d700 1 -- 172.21.15.61:0/1691340534 wait complete. 2023-10-18T09:18:02.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.655+0000 7f1ea099d700 1 Processor -- start 2023-10-18T09:18:02.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 -- start start 2023-10-18T09:18:02.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c102150 0x7f1e9c19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19e0c0 con 0x7f1e9c1005c0 2023-10-18T09:18:02.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19e200 con 0x7f1e9c1017b0 2023-10-18T09:18:02.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1ea099d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c19e390 con 0x7f1e9c102150 2023-10-18T09:18:02.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1e9affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42490/0 (socket says 172.21.15.61:42490) 2023-10-18T09:18:02.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1863529584 learned_addr learned my addr 172.21.15.61:0/1863529584 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.656+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c102150 0x7f1e9c19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 -- 172.21.15.61:0/1863529584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c102150 msgr2=0x7f1e9c19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c102150 0x7f1e9c19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 -- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 msgr2=0x7f1e9c196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 -- 172.21.15.61:0/1863529584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0095f0 con 0x7f1e9c1017b0 2023-10-18T09:18:02.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c196ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1e98ff9700 1 -- 172.21.15.61:0/1863529584 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9000c9c0 con 0x7f1e9c1017b0 2023-10-18T09:18:02.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c19e670 con 0x7f1e9c1017b0 2023-10-18T09:18:02.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.657+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c19ebd0 con 0x7f1e9c1017b0 2023-10-18T09:18:02.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.658+0000 7f1e98ff9700 1 -- 172.21.15.61:0/1863529584 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e90010430 con 0x7f1e9c1017b0 2023-10-18T09:18:02.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.658+0000 7f1e98ff9700 1 -- 172.21.15.61:0/1863529584 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90018680 con 0x7f1e9c1017b0 2023-10-18T09:18:02.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.661+0000 7f1e98ff9700 1 -- 172.21.15.61:0/1863529584 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f1e90010810 con 0x7f1e9c1017b0 2023-10-18T09:18:02.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.662+0000 7f1eb54a1b80 1 Processor -- start 2023-10-18T09:18:02.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.662+0000 7f1eb54a1b80 1 -- start start 2023-10-18T09:18:02.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c8128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c813260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c813c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c785690 con 0x55ac2c812480 2023-10-18T09:18:02.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c76bc10 con 0x55ac2c8137a0 2023-10-18T09:18:02.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c769530 con 0x55ac2c812de0 2023-10-18T09:18:02.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e9bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c8128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e9bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c8128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42504/0 (socket says 172.21.15.61:42504) 2023-10-18T09:18:02.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c813260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e9bfff700 1 -- 172.21.15.61:0/1548293398 learned_addr learned my addr 172.21.15.61:0/1548293398 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e98ff9700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 0x7f1e840a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c813c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.663+0000 7f1e98ff9700 1 -- 172.21.15.61:0/1863529584 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e90014070 con 0x7f1e9c1017b0 2023-10-18T09:18:02.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1548293398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 msgr2=0x55ac2c813c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c813c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1548293398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 msgr2=0x55ac2c8128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c8128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 0x7f1e840a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1548293398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2c7223a0 con 0x55ac2c812de0 2023-10-18T09:18:02.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c813260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c006010 tx=0x7f1e8c005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21182a3aeec942 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e81ffb700 1 -- 172.21.15.61:0/1548293398 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c00ba10 con 0x55ac2c812de0 2023-10-18T09:18:02.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 0x7f1e840a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400fb80 tx=0x7f1e94004b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e81ffb700 1 -- 172.21.15.61:0/1548293398 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e8c01e040 con 0x55ac2c812de0 2023-10-18T09:18:02.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.664+0000 7f1e81ffb700 1 -- 172.21.15.61:0/1548293398 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c003b30 con 0x55ac2c812de0 2023-10-18T09:18:02.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1548293398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 msgr2=0x55ac2c813260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c813260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c006010 tx=0x7f1e8c005bc0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1548293398 shutdown_connections 2023-10-18T09:18:02.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c813c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c8128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1548293398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c813260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1548293398 >> 172.21.15.61:0/1548293398 conn(0x55ac2c7866d0 msgr2=0x55ac2c80ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1548293398 shutdown_connections 2023-10-18T09:18:02.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.665+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1548293398 wait complete. 2023-10-18T09:18:02.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.666+0000 7f1eb54a1b80 1 -- start start 2023-10-18T09:18:02.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.666+0000 7f1eb54a1b80 1 Processor -- start 2023-10-18T09:18:02.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c812480 0x55ac2c81bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c8137a0 0x55ac2c823190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c764e20 con 0x55ac2c812de0 2023-10-18T09:18:02.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c7016c0 con 0x55ac2c812480 2023-10-18T09:18:02.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c701360 con 0x55ac2c8137a0 2023-10-18T09:18:02.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1eb54a1b80 10 client.0 _ll_register_callbacks cb 0x55ac2c64ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1e9affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1e9bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c8137a0 0x55ac2c823190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1e9affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42512/0 (socket says 172.21.15.61:42512) 2023-10-18T09:18:02.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1e9affd700 1 -- 172.21.15.61:0/2774934546 learned_addr learned my addr 172.21.15.61:0/2774934546 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.667+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c812480 0x55ac2c81bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 -- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c8137a0 msgr2=0x55ac2c823190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c8137a0 0x55ac2c823190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 -- 172.21.15.61:0/2774934546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c812480 msgr2=0x55ac2c81bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c812480 0x55ac2c81bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 -- 172.21.15.61:0/2774934546 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c109350 con 0x55ac2c812de0 2023-10-18T09:18:02.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e9affd700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e90096c10 tx=0x7f1e900696a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8180c0e0525cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e7affd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90099890 con 0x55ac2c812de0 2023-10-18T09:18:02.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e90065a60 con 0x55ac2c812de0 2023-10-18T09:18:02.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac2c769ea0 con 0x55ac2c812de0 2023-10-18T09:18:02.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e900cd170 con 0x55ac2c812de0 2023-10-18T09:18:02.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1eb54a1b80 10 client.15291 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.668+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ac2c58fd90 con 0x55ac2c812de0 2023-10-18T09:18:02.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac2c769ea0 con 0x55ac2c812de0 2023-10-18T09:18:02.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1e900a20a0 con 0x55ac2c812de0 2023-10-18T09:18:02.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-18T09:18:02.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 20 client.15291 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c827820 0x55ac2c826330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 4 client.15291 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac2c7e1860 con 0x55ac2c827820 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.669+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c827820 0x55ac2c826330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.670+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c827820 0x55ac2c826330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c435d56343c4b15e server_cookie=b2c7f9580268e003 in_seq=0 out_seq=0 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.670+0000 7f1e7affd700 10 client.15291 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1e7affd700 1 -- 172.21.15.61:0/2774934546 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac2c7e1860 con 0x55ac2c827820 2023-10-18T09:18:02.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1e7affd700 10 client.15291 handle_command_reply: tid=0 2023-10-18T09:18:02.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 client.15291 shutdown 2023-10-18T09:18:02.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 msgr2=0x55ac2c81c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1e90096c10 tx=0x7f1e900696a0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 shutdown_connections 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c827820 0x55ac2c826330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c8137a0 0x55ac2c823190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c812480 0x55ac2c81bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1e7affd700 0 client.15291 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/2774934546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81c150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 >> 172.21.15.61:0/2774934546 conn(0x55ac2c7866d0 msgr2=0x55ac2c811280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 shutdown_connections 2023-10-18T09:18:02.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/2774934546 wait complete. 2023-10-18T09:18:02.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.671+0000 7f1eb54a1b80 20 client.15291 trim_cache size 0 max 16384 2023-10-18T09:18:02.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.689+0000 7f1eb54a1b80 1 Processor -- start 2023-10-18T09:18:02.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.689+0000 7f1eb54a1b80 1 -- start start 2023-10-18T09:18:02.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812480 0x55ac2c8077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c819ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c5e58d0 con 0x55ac2c812de0 2023-10-18T09:18:02.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c80e790 con 0x55ac2c8137a0 2023-10-18T09:18:02.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c71c720 con 0x55ac2c812480 2023-10-18T09:18:02.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42528/0 (socket says 172.21.15.61:42528) 2023-10-18T09:18:02.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 -- 172.21.15.61:0/3164768775 learned_addr learned my addr 172.21.15.61:0/3164768775 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812480 0x55ac2c8077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c819ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 -- 172.21.15.61:0/3164768775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812480 msgr2=0x55ac2c8077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812480 0x55ac2c8077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 -- 172.21.15.61:0/3164768775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 msgr2=0x55ac2c819ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c819ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9affd700 1 -- 172.21.15.61:0/3164768775 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac2c578c10 con 0x55ac2c812de0 2023-10-18T09:18:02.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.690+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c819ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1e9affd700 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1e90099e50 tx=0x7f1e90098750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdf84e14f9ce076 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1e827fc700 1 -- 172.21.15.61:0/3164768775 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900a22a0 con 0x55ac2c812de0 2023-10-18T09:18:02.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1e827fc700 1 -- 172.21.15.61:0/3164768775 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e900cd170 con 0x55ac2c812de0 2023-10-18T09:18:02.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1e827fc700 1 -- 172.21.15.61:0/3164768775 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900998f0 con 0x55ac2c812de0 2023-10-18T09:18:02.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/3164768775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 msgr2=0x55ac2c81b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1e90099e50 tx=0x7f1e90098750 comp rx=0 tx=0).stop 2023-10-18T09:18:02.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/3164768775 shutdown_connections 2023-10-18T09:18:02.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c819ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.691+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812480 0x55ac2c8077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/3164768775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812de0 0x55ac2c81b190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/3164768775 >> 172.21.15.61:0/3164768775 conn(0x55ac2c7866d0 msgr2=0x55ac2c813b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/3164768775 shutdown_connections 2023-10-18T09:18:02.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/3164768775 wait complete. 2023-10-18T09:18:02.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 -- start start 2023-10-18T09:18:02.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.692+0000 7f1eb54a1b80 1 Processor -- start 2023-10-18T09:18:02.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c818670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c818bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c807470 con 0x55ac2c812480 2023-10-18T09:18:02.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c819120 con 0x55ac2c8137a0 2023-10-18T09:18:02.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac2c8192e0 con 0x55ac2c812de0 2023-10-18T09:18:02.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1eb54a1b80 10 client.0 _ll_register_callbacks cb 0x55ac2c64ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42534/0 (socket says 172.21.15.61:42534) 2023-10-18T09:18:02.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1561363899 learned_addr learned my addr 172.21.15.61:0/1561363899 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c818670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.693+0000 7f1e9bfff700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c818bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 msgr2=0x55ac2c818670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c818670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1561363899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 msgr2=0x55ac2c818bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c818bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 -- 172.21.15.61:0/1561363899 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac2c58fd90 con 0x55ac2c812480 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e9b7fe700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000c00 tx=0x7f1e8c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c345e43a3600ff8e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e7affd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c02fc60 con 0x55ac2c812480 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac2c75de00 con 0x55ac2c812480 2023-10-18T09:18:02.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.694+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e8c02fdc0 con 0x55ac2c812480 2023-10-18T09:18:02.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e8c02f040 con 0x55ac2c812480 2023-10-18T09:18:02.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 10 client.15297 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ac2c5b6200 con 0x55ac2c812480 2023-10-18T09:18:02.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac2c75de00 con 0x55ac2c812480 2023-10-18T09:18:02.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1e8c003860 con 0x55ac2c812480 2023-10-18T09:18:02.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 10 client.15297 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-18T09:18:02.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 20 client.15297 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c884180 0x55ac2c880600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 4 client.15297 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:02.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac2c75db20 con 0x55ac2c884180 2023-10-18T09:18:02.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.695+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c884180 0x55ac2c880600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1e7affd700 10 client.15297 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:02.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1e9affd700 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c884180 0x55ac2c880600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f802c54b9bc67adb server_cookie=eee6b54d1d7a9672 in_seq=0 out_seq=0 2023-10-18T09:18:02.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1e7affd700 1 -- 172.21.15.61:0/1561363899 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ac2c75db20 con 0x55ac2c884180 2023-10-18T09:18:02.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1e7affd700 10 client.15297 handle_command_reply: tid=0 2023-10-18T09:18:02.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 client.15297 shutdown 2023-10-18T09:18:02.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 msgr2=0x55ac2c806c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c000c00 tx=0x7f1e8c009580 comp rx=0 tx=0).stop 2023-10-18T09:18:02.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 shutdown_connections 2023-10-18T09:18:02.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac2c8137a0 0x55ac2c818bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ac2c812480 0x55ac2c806c50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55ac2c884180 0x55ac2c880600 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 --2- 172.21.15.61:0/1561363899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ac2c812de0 0x55ac2c818670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1e7affd700 0 client.15297 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:02.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.696+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 >> 172.21.15.61:0/1561363899 conn(0x55ac2c7866d0 msgr2=0x55ac2c81b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 shutdown_connections 2023-10-18T09:18:02.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1eb54a1b80 1 -- 172.21.15.61:0/1561363899 wait complete. 2023-10-18T09:18:02.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1eb54a1b80 20 client.15297 trim_cache size 0 max 16384 2023-10-18T09:18:02.713 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 3, 2023-10-18T09:18:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24646, 2023-10-18T09:18:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 65.180969907999994, 2023-10-18T09:18:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 90, 2023-10-18T09:18:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 117.84007739800001 2023-10-18T09:18:02.716 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:02.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 msgr2=0x7f1e840a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 0x7f1e840a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400fb80 tx=0x7f1e94004b40 comp rx=0 tx=0).stop 2023-10-18T09:18:02.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 msgr2=0x7f1e9c196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c196ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000e9f0 tx=0x7f1e9000ed00 comp rx=0 tx=0).stop 2023-10-18T09:18:02.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 shutdown_connections 2023-10-18T09:18:02.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1e840a1510 0x7f1e840a39d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e9c102150 0x7f1e9c19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1e9c1005c0 0x7f1e9c196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 --2- 172.21.15.61:0/1863529584 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e9c1017b0 0x7f1e9c196ad0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.697+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 >> 172.21.15.61:0/1863529584 conn(0x7f1e9c0fbaf0 msgr2=0x7f1e9c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.698+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 shutdown_connections 2023-10-18T09:18:02.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.698+0000 7f1ea099d700 1 -- 172.21.15.61:0/1863529584 wait complete. 2023-10-18T09:18:02.720 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-18T09:18:02.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 -- 172.21.15.61:0/1958823829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f24081022b0 msgr2=0x7f2408108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 --2- 172.21.15.61:0/1958823829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f24081022b0 0x7f2408108b60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f240000b0d0 tx=0x7f240000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 -- 172.21.15.61:0/1958823829 shutdown_connections 2023-10-18T09:18:02.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 --2- 172.21.15.61:0/1958823829 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f24081022b0 0x7f2408108b60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 --2- 172.21.15.61:0/1958823829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408100720 0x7f2408100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 --2- 172.21.15.61:0/1958823829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2408101910 0x7f2408101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 -- 172.21.15.61:0/1958823829 >> 172.21.15.61:0/1958823829 conn(0x7f24080fbc90 msgr2=0x7f24080fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 -- 172.21.15.61:0/1958823829 shutdown_connections 2023-10-18T09:18:02.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.924+0000 7f240d9ff700 1 -- 172.21.15.61:0/1958823829 wait complete. 2023-10-18T09:18:02.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.925+0000 7f240d9ff700 1 Processor -- start 2023-10-18T09:18:02.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.925+0000 7f240d9ff700 1 -- start start 2023-10-18T09:18:02.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408101910 0x7f2408072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 0x7f2408073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24080738c0 con 0x7f2408100720 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408073a30 con 0x7f24081022b0 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f240d9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24081a0820 con 0x7f2408101910 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42554/0 (socket says 172.21.15.61:42554) 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 -- 172.21.15.61:0/2347237764 learned_addr learned my addr 172.21.15.61:0/2347237764 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f24067fc700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408101910 0x7f2408072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f24077fe700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 0x7f2408073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 -- 172.21.15.61:0/2347237764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408101910 msgr2=0x7f2408072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408101910 0x7f2408072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 -- 172.21.15.61:0/2347237764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 msgr2=0x7f2408073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 0x7f2408073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 -- 172.21.15.61:0/2347237764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009520 con 0x7f2408100720 2023-10-18T09:18:02.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f24077fe700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 0x7f2408073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.926+0000 7f2406ffd700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f23f800ea50 tx=0x7f23f800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.927+0000 7f240c9fd700 1 -- 172.21.15.61:0/2347237764 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800cc30 con 0x7f2408100720 2023-10-18T09:18:02.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.927+0000 7f240c9fd700 1 -- 172.21.15.61:0/2347237764 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f800cd90 con 0x7f2408100720 2023-10-18T09:18:02.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.927+0000 7f240c9fd700 1 -- 172.21.15.61:0/2347237764 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8018710 con 0x7f2408100720 2023-10-18T09:18:02.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.927+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2400009d00 con 0x7f2408100720 2023-10-18T09:18:02.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.927+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24081a0de0 con 0x7f2408100720 2023-10-18T09:18:02.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.931+0000 7f240c9fd700 1 -- 172.21.15.61:0/2347237764 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f23f80188d0 con 0x7f2408100720 2023-10-18T09:18:02.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.932+0000 7f24213d5b80 1 Processor -- start 2023-10-18T09:18:02.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 -- start start 2023-10-18T09:18:02.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60671f690 con 0x55e6067ac580 2023-10-18T09:18:02.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e606705c10 con 0x55e6067ad280 2023-10-18T09:18:02.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24213d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e606703530 con 0x55e6067adbe0 2023-10-18T09:18:02.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f240c9fd700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 0x7f23f00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f240c9fd700 1 -- 172.21.15.61:0/2347237764 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f23f8014070 con 0x7f2408100720 2023-10-18T09:18:02.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:50136/0 (socket says 172.21.15.61:50136) 2023-10-18T09:18:02.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f2406ffd700 1 -- 172.21.15.61:0/1533040529 learned_addr learned my addr 172.21.15.61:0/1533040529 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24067fc700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24077fe700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 0x7f23f00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.933+0000 7f24077fe700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 -- 172.21.15.61:0/1533040529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 msgr2=0x55e6067ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 -- 172.21.15.61:0/1533040529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 msgr2=0x55e6067aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 -- 172.21.15.61:0/1533040529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6066bc3a0 con 0x55e6067adbe0 2023-10-18T09:18:02.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f24067fc700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f24077fe700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:02.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f2406ffd700 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23f80cd830 tx=0x7f23f801d850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987547fe95c072fe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f24077fe700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 0x7f23f00a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2400009cd0 tx=0x7f2400006000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f23edffb700 1 -- 172.21.15.61:0/1533040529 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80850b0 con 0x55e6067adbe0 2023-10-18T09:18:02.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f23edffb700 1 -- 172.21.15.61:0/1533040529 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f809ace0 con 0x55e6067adbe0 2023-10-18T09:18:02.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.934+0000 7f23edffb700 1 -- 172.21.15.61:0/1533040529 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f801e500 con 0x55e6067adbe0 2023-10-18T09:18:02.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- 172.21.15.61:0/1533040529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 msgr2=0x55e6067ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23f80cd830 tx=0x7f23f801d850 comp rx=0 tx=0).stop 2023-10-18T09:18:02.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- 172.21.15.61:0/1533040529 shutdown_connections 2023-10-18T09:18:02.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 --2- 172.21.15.61:0/1533040529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067ac040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- 172.21.15.61:0/1533040529 >> 172.21.15.61:0/1533040529 conn(0x55e6067206d0 msgr2=0x55e6067a6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- 172.21.15.61:0/1533040529 shutdown_connections 2023-10-18T09:18:02.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- 172.21.15.61:0/1533040529 wait complete. 2023-10-18T09:18:02.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.935+0000 7f24213d5b80 1 -- start start 2023-10-18T09:18:02.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 Processor -- start 2023-10-18T09:18:02.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067ad280 0x55e6067b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6066fee20 con 0x55e6067adbe0 2023-10-18T09:18:02.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60669b6c0 con 0x55e6067ac580 2023-10-18T09:18:02.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60669b360 con 0x55e6067ad280 2023-10-18T09:18:02.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24213d5b80 10 client.0 _ll_register_callbacks cb 0x55e6065e5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24077fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24077fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42568/0 (socket says 172.21.15.61:42568) 2023-10-18T09:18:02.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.936+0000 7f24077fe700 1 -- 172.21.15.61:0/3406495800 learned_addr learned my addr 172.21.15.61:0/3406495800 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f2406ffd700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24067fc700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067ad280 0x55e6067b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 -- 172.21.15.61:0/3406495800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067ad280 msgr2=0x55e6067b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067ad280 0x55e6067b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 -- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 msgr2=0x55e6067b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 -- 172.21.15.61:0/3406495800 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24081a0de0 con 0x55e6067adbe0 2023-10-18T09:18:02.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f2406ffd700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:02.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f24077fe700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2400015000 tx=0x7f240000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659c7d5473602866 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f23e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.937+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000e040 con 0x55e6067adbe0 2023-10-18T09:18:02.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e606703ea0 con 0x55e6067adbe0 2023-10-18T09:18:02.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2400004e40 con 0x55e6067adbe0 2023-10-18T09:18:02.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2400005070 con 0x55e6067adbe0 2023-10-18T09:18:02.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 10 client.15309 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e606529d90 con 0x55e6067adbe0 2023-10-18T09:18:02.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e606703ea0 con 0x55e6067adbe0 2023-10-18T09:18:02.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f240001b730 con 0x55e6067adbe0 2023-10-18T09:18:02.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 20 client.15309 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e6067c1750 0x55e6067c0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 4 client.15309 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.938+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e60677b860 con 0x55e6067c1750 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.939+0000 7f2406ffd700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e6067c1750 0x55e6067c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.939+0000 7f2406ffd700 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e6067c1750 0x55e6067c0220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e12f1e10b404db63 server_cookie=a97a03bced3186aa in_seq=0 out_seq=0 2023-10-18T09:18:02.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.939+0000 7f23e6ffd700 10 client.15309 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:02.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f23e6ffd700 1 -- 172.21.15.61:0/3406495800 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e60677b860 con 0x55e6067c1750 2023-10-18T09:18:02.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f23e6ffd700 10 client.15309 handle_command_reply: tid=0 2023-10-18T09:18:02.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 client.15309 shutdown 2023-10-18T09:18:02.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 msgr2=0x55e6067bd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2400015000 tx=0x7f240000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 shutdown_connections 2023-10-18T09:18:02.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e6067c1750 0x55e6067c0220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067adbe0 0x55e6067bd030 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 --2- 172.21.15.61:0/3406495800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067ad280 0x55e6067b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 >> 172.21.15.61:0/3406495800 conn(0x55e6067206d0 msgr2=0x55e6067ab030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f23e6ffd700 0 client.15309 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:02.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 shutdown_connections 2023-10-18T09:18:02.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.940+0000 7f24213d5b80 1 -- 172.21.15.61:0/3406495800 wait complete. 2023-10-18T09:18:02.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.941+0000 7f24213d5b80 20 client.15309 trim_cache size 0 max 16384 2023-10-18T09:18:02.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.959+0000 7f24213d5b80 1 Processor -- start 2023-10-18T09:18:02.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.959+0000 7f24213d5b80 1 -- start start 2023-10-18T09:18:02.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60657f8d0 con 0x55e6067ac580 2023-10-18T09:18:02.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e60677b860 con 0x55e6067ad280 2023-10-18T09:18:02.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24213d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6067000b0 con 0x55e6067adbe0 2023-10-18T09:18:02.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42572/0 (socket says 172.21.15.61:42572) 2023-10-18T09:18:02.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 -- 172.21.15.61:0/388582099 learned_addr learned my addr 172.21.15.61:0/388582099 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24067fc700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24077fe700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 -- 172.21.15.61:0/388582099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 msgr2=0x55e6067bcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067bcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 -- 172.21.15.61:0/388582099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 msgr2=0x55e6067a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f2406ffd700 1 -- 172.21.15.61:0/388582099 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e606512c10 con 0x55e6067ac580 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.960+0000 7f24067fc700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:02.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.961+0000 7f2406ffd700 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f23f809a2e0 tx=0x7f23f80ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a409ad9a5ecbac9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.961+0000 7f23ee7fc700 1 -- 172.21.15.61:0/388582099 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80850b0 con 0x55e6067ac580 2023-10-18T09:18:02.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.961+0000 7f23ee7fc700 1 -- 172.21.15.61:0/388582099 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f80b5070 con 0x55e6067ac580 2023-10-18T09:18:02.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.961+0000 7f23ee7fc700 1 -- 172.21.15.61:0/388582099 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80b8590 con 0x55e6067ac580 2023-10-18T09:18:02.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- 172.21.15.61:0/388582099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 msgr2=0x55e6067b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f23f809a2e0 tx=0x7f23f80ba040 comp rx=0 tx=0).stop 2023-10-18T09:18:02.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- 172.21.15.61:0/388582099 shutdown_connections 2023-10-18T09:18:02.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067bcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ac580 0x55e6067b4fc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 --2- 172.21.15.61:0/388582099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ad280 0x55e6067a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- 172.21.15.61:0/388582099 >> 172.21.15.61:0/388582099 conn(0x55e6067206d0 msgr2=0x55e6067ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- 172.21.15.61:0/388582099 shutdown_connections 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- 172.21.15.61:0/388582099 wait complete. 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.962+0000 7f24213d5b80 1 -- start start 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 Processor -- start 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067bcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6067ba530 con 0x55e6067ad280 2023-10-18T09:18:02.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6067ba6a0 con 0x55e6067ac580 2023-10-18T09:18:02.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e606815140 con 0x55e6067adbe0 2023-10-18T09:18:02.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24213d5b80 10 client.0 _ll_register_callbacks cb 0x55e6065e5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:02.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.963+0000 7f24067fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42576/0 (socket says 172.21.15.61:42576) 2023-10-18T09:18:02.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 -- 172.21.15.61:0/235104286 learned_addr learned my addr 172.21.15.61:0/235104286 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:02.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f2406ffd700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067bcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24077fe700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 -- 172.21.15.61:0/235104286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 msgr2=0x55e6067b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 -- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 msgr2=0x55e6067bcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:02.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067bcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 -- 172.21.15.61:0/235104286 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc0098e0 con 0x55e6067ad280 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24067fc700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f23fc012a10 tx=0x7f23fc012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5090ac28d1ec1817 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f23e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc009b80 con 0x55e6067ad280 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23f809ace0 con 0x55e6067ad280 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.964+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f23fc004460 con 0x55e6067ad280 2023-10-18T09:18:02.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23fc011830 con 0x55e6067ad280 2023-10-18T09:18:02.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 10 client.15315 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:02.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e6066bc3a0 con 0x55e6067ad280 2023-10-18T09:18:02.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23f809ace0 con 0x55e6067ad280 2023-10-18T09:18:02.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f23fc00f450 con 0x55e6067ad280 2023-10-18T09:18:02.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 10 client.15315 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:02.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-18T09:18:02.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 20 client.15315 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:02.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e60681a4d0 0x55e606816970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:02.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 4 client.15315 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:02.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.965+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e6066f7e00 con 0x55e60681a4d0 2023-10-18T09:18:02.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f2406ffd700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e60681a4d0 0x55e606816970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:02.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f23e6ffd700 10 client.15315 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:02.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f2406ffd700 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e60681a4d0 0x55e606816970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=646dc28cb249d15f server_cookie=8dc64ce5f4ae4974 in_seq=0 out_seq=0 2023-10-18T09:18:02.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f23e6ffd700 1 -- 172.21.15.61:0/235104286 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e6066f7e00 con 0x55e60681a4d0 2023-10-18T09:18:02.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f23e6ffd700 10 client.15315 handle_command_reply: tid=0 2023-10-18T09:18:02.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f24213d5b80 1 client.15315 shutdown 2023-10-18T09:18:02.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 msgr2=0x55e6067b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.966+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f23fc012a10 tx=0x7f23fc012d20 comp rx=0 tx=0).stop 2023-10-18T09:18:02.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 shutdown_connections 2023-10-18T09:18:02.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e60681a4d0 0x55e606816970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6067adbe0 0x55e6067b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e6067ac580 0x55e6067bcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f23e6ffd700 0 client.15315 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:02.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 --2- 172.21.15.61:0/235104286 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e6067ad280 0x55e6067b9ab0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 >> 172.21.15.61:0/235104286 conn(0x55e6067206d0 msgr2=0x55e6067a6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 shutdown_connections 2023-10-18T09:18:02.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 1 -- 172.21.15.61:0/235104286 wait complete. 2023-10-18T09:18:02.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f24213d5b80 20 client.15315 trim_cache size 0 max 16384 2023-10-18T09:18:02.983 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:02.984 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:02.984 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 2, 2023-10-18T09:18:02.984 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24653, 2023-10-18T09:18:02.984 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:02.984 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:02.985 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:02.985 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 66.458074847999995, 2023-10-18T09:18:02.985 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:02.985 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 90, 2023-10-18T09:18:02.985 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:02.986 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 113.665078892 2023-10-18T09:18:02.986 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:02.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 msgr2=0x7f23f00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 0x7f23f00a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2400009cd0 tx=0x7f2400006000 comp rx=0 tx=0).stop 2023-10-18T09:18:02.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 msgr2=0x7f2408074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:02.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.967+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f23f800ea50 tx=0x7f23f800ee10 comp rx=0 tx=0).stop 2023-10-18T09:18:02.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 shutdown_connections 2023-10-18T09:18:02.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f23f00a1510 0x7f23f00a39d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24081022b0 0x7f2408073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2408100720 0x7f2408074550 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 --2- 172.21.15.61:0/2347237764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2408101910 0x7f2408072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:02.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 >> 172.21.15.61:0/2347237764 conn(0x7f24080fbc90 msgr2=0x7f2408104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:02.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 shutdown_connections 2023-10-18T09:18:02.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:02.968+0000 7f240d9ff700 1 -- 172.21.15.61:0/2347237764 wait complete. 2023-10-18T09:18:02.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-18T09:18:03.053 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:02 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2306647594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:03.053 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:02 smithi061 ceph-mon[98573]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:03.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.194+0000 7f0611bff700 1 -- 172.21.15.61:0/3827429853 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 msgr2=0x7f060c072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.194+0000 7f0611bff700 1 --2- 172.21.15.61:0/3827429853 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c072c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009990 tx=0x7f05fc009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 -- 172.21.15.61:0/3827429853 shutdown_connections 2023-10-18T09:18:03.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 --2- 172.21.15.61:0/3827429853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c1085f0 0x7f060c10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 --2- 172.21.15.61:0/3827429853 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c072c20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 --2- 172.21.15.61:0/3827429853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c0731f0 0x7f060c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 -- 172.21.15.61:0/3827429853 >> 172.21.15.61:0/3827429853 conn(0x7f060c0fbc90 msgr2=0x7f060c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 -- 172.21.15.61:0/3827429853 shutdown_connections 2023-10-18T09:18:03.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.195+0000 7f0611bff700 1 -- 172.21.15.61:0/3827429853 wait complete. 2023-10-18T09:18:03.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 Processor -- start 2023-10-18T09:18:03.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 -- start start 2023-10-18T09:18:03.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 0x7f060c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c1085f0 0x7f060c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1a0330 con 0x7f060c0747d0 2023-10-18T09:18:03.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1a0470 con 0x7f060c0731f0 2023-10-18T09:18:03.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f0611bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1a0600 con 0x7f060c1085f0 2023-10-18T09:18:03.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f060affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f060bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c1085f0 0x7f060c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f060affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42590/0 (socket says 172.21.15.61:42590) 2023-10-18T09:18:03.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f060affd700 1 -- 172.21.15.61:0/2073860206 learned_addr learned my addr 172.21.15.61:0/2073860206 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.196+0000 7f060b7fe700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 0x7f060c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 -- 172.21.15.61:0/2073860206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c1085f0 msgr2=0x7f060c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c1085f0 0x7f060c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 -- 172.21.15.61:0/2073860206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 msgr2=0x7f060c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 0x7f060c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 -- 172.21.15.61:0/2073860206 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0095f0 con 0x7f060c0747d0 2023-10-18T09:18:03.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060b7fe700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 0x7f060c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f060affd700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f060000e9a0 tx=0x7f060000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f0608ff9700 1 -- 172.21.15.61:0/2073860206 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600009790 con 0x7f060c0747d0 2023-10-18T09:18:03.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c1a08e0 con 0x7f060c0747d0 2023-10-18T09:18:03.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f0608ff9700 1 -- 172.21.15.61:0/2073860206 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0600010430 con 0x7f060c0747d0 2023-10-18T09:18:03.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f0608ff9700 1 -- 172.21.15.61:0/2073860206 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600018680 con 0x7f060c0747d0 2023-10-18T09:18:03.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.197+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c1a0df0 con 0x7f060c0747d0 2023-10-18T09:18:03.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.201+0000 7f0608ff9700 1 -- 172.21.15.61:0/2073860206 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f06000105a0 con 0x7f060c0747d0 2023-10-18T09:18:03.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.202+0000 7f062544fb80 1 Processor -- start 2023-10-18T09:18:03.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 -- start start 2023-10-18T09:18:03.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b51146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5086690 con 0x55a8b5113580 2023-10-18T09:18:03.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b506cc10 con 0x55a8b5114be0 2023-10-18T09:18:03.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f062544fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b506a530 con 0x55a8b5114280 2023-10-18T09:18:03.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f0608ff9700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 0x7f05f40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f0608ff9700 1 -- 172.21.15.61:0/2073860206 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0600014070 con 0x7f060c0747d0 2023-10-18T09:18:03.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54292/0 (socket says 172.21.15.61:54292) 2023-10-18T09:18:03.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f060b7fe700 1 -- 172.21.15.61:0/3156194536 learned_addr learned my addr 172.21.15.61:0/3156194536 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f060bfff700 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b51146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.203+0000 7f060affd700 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060bfff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 0x7f05f40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 -- 172.21.15.61:0/3156194536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 msgr2=0x55a8b51146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b51146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 -- 172.21.15.61:0/3156194536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 msgr2=0x55a8b5113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 -- 172.21.15.61:0/3156194536 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8b50233a0 con 0x55a8b5114be0 2023-10-18T09:18:03.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009990 tx=0x7f05fc005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b72c01ee7b283c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f060bfff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 0x7f05f40a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0604009ec0 tx=0x7f060400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f05f1ffb700 1 -- 172.21.15.61:0/3156194536 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01d070 con 0x55a8b5114be0 2023-10-18T09:18:03.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f05f1ffb700 1 -- 172.21.15.61:0/3156194536 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc01e040 con 0x55a8b5114be0 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.204+0000 7f05f1ffb700 1 -- 172.21.15.61:0/3156194536 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005610 con 0x55a8b5114be0 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 -- 172.21.15.61:0/3156194536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 msgr2=0x55a8b5113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009990 tx=0x7f05fc005ec0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 -- 172.21.15.61:0/3156194536 shutdown_connections 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b51146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 --2- 172.21.15.61:0/3156194536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5113040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 -- 172.21.15.61:0/3156194536 >> 172.21.15.61:0/3156194536 conn(0x55a8b50876d0 msgr2=0x55a8b510db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 -- 172.21.15.61:0/3156194536 shutdown_connections 2023-10-18T09:18:03.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.205+0000 7f062544fb80 1 -- 172.21.15.61:0/3156194536 wait complete. 2023-10-18T09:18:03.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 -- start start 2023-10-18T09:18:03.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 Processor -- start 2023-10-18T09:18:03.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b511cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5123f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5065e20 con 0x55a8b5113580 2023-10-18T09:18:03.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b50026c0 con 0x55a8b5114be0 2023-10-18T09:18:03.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.206+0000 7f062544fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5002360 con 0x55a8b5114280 2023-10-18T09:18:03.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f062544fb80 10 client.0 _ll_register_callbacks cb 0x55a8b4f4ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:03.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5123f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42600/0 (socket says 172.21.15.61:42600) 2023-10-18T09:18:03.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060b7fe700 1 -- 172.21.15.61:0/2040976124 learned_addr learned my addr 172.21.15.61:0/2040976124 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060affd700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b511cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 -- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 msgr2=0x55a8b511cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b511cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 -- 172.21.15.61:0/2040976124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 msgr2=0x55a8b511cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 -- 172.21.15.61:0/2040976124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c10b680 con 0x55a8b5114be0 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060b7fe700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f060bfff700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5123f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0604019af0 tx=0x7f0604004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2e56dddbda1b53 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.207+0000 7f05eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:03.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060400e040 con 0x55a8b5114be0 2023-10-18T09:18:03.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8b506aea0 con 0x55a8b5114be0 2023-10-18T09:18:03.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0604019280 con 0x55a8b5114be0 2023-10-18T09:18:03.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0604003a00 con 0x55a8b5114be0 2023-10-18T09:18:03.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f062544fb80 10 client.25310 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:03.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a8b4e90d90 con 0x55a8b5114be0 2023-10-18T09:18:03.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8b506aea0 con 0x55a8b5114be0 2023-10-18T09:18:03.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.208+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f06040040e0 con 0x55a8b5114be0 2023-10-18T09:18:03.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 10 client.25310 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:03.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 10 client.25310 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-18T09:18:03.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 20 client.25310 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:03.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5128790 0x55a8b51271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 4 client.25310 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:03.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8b50e2860 con 0x55a8b5128790 2023-10-18T09:18:03.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f060b7fe700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5128790 0x55a8b51271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.210+0000 7f05eaffd700 10 client.25310 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:03.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.209+0000 7f060b7fe700 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5128790 0x55a8b51271e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd1dc387a5ac909d server_cookie=bb628e9319757e8a in_seq=0 out_seq=0 2023-10-18T09:18:03.227 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:02 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2306647594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:03.227 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:02 smithi110 ceph-mon[134818]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:03.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.210+0000 7f05eaffd700 1 -- 172.21.15.61:0/2040976124 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a8b50e2860 con 0x55a8b5128790 2023-10-18T09:18:03.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.210+0000 7f05eaffd700 10 client.25310 handle_command_reply: tid=0 2023-10-18T09:18:03.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 client.25310 shutdown 2023-10-18T09:18:03.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 msgr2=0x55a8b5123f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5123f90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0604019af0 tx=0x7f0604004b40 comp rx=0 tx=0).stop 2023-10-18T09:18:03.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 shutdown_connections 2023-10-18T09:18:03.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5128790 0x55a8b51271e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b5123f90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f05eaffd700 0 client.25310 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:03.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b511cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 --2- 172.21.15.61:0/2040976124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b511cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 >> 172.21.15.61:0/2040976124 conn(0x55a8b50876d0 msgr2=0x55a8b510dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 shutdown_connections 2023-10-18T09:18:03.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 1 -- 172.21.15.61:0/2040976124 wait complete. 2023-10-18T09:18:03.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.211+0000 7f062544fb80 20 client.25310 trim_cache size 0 max 16384 2023-10-18T09:18:03.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.229+0000 7f062544fb80 1 Processor -- start 2023-10-18T09:18:03.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.229+0000 7f062544fb80 1 -- start start 2023-10-18T09:18:03.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b51271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b4ee68d0 con 0x55a8b5113580 2023-10-18T09:18:03.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5024590 con 0x55a8b5114be0 2023-10-18T09:18:03.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f062544fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b501d720 con 0x55a8b5114280 2023-10-18T09:18:03.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42616/0 (socket says 172.21.15.61:42616) 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 -- 172.21.15.61:0/1889795387 learned_addr learned my addr 172.21.15.61:0/1889795387 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060bfff700 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b51271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060affd700 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 -- 172.21.15.61:0/1889795387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 msgr2=0x55a8b5123f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 -- 172.21.15.61:0/1889795387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 msgr2=0x55a8b51271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b51271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.230+0000 7f060b7fe700 1 -- 172.21.15.61:0/1889795387 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8b4e79c10 con 0x55a8b5113580 2023-10-18T09:18:03.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f060b7fe700 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b680 tx=0x7f05fc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97f323e557d9f18 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f05f27fc700 1 -- 172.21.15.61:0/1889795387 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01d040 con 0x55a8b5113580 2023-10-18T09:18:03.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f05f27fc700 1 -- 172.21.15.61:0/1889795387 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc005920 con 0x55a8b5113580 2023-10-18T09:18:03.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f05f27fc700 1 -- 172.21.15.61:0/1889795387 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc00f460 con 0x55a8b5113580 2023-10-18T09:18:03.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 -- 172.21.15.61:0/1889795387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 msgr2=0x55a8b5113040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b680 tx=0x7f05fc005fb0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 -- 172.21.15.61:0/1889795387 shutdown_connections 2023-10-18T09:18:03.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114be0 0x55a8b51271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5113580 0x55a8b5113040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 --2- 172.21.15.61:0/1889795387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.231+0000 7f062544fb80 1 -- 172.21.15.61:0/1889795387 >> 172.21.15.61:0/1889795387 conn(0x55a8b50876d0 msgr2=0x55a8b511aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.232+0000 7f062544fb80 1 -- 172.21.15.61:0/1889795387 shutdown_connections 2023-10-18T09:18:03.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.232+0000 7f062544fb80 1 -- 172.21.15.61:0/1889795387 wait complete. 2023-10-18T09:18:03.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.232+0000 7f062544fb80 1 -- start start 2023-10-18T09:18:03.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.232+0000 7f062544fb80 1 Processor -- start 2023-10-18T09:18:03.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.232+0000 7f062544fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 0x55a8b5118510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5118c20 con 0x55a8b5114be0 2023-10-18T09:18:03.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5118d90 con 0x55a8b5114280 2023-10-18T09:18:03.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8b5118f50 con 0x55a8b5113580 2023-10-18T09:18:03.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f062544fb80 10 client.0 _ll_register_callbacks cb 0x55a8b4f4ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:03.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 0x55a8b5118510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54304/0 (socket says 172.21.15.61:54304) 2023-10-18T09:18:03.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:50184/0 (socket says 172.21.15.61:50184) 2023-10-18T09:18:03.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060affd700 1 -- 172.21.15.61:0/3791622614 learned_addr learned my addr 172.21.15.61:0/3791622614 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 -- 172.21.15.61:0/3791622614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 msgr2=0x55a8b5123f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 -- 172.21.15.61:0/3791622614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 msgr2=0x55a8b5118510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 0x55a8b5118510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060b7fe700 1 -- 172.21.15.61:0/3791622614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8b50233a0 con 0x55a8b5113580 2023-10-18T09:18:03.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060affd700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.233+0000 7f060bfff700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 0x55a8b5118510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f060b7fe700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc005180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69dcefb1a886d2a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:18:03.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004bf0 con 0x55a8b5113580 2023-10-18T09:18:03.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05fc01e040 con 0x55a8b5113580 2023-10-18T09:18:03.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05fc00f460 con 0x55a8b5113580 2023-10-18T09:18:03.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05fc004d50 con 0x55a8b5113580 2023-10-18T09:18:03.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f062544fb80 10 client.25342 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:03.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a8b4eb7200 con 0x55a8b5113580 2023-10-18T09:18:03.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc01e040 con 0x55a8b5113580 2023-10-18T09:18:03.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.234+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f05fc022660 con 0x55a8b5113580 2023-10-18T09:18:03.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 10 client.25342 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:03.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 10 client.25342 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-18T09:18:03.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 20 client.25342 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:03.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5180a30 0x55a8b517ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 4 client.25342 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:03.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8b505ee00 con 0x55a8b5180a30 2023-10-18T09:18:03.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f060affd700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5180a30 0x55a8b517ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f060affd700 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5180a30 0x55a8b517ceb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4ff89d3f5691050 server_cookie=48cf598da6c49caf in_seq=0 out_seq=0 2023-10-18T09:18:03.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.235+0000 7f05eaffd700 10 client.25342 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:03.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f05eaffd700 1 -- 172.21.15.61:0/3791622614 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a8b505ee00 con 0x55a8b5180a30 2023-10-18T09:18:03.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f05eaffd700 10 client.25342 handle_command_reply: tid=0 2023-10-18T09:18:03.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 client.25342 shutdown 2023-10-18T09:18:03.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 msgr2=0x55a8b5119de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc005180 comp rx=0 tx=0).stop 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 shutdown_connections 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55a8b5180a30 0x55a8b517ceb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a8b5114be0 0x55a8b5118510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a8b5113580 0x55a8b5119de0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 --2- 172.21.15.61:0/3791622614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a8b5114280 0x55a8b5123f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f05eaffd700 0 client.25342 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:03.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 >> 172.21.15.61:0/3791622614 conn(0x55a8b50876d0 msgr2=0x55a8b51271e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 shutdown_connections 2023-10-18T09:18:03.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 1 -- 172.21.15.61:0/3791622614 wait complete. 2023-10-18T09:18:03.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.236+0000 7f062544fb80 20 client.25342 trim_cache size 0 max 16384 2023-10-18T09:18:03.254 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:03.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:03.254 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 1, 2023-10-18T09:18:03.254 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24676, 2023-10-18T09:18:03.255 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:03.255 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:03.255 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:03.255 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 67.322674036999999, 2023-10-18T09:18:03.255 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:03.256 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 90, 2023-10-18T09:18:03.256 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:03.256 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 111.731002784 2023-10-18T09:18:03.256 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:03.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 msgr2=0x7f05f40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 0x7f05f40a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0604009ec0 tx=0x7f060400b040 comp rx=0 tx=0).stop 2023-10-18T09:18:03.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 msgr2=0x7f060c198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f060000e9a0 tx=0x7f060000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 shutdown_connections 2023-10-18T09:18:03.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f05f40a1510 0x7f05f40a39d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f060c1085f0 0x7f060c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f060c0731f0 0x7f060c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 --2- 172.21.15.61:0/2073860206 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f060c0747d0 0x7f060c198d40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 >> 172.21.15.61:0/2073860206 conn(0x7f060c0fbc90 msgr2=0x7f060c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 shutdown_connections 2023-10-18T09:18:03.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.237+0000 7f0611bff700 1 -- 172.21.15.61:0/2073860206 wait complete. 2023-10-18T09:18:03.260 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T09:18:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:02 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2306647594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:02 smithi170 ceph-mon[135150]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:03.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 -- 172.21.15.61:0/2769101146 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 msgr2=0x7f409410aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 --2- 172.21.15.61:0/2769101146 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409410aa60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b0a0 tx=0x7f408c00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 -- 172.21.15.61:0/2769101146 shutdown_connections 2023-10-18T09:18:03.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 --2- 172.21.15.61:0/2769101146 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409410aa60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 --2- 172.21.15.61:0/2769101146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940747d0 0x7f4094072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 --2- 172.21.15.61:0/2769101146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940731f0 0x7f4094073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 -- 172.21.15.61:0/2769101146 >> 172.21.15.61:0/2769101146 conn(0x7f40940fbc90 msgr2=0x7f40940fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 -- 172.21.15.61:0/2769101146 shutdown_connections 2023-10-18T09:18:03.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.464+0000 7f409bbff700 1 -- 172.21.15.61:0/2769101146 wait complete. 2023-10-18T09:18:03.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.465+0000 7f409bbff700 1 Processor -- start 2023-10-18T09:18:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.465+0000 7f409bbff700 1 -- start start 2023-10-18T09:18:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.465+0000 7f409bbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940731f0 0x7f4094198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.465+0000 7f409bbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409bbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409bbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a04a0 con 0x7f40941085f0 2023-10-18T09:18:03.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409bbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a05e0 con 0x7f40940747d0 2023-10-18T09:18:03.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409bbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40941a0770 con 0x7f40940731f0 2023-10-18T09:18:03.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54320/0 (socket says 172.21.15.61:54320) 2023-10-18T09:18:03.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 -- 172.21.15.61:0/2223700494 learned_addr learned my addr 172.21.15.61:0/2223700494 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409a19c700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409999b700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940731f0 0x7f4094198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 -- 172.21.15.61:0/2223700494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940731f0 msgr2=0x7f4094198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940731f0 0x7f4094198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 -- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 msgr2=0x7f409419fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409419fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409919a700 1 -- 172.21.15.61:0/2223700494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084009520 con 0x7f40940747d0 2023-10-18T09:18:03.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.466+0000 7f409a19c700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409419fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f409919a700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f408400eb30 tx=0x7f408400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f408affd700 1 -- 172.21.15.61:0/2223700494 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084009880 con 0x7f40940747d0 2023-10-18T09:18:03.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c009d00 con 0x7f40940747d0 2023-10-18T09:18:03.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f408affd700 1 -- 172.21.15.61:0/2223700494 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40840044e0 con 0x7f40940747d0 2023-10-18T09:18:03.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f408affd700 1 -- 172.21.15.61:0/2223700494 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084003d80 con 0x7f40940747d0 2023-10-18T09:18:03.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.467+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40941a0e20 con 0x7f40940747d0 2023-10-18T09:18:03.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.470+0000 7f408affd700 1 -- 172.21.15.61:0/2223700494 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f4084003f20 con 0x7f40940747d0 2023-10-18T09:18:03.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 Processor -- start 2023-10-18T09:18:03.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 -- start start 2023-10-18T09:18:03.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e65c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65b580 0x55d83e65ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5ce690 con 0x55d83e65cbe0 2023-10-18T09:18:03.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5b4c10 con 0x55d83e65c280 2023-10-18T09:18:03.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.472+0000 7f40af537b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5b2530 con 0x55d83e65b580 2023-10-18T09:18:03.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f408affd700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 0x7f40800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42660/0 (socket says 172.21.15.61:42660) 2023-10-18T09:18:03.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 -- 172.21.15.61:0/1638224877 learned_addr learned my addr 172.21.15.61:0/1638224877 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f408affd700 1 -- 172.21.15.61:0/2223700494 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4084014070 con 0x7f40940747d0 2023-10-18T09:18:03.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409a19c700 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e65c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409919a700 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65b580 0x55d83e65ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409a19c700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 0x7f40800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 -- 172.21.15.61:0/1638224877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65b580 msgr2=0x55d83e65ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65b580 0x55d83e65ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 -- 172.21.15.61:0/1638224877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 msgr2=0x55d83e65c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e65c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.473+0000 7f409999b700 1 -- 172.21.15.61:0/1638224877 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d83e56b3a0 con 0x55d83e65cbe0 2023-10-18T09:18:03.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f409999b700 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d80 tx=0x7f4090011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c279efe49d1443 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f409a19c700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 0x7f40800a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f408c009cd0 tx=0x7f408c00b740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f407ffff700 1 -- 172.21.15.61:0/1638224877 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090004230 con 0x55d83e65cbe0 2023-10-18T09:18:03.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f407ffff700 1 -- 172.21.15.61:0/1638224877 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4090004390 con 0x55d83e65cbe0 2023-10-18T09:18:03.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f407ffff700 1 -- 172.21.15.61:0/1638224877 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090019880 con 0x55d83e65cbe0 2023-10-18T09:18:03.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f40af537b80 1 -- 172.21.15.61:0/1638224877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 msgr2=0x55d83e65b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.474+0000 7f40af537b80 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d80 tx=0x7f4090011720 comp rx=0 tx=0).stop 2023-10-18T09:18:03.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 -- 172.21.15.61:0/1638224877 shutdown_connections 2023-10-18T09:18:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65b580 0x55d83e65ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e65c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 --2- 172.21.15.61:0/1638224877 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e65b040 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 -- 172.21.15.61:0/1638224877 >> 172.21.15.61:0/1638224877 conn(0x55d83e5cf6d0 msgr2=0x55d83e655b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 -- 172.21.15.61:0/1638224877 shutdown_connections 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 -- 172.21.15.61:0/1638224877 wait complete. 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 -- start start 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.475+0000 7f40af537b80 1 Processor -- start 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65b580 0x55d83e664ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e664ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5ade20 con 0x55d83e65b580 2023-10-18T09:18:03.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e54a6c0 con 0x55d83e65c280 2023-10-18T09:18:03.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e54a360 con 0x55d83e65cbe0 2023-10-18T09:18:03.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f40af537b80 10 client.0 _ll_register_callbacks cb 0x55d83e494a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:03.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f409a19c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f409999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65b580 0x55d83e664ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f409a19c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:50222/0 (socket says 172.21.15.61:50222) 2023-10-18T09:18:03.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f409a19c700 1 -- 172.21.15.61:0/3654684599 learned_addr learned my addr 172.21.15.61:0/3654684599 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.476+0000 7f409919a700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e664ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 -- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 msgr2=0x55d83e664ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e664ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 -- 172.21.15.61:0/3654684599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65b580 msgr2=0x55d83e664ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65b580 0x55d83e664ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 -- 172.21.15.61:0/3654684599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40900099e0 con 0x55d83e65cbe0 2023-10-18T09:18:03.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f409a19c700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b0a0 tx=0x7f408c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c1fa2d2f1ac0ae server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f407d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:18:03.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c00e040 con 0x55d83e65cbe0 2023-10-18T09:18:03.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d83e5b2ea0 con 0x55d83e65cbe0 2023-10-18T09:18:03.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f408c020d70 con 0x55d83e65cbe0 2023-10-18T09:18:03.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408c005040 con 0x55d83e65cbe0 2023-10-18T09:18:03.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.477+0000 7f40af537b80 10 client.25354 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:03.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d83e3d8d90 con 0x55d83e65cbe0 2023-10-18T09:18:03.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d83e5b2ea0 con 0x55d83e65cbe0 2023-10-18T09:18:03.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f408c02a070 con 0x55d83e65cbe0 2023-10-18T09:18:03.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 10 client.25354 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:03.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 10 client.25354 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:18:03.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 20 client.25354 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:03.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e670760 0x55d83e66f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 4 client.25354 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:03.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d83e62a860 con 0x55d83e670760 2023-10-18T09:18:03.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.478+0000 7f409999b700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e670760 0x55d83e66f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.479+0000 7f407d7fa700 10 client.25354 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:03.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.479+0000 7f409999b700 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e670760 0x55d83e66f1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9953df463f110c server_cookie=946e29aac72cf03a in_seq=0 out_seq=0 2023-10-18T09:18:03.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f407d7fa700 1 -- 172.21.15.61:0/3654684599 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d83e62a860 con 0x55d83e670760 2023-10-18T09:18:03.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f407d7fa700 10 client.25354 handle_command_reply: tid=0 2023-10-18T09:18:03.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 client.25354 shutdown 2023-10-18T09:18:03.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 msgr2=0x55d83e66c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b0a0 tx=0x7f408c0099e0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 shutdown_connections 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e670760 0x55d83e66f1d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e66c030 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65b580 0x55d83e664ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f407d7fa700 0 client.25354 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:03.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 --2- 172.21.15.61:0/3654684599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65c280 0x55d83e664ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 >> 172.21.15.61:0/3654684599 conn(0x55d83e5cf6d0 msgr2=0x55d83e655ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 shutdown_connections 2023-10-18T09:18:03.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.480+0000 7f40af537b80 1 -- 172.21.15.61:0/3654684599 wait complete. 2023-10-18T09:18:03.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.481+0000 7f40af537b80 20 client.25354 trim_cache size 0 max 16384 2023-10-18T09:18:03.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 Processor -- start 2023-10-18T09:18:03.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 -- start start 2023-10-18T09:18:03.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e65b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65c280 0x55d83e66bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e42e8d0 con 0x55d83e65cbe0 2023-10-18T09:18:03.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5b2ed0 con 0x55d83e65b580 2023-10-18T09:18:03.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f40af537b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e5a9f10 con 0x55d83e65c280 2023-10-18T09:18:03.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f409a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f409a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42674/0 (socket says 172.21.15.61:42674) 2023-10-18T09:18:03.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f409a19c700 1 -- 172.21.15.61:0/3611648413 learned_addr learned my addr 172.21.15.61:0/3611648413 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f409999b700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e65b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.499+0000 7f409919a700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65c280 0x55d83e66bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 -- 172.21.15.61:0/3611648413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65c280 msgr2=0x55d83e66bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65c280 0x55d83e66bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 -- 172.21.15.61:0/3611648413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 msgr2=0x55d83e65b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e65b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 -- 172.21.15.61:0/3611648413 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d83e3c1c10 con 0x55d83e65cbe0 2023-10-18T09:18:03.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409999b700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e65b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:03.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f409a19c700 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f408c0095e0 tx=0x7f408c009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180f73335c6b01b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f4088bff700 1 -- 172.21.15.61:0/3611648413 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c007830 con 0x55d83e65cbe0 2023-10-18T09:18:03.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f4088bff700 1 -- 172.21.15.61:0/3611648413 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408c003e90 con 0x55d83e65cbe0 2023-10-18T09:18:03.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.500+0000 7f4088bff700 1 -- 172.21.15.61:0/3611648413 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c005310 con 0x55d83e65cbe0 2023-10-18T09:18:03.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- 172.21.15.61:0/3611648413 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 msgr2=0x55d83e664f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f408c0095e0 tx=0x7f408c009bc0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- 172.21.15.61:0/3611648413 shutdown_connections 2023-10-18T09:18:03.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65cbe0 0x55d83e664f70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e65b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 --2- 172.21.15.61:0/3611648413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65c280 0x55d83e66bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- 172.21.15.61:0/3611648413 >> 172.21.15.61:0/3611648413 conn(0x55d83e5cf6d0 msgr2=0x55d83e664860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- 172.21.15.61:0/3611648413 shutdown_connections 2023-10-18T09:18:03.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- 172.21.15.61:0/3611648413 wait complete. 2023-10-18T09:18:03.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.501+0000 7f40af537b80 1 -- start start 2023-10-18T09:18:03.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 Processor -- start 2023-10-18T09:18:03.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e666c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e6c4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e66c1a0 con 0x55d83e65c280 2023-10-18T09:18:03.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e66c310 con 0x55d83e65b580 2023-10-18T09:18:03.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d83e6c4690 con 0x55d83e65cbe0 2023-10-18T09:18:03.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f40af537b80 10 client.0 _ll_register_callbacks cb 0x55d83e494a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:03.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f409919a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f409919a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42686/0 (socket says 172.21.15.61:42686) 2023-10-18T09:18:03.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.502+0000 7f409919a700 1 -- 172.21.15.61:0/914754160 learned_addr learned my addr 172.21.15.61:0/914754160 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:03.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409a19c700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e6c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409999b700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e666c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 -- 172.21.15.61:0/914754160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 msgr2=0x55d83e6c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e6c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 -- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 msgr2=0x55d83e666c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:03.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e666c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 -- 172.21.15.61:0/914754160 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d83e56b3a0 con 0x55d83e65c280 2023-10-18T09:18:03.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f409919a700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4084096a70 tx=0x7f4084004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4bc4d0e99e9789 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:03.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f407d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:03.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408409ac70 con 0x55d83e65c280 2023-10-18T09:18:03.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f408409a570 con 0x55d83e65c280 2023-10-18T09:18:03.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.503+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4090004390 con 0x55d83e65c280 2023-10-18T09:18:03.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f408409e6c0 con 0x55d83e65c280 2023-10-18T09:18:03.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 10 client.15351 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:03.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d83e3d8d90 con 0x55d83e65c280 2023-10-18T09:18:03.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f408407d020 con 0x55d83e65c280 2023-10-18T09:18:03.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4084099a60 con 0x55d83e65c280 2023-10-18T09:18:03.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:03.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 10 client.15351 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:18:03.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 20 client.15351 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:03.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e6c9a30 0x55d83e6c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:03.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 4 client.15351 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:03.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d83e5a6e00 con 0x55d83e6c9a30 2023-10-18T09:18:03.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.504+0000 7f409999b700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e6c9a30 0x55d83e6c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:03.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f409999b700 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e6c9a30 0x55d83e6c5ed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9a3568adeb9e20 server_cookie=147b01d0dd80259 in_seq=0 out_seq=0 2023-10-18T09:18:03.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f407d7fa700 10 client.15351 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:03.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f407d7fa700 1 -- 172.21.15.61:0/914754160 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d83e5a6e00 con 0x55d83e6c9a30 2023-10-18T09:18:03.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f407d7fa700 10 client.15351 handle_command_reply: tid=0 2023-10-18T09:18:03.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 client.15351 shutdown 2023-10-18T09:18:03.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 msgr2=0x55d83e6671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4084096a70 tx=0x7f4084004280 comp rx=0 tx=0).stop 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 shutdown_connections 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55d83e6c9a30 0x55d83e6c5ed0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d83e65cbe0 0x55d83e6c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83e65b580 0x55d83e666c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f407d7fa700 0 client.15351 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:03.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 --2- 172.21.15.61:0/914754160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d83e65c280 0x55d83e6671c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 >> 172.21.15.61:0/914754160 conn(0x55d83e5cf6d0 msgr2=0x55d83e664860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.505+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 shutdown_connections 2023-10-18T09:18:03.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f40af537b80 1 -- 172.21.15.61:0/914754160 wait complete. 2023-10-18T09:18:03.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f40af537b80 20 client.15351 trim_cache size 0 max 16384 2023-10-18T09:18:03.523 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:03.523 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:03.523 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 0, 2023-10-18T09:18:03.524 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24692, 2023-10-18T09:18:03.524 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:03.524 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:03.524 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:03.524 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 96.657507632000005, 2023-10-18T09:18:03.525 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:03.525 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 90, 2023-10-18T09:18:03.525 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:03.525 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 103.297887872 2023-10-18T09:18:03.525 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:03.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 msgr2=0x7f40800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 0x7f40800a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f408c009cd0 tx=0x7f408c00b740 comp rx=0 tx=0).stop 2023-10-18T09:18:03.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 msgr2=0x7f4094198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:03.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f408400eb30 tx=0x7f408400ee40 comp rx=0 tx=0).stop 2023-10-18T09:18:03.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 shutdown_connections 2023-10-18T09:18:03.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40800a1510 0x7f40800a39d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f40941085f0 0x7f409419fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40940731f0 0x7f4094198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 --2- 172.21.15.61:0/2223700494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40940747d0 0x7f4094198eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:03.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 >> 172.21.15.61:0/2223700494 conn(0x7f40940fbc90 msgr2=0x7f4094104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:03.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.506+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 shutdown_connections 2023-10-18T09:18:03.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:03.507+0000 7f409bbff700 1 -- 172.21.15.61:0/2223700494 wait complete. 2023-10-18T09:18:03.530 INFO:tasks.cephadm:Setup complete, yielding 2023-10-18T09:18:03.530 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-18T09:18:03.540 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:18:03.540 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-18T09:18:04.284 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:05.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.806+0000 7fa4fe669700 1 -- 172.21.15.61:0/2952311843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 msgr2=0x7fa4f8108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:05.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.806+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2952311843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f8108c90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4f000b0d0 tx=0x7fa4f000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.806+0000 7fa4fe669700 1 -- 172.21.15.61:0/2952311843 shutdown_connections 2023-10-18T09:18:05.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.806+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2952311843 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f8108c90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.806+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2952311843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8100840 0x7fa4f8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.807+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2952311843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8101a40 0x7fa4f8101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.807+0000 7fa4fe669700 1 -- 172.21.15.61:0/2952311843 >> 172.21.15.61:0/2952311843 conn(0x7fa4f80fbe00 msgr2=0x7fa4f80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:05.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.807+0000 7fa4fe669700 1 -- 172.21.15.61:0/2952311843 shutdown_connections 2023-10-18T09:18:05.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.807+0000 7fa4fe669700 1 -- 172.21.15.61:0/2952311843 wait complete. 2023-10-18T09:18:05.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 Processor -- start 2023-10-18T09:18:05.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 -- start start 2023-10-18T09:18:05.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 0x7fa4f8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:05.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 0x7fa4f8198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:05.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:05.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81a05b0 con 0x7fa4f81023e0 2023-10-18T09:18:05.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81a06f0 con 0x7fa4f8101a40 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.808+0000 7fa4fe669700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81a0880 con 0x7fa4f8100840 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4f7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 0x7fa4f8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4f77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 0x7fa4f8198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42712/0 (socket says 172.21.15.61:42712) 2023-10-18T09:18:05.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 -- 172.21.15.61:0/2851441816 learned_addr learned my addr 172.21.15.61:0/2851441816 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:05.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 -- 172.21.15.61:0/2851441816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 msgr2=0x7fa4f8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:05.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 0x7fa4f8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 -- 172.21.15.61:0/2851441816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 msgr2=0x7fa4f8198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:05.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 0x7fa4f8198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:05.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 -- 172.21.15.61:0/2851441816 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0009d00 con 0x7fa4f81023e0 2023-10-18T09:18:05.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4f7fff700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 0x7fa4f8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:05.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4f77fe700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 0x7fa4f8198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:05.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.809+0000 7fa4fcc06700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0008e40 tx=0x7fa4f0008e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:05.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.810+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f000e040 con 0x7fa4f81023e0 2023-10-18T09:18:05.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.810+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f001d930 con 0x7fa4f81023e0 2023-10-18T09:18:05.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.810+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f81a0b00 con 0x7fa4f81023e0 2023-10-18T09:18:05.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.810+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f001b9f0 con 0x7fa4f81023e0 2023-10-18T09:18:05.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.810+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f81a0fc0 con 0x7fa4f81023e0 2023-10-18T09:18:05.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.811+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f804e4c0 con 0x7fa4f81023e0 2023-10-18T09:18:05.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.813+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa4f001daa0 con 0x7fa4f81023e0 2023-10-18T09:18:05.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.814+0000 7fa4f57fa700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 0x7fa4e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:05.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.814+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa4f00ce500 con 0x7fa4f81023e0 2023-10-18T09:18:05.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.814+0000 7fa4f7fff700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 0x7fa4e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:05.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.815+0000 7fa4f7fff700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 0x7fa4e00a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa4f81018a0 tx=0x7fa4e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:05.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:05.818+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa4f0097720 con 0x7fa4f81023e0 2023-10-18T09:18:06.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.048+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4f8074140 con 0x7fa4e00a1590 2023-10-18T09:18:06.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.050+0000 7fa4f57fa700 1 -- 172.21.15.61:0/2851441816 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa4f8074140 con 0x7fa4e00a1590 2023-10-18T09:18:06.052 INFO:teuthology.orchestra.run.smithi061.stdout:Backend: cephadm 2023-10-18T09:18:06.052 INFO:teuthology.orchestra.run.smithi061.stdout:Available: Yes 2023-10-18T09:18:06.053 INFO:teuthology.orchestra.run.smithi061.stdout:Paused: No 2023-10-18T09:18:06.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 msgr2=0x7fa4e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:06.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 0x7fa4e00a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa4f81018a0 tx=0x7fa4e8009260 comp rx=0 tx=0).stop 2023-10-18T09:18:06.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 msgr2=0x7fa4f819ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:06.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0008e40 tx=0x7fa4f0008e70 comp rx=0 tx=0).stop 2023-10-18T09:18:06.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 shutdown_connections 2023-10-18T09:18:06.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa4e00a1590 0x7fa4e00a3a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:06.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa4f81023e0 0x7fa4f819ffe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:06.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4f8100840 0x7fa4f8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:06.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 --2- 172.21.15.61:0/2851441816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4f8101a40 0x7fa4f8198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:06.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 >> 172.21.15.61:0/2851441816 conn(0x7fa4f80fbe00 msgr2=0x7fa4f8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:06.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 shutdown_connections 2023-10-18T09:18:06.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:06.052+0000 7fa4fe669700 1 -- 172.21.15.61:0/2851441816 wait complete. 2023-10-18T09:18:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:05 smithi110 ceph-mon[134818]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:06.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:05 smithi061 ceph-mon[98573]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:06.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:05 smithi170 ceph-mon[135150]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:06.589 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-18T09:18:07.239 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:07.798 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:07 smithi061 ceph-mon[98573]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:07.798 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:07 smithi061 ceph-mon[98573]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:07.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:07 smithi170 ceph-mon[135150]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:07.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:07 smithi170 ceph-mon[135150]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:07.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:07 smithi110 ceph-mon[134818]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:07.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:07 smithi110 ceph-mon[134818]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:08.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 -- 172.21.15.61:0/103026267 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 msgr2=0x7fa5fc10aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:08.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 --2- 172.21.15.61:0/103026267 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc10aba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400b0a0 tx=0x7fa5f400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 -- 172.21.15.61:0/103026267 shutdown_connections 2023-10-18T09:18:08.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 --2- 172.21.15.61:0/103026267 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc10aba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 --2- 172.21.15.61:0/103026267 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc074820 0x7fa5fc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 --2- 172.21.15.61:0/103026267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc073250 0x7fa5fc0736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 -- 172.21.15.61:0/103026267 >> 172.21.15.61:0/103026267 conn(0x7fa5fc0fbe00 msgr2=0x7fa5fc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:08.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 -- 172.21.15.61:0/103026267 shutdown_connections 2023-10-18T09:18:08.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.747+0000 7fa603d6c700 1 -- 172.21.15.61:0/103026267 wait complete. 2023-10-18T09:18:08.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 Processor -- start 2023-10-18T09:18:08.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 -- start start 2023-10-18T09:18:08.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 0x7fa5fc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:08.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc074820 0x7fa5fc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:08.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:08.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc1a0600 con 0x7fa5fc108710 2023-10-18T09:18:08.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc1a0740 con 0x7fa5fc073250 2023-10-18T09:18:08.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.748+0000 7fa603d6c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc1a08d0 con 0x7fa5fc074820 2023-10-18T09:18:08.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:08.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa601b08700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 0x7fa5fc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:08.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa601b08700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 0x7fa5fc198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:54392/0 (socket says 172.21.15.61:54392) 2023-10-18T09:18:08.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa601b08700 1 -- 172.21.15.61:0/2559236184 learned_addr learned my addr 172.21.15.61:0/2559236184 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:08.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa601307700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc074820 0x7fa5fc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:08.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 -- 172.21.15.61:0/2559236184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc074820 msgr2=0x7fa5fc199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:08.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc074820 0x7fa5fc199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 -- 172.21.15.61:0/2559236184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 msgr2=0x7fa5fc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:08.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 0x7fa5fc198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:08.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 -- 172.21.15.61:0/2559236184 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f4009d00 con 0x7fa5fc108710 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.749+0000 7fa602309700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc1a0030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400b070 tx=0x7fa5f4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.750+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f400e040 con 0x7fa5fc108710 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.750+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc1a0b50 con 0x7fa5fc108710 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.750+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f400be40 con 0x7fa5fc108710 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.750+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f401b460 con 0x7fa5fc108710 2023-10-18T09:18:08.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.750+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc1a0fc0 con 0x7fa5fc108710 2023-10-18T09:18:08.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.751+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc1074d0 con 0x7fa5fc108710 2023-10-18T09:18:08.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.753+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fa5f401d470 con 0x7fa5fc108710 2023-10-18T09:18:08.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.754+0000 7fa5f2ffd700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 0x7fa5e80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:08.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.754+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa5f40cdff0 con 0x7fa5fc108710 2023-10-18T09:18:08.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.754+0000 7fa601b08700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 0x7fa5e80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:08.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.755+0000 7fa601b08700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 0x7fa5e80a3a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005950 tx=0x7fa5f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:08.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.759+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5f4101100 con 0x7fa5fc108710 2023-10-18T09:18:08.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.980+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5fc02d070 con 0x7fa5e80a1590 2023-10-18T09:18:08.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.995+0000 7fa5f2ffd700 1 -- 172.21.15.61:0/2559236184 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fa5fc02d070 con 0x7fa5e80a1590 2023-10-18T09:18:08.997 INFO:teuthology.orchestra.run.smithi061.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-18T09:18:08.997 INFO:teuthology.orchestra.run.smithi061.stdout:mds.a smithi061 running (111s) 24s ago 110s 16.0M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 00d0119d4836 2023-10-18T09:18:08.998 INFO:teuthology.orchestra.run.smithi061.stdout:mds.b smithi110 running (109s) 36s ago 108s 31.1M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 d8c60f9a4a84 2023-10-18T09:18:08.998 INFO:teuthology.orchestra.run.smithi061.stdout:mds.c smithi170 running (2m) 36s ago 2m 17.4M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 4a7fb03c2f56 2023-10-18T09:18:08.998 INFO:teuthology.orchestra.run.smithi061.stdout:mds.d smithi061 running (2m) 24s ago 2m 13.9M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 6284a8d93602 2023-10-18T09:18:08.998 INFO:teuthology.orchestra.run.smithi061.stdout:mds.e smithi110 running (2m) 36s ago 2m 17.9M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 713b4ffd248d 2023-10-18T09:18:08.998 INFO:teuthology.orchestra.run.smithi061.stdout:mds.f smithi170 running (2m) 36s ago 2m 21.4M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 007a91b4f85a 2023-10-18T09:18:08.999 INFO:teuthology.orchestra.run.smithi061.stdout:mds.g smithi061 running (114s) 24s ago 114s 17.7M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 2fd453c37166 2023-10-18T09:18:08.999 INFO:teuthology.orchestra.run.smithi061.stdout:mds.h smithi110 running (2m) 36s ago 119s 20.5M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 d3620978b1c4 2023-10-18T09:18:08.999 INFO:teuthology.orchestra.run.smithi061.stdout:mds.i smithi170 running (117s) 36s ago 117s 18.3M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 83140e29df1b 2023-10-18T09:18:08.999 INFO:teuthology.orchestra.run.smithi061.stdout:mds.j smithi061 running (2m) 24s ago 2m 13.9M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 33ab81c00017 2023-10-18T09:18:08.999 INFO:teuthology.orchestra.run.smithi061.stdout:mds.k smithi110 running (2m) 36s ago 2m 19.7M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 7089d5bedc01 2023-10-18T09:18:09.000 INFO:teuthology.orchestra.run.smithi061.stdout:mds.l smithi170 running (2m) 36s ago 2m 17.4M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 e0e11fe610bb 2023-10-18T09:18:09.000 INFO:teuthology.orchestra.run.smithi061.stdout:mgr.x smithi061 *:9283,8765 running (9m) 24s ago 9m 529M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 99c5d242f278 2023-10-18T09:18:09.000 INFO:teuthology.orchestra.run.smithi061.stdout:mgr.y smithi110 *:8443,8765 running (7m) 36s ago 7m 449M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 fa63f2f8dfd5 2023-10-18T09:18:09.000 INFO:teuthology.orchestra.run.smithi061.stdout:mgr.z smithi170 *:8443,8765 running (7m) 36s ago 7m 453M - 18.0.0-6786-gf7420d7f 34f3475ed6c6 aa46dbb587f3 2023-10-18T09:18:09.000 INFO:teuthology.orchestra.run.smithi061.stdout:mon.a smithi061 running (9m) 24s ago 9m 98.2M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 0a47830f9083 2023-10-18T09:18:09.001 INFO:teuthology.orchestra.run.smithi061.stdout:mon.b smithi110 running (7m) 36s ago 7m 66.9M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 a6933b532edd 2023-10-18T09:18:09.001 INFO:teuthology.orchestra.run.smithi061.stdout:mon.c smithi170 running (8m) 36s ago 8m 68.6M 2048M 18.0.0-6786-gf7420d7f 34f3475ed6c6 b42666a530eb 2023-10-18T09:18:09.001 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0 smithi061 running (6m) 24s ago 6m 71.3M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 be2d6103b349 2023-10-18T09:18:09.001 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1 smithi110 running (6m) 36s ago 6m 78.8M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 cc3113c06dba 2023-10-18T09:18:09.001 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2 smithi170 running (6m) 36s ago 6m 78.4M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 19fef01ff9c4 2023-10-18T09:18:09.002 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3 smithi061 running (5m) 24s ago 5m 66.6M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 23aa615a0fff 2023-10-18T09:18:09.002 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4 smithi110 running (5m) 36s ago 5m 66.1M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 554d91e35296 2023-10-18T09:18:09.002 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5 smithi170 running (4m) 36s ago 4m 69.7M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 2f7f2b4050ca 2023-10-18T09:18:09.002 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6 smithi061 running (4m) 24s ago 4m 63.1M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 7a4f643a0628 2023-10-18T09:18:09.002 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 smithi110 running (4m) 36s ago 4m 67.9M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 3a4f71945de7 2023-10-18T09:18:09.003 INFO:teuthology.orchestra.run.smithi061.stdout:osd.8 smithi170 running (3m) 36s ago 3m 64.1M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 1ed89aaeddb6 2023-10-18T09:18:09.003 INFO:teuthology.orchestra.run.smithi061.stdout:osd.9 smithi061 running (3m) 24s ago 3m 62.0M 4317M 18.0.0-6786-gf7420d7f 34f3475ed6c6 1a9b811c3d23 2023-10-18T09:18:09.003 INFO:teuthology.orchestra.run.smithi061.stdout:osd.10 smithi110 running (3m) 36s ago 2m 63.2M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 cedfc6fea795 2023-10-18T09:18:09.003 INFO:teuthology.orchestra.run.smithi061.stdout:osd.11 smithi170 running (2m) 36s ago 2m 59.8M 4292M 18.0.0-6786-gf7420d7f 34f3475ed6c6 51a2efa79523 2023-10-18T09:18:09.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 msgr2=0x7fa5e80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:09.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 0x7fa5e80a3a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005950 tx=0x7fa5f8009260 comp rx=0 tx=0).stop 2023-10-18T09:18:09.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 msgr2=0x7fa5fc1a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:09.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc1a0030 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400b070 tx=0x7fa5f4007930 comp rx=0 tx=0).stop 2023-10-18T09:18:09.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 shutdown_connections 2023-10-18T09:18:09.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5e80a1590 0x7fa5e80a3a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:09.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5fc108710 0x7fa5fc1a0030 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:09.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5fc073250 0x7fa5fc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:09.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 --2- 172.21.15.61:0/2559236184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5fc074820 0x7fa5fc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:09.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 >> 172.21.15.61:0/2559236184 conn(0x7fa5fc0fbe00 msgr2=0x7fa5fc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:09.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 shutdown_connections 2023-10-18T09:18:09.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:08.997+0000 7fa603d6c700 1 -- 172.21.15.61:0/2559236184 wait complete. 2023-10-18T09:18:09.543 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-18T09:18:10.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:09 smithi110 ceph-mon[134818]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:10.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:09 smithi061 ceph-mon[98573]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:10.291 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:10.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:09 smithi170 ceph-mon[135150]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:11.167 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:10 smithi061 ceph-mon[98573]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:11.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:10 smithi110 ceph-mon[134818]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:10 smithi170 ceph-mon[135150]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:11.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 -- 172.21.15.61:0/1103549881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc100840 msgr2=0x7f57cc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:11.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 --2- 172.21.15.61:0/1103549881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc100840 0x7f57cc100c60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0099c0 tx=0x7f57bc009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 -- 172.21.15.61:0/1103549881 shutdown_connections 2023-10-18T09:18:11.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 --2- 172.21.15.61:0/1103549881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc1023e0 0x7f57cc108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 --2- 172.21.15.61:0/1103549881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc100840 0x7f57cc100c60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 --2- 172.21.15.61:0/1103549881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 0x7f57cc101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 -- 172.21.15.61:0/1103549881 >> 172.21.15.61:0/1103549881 conn(0x7f57cc0fbe00 msgr2=0x7f57cc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:11.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 -- 172.21.15.61:0/1103549881 shutdown_connections 2023-10-18T09:18:11.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.730+0000 7f57d10e1700 1 -- 172.21.15.61:0/1103549881 wait complete. 2023-10-18T09:18:11.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.731+0000 7f57d10e1700 1 Processor -- start 2023-10-18T09:18:11.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.731+0000 7f57d10e1700 1 -- start start 2023-10-18T09:18:11.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.731+0000 7f57d10e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:11.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57d10e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 0x7f57cc196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:11.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57d10e1700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 0x7f57cc19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:11.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57d10e1700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19e2e0 con 0x7f57cc1023e0 2023-10-18T09:18:11.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57d10e1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19e420 con 0x7f57cc101a40 2023-10-18T09:18:11.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57d10e1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19e5b0 con 0x7f57cc100840 2023-10-18T09:18:11.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cb59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 0x7f57cc19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:11.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:11.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 0x7f57cc196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:11.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57264/0 (socket says 172.21.15.61:57264) 2023-10-18T09:18:11.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 -- 172.21.15.61:0/2122050514 learned_addr learned my addr 172.21.15.61:0/2122050514 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:11.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 -- 172.21.15.61:0/2122050514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 msgr2=0x7f57cc196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:11.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 0x7f57cc196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 -- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 msgr2=0x7f57cc19dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:11.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 0x7f57cc19dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cad9d700 1 -- 172.21.15.61:0/2122050514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0095f0 con 0x7f57cc100840 2023-10-18T09:18:11.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.732+0000 7f57cb59e700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 0x7f57cc19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:11.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57cad9d700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0058e0 tx=0x7f57bc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:11.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc01d070 con 0x7f57cc100840 2023-10-18T09:18:11.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc19e830 con 0x7f57cc100840 2023-10-18T09:18:11.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc005050 con 0x7f57cc100840 2023-10-18T09:18:11.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004ab0 con 0x7f57cc100840 2023-10-18T09:18:11.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.733+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc19eca0 con 0x7f57cc100840 2023-10-18T09:18:11.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.734+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc107d20 con 0x7f57cc100840 2023-10-18T09:18:11.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.736+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f57bc005430 con 0x7f57cc100840 2023-10-18T09:18:11.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.737+0000 7f57b3fff700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 0x7f57b40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:11.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.738+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57bc0ce340 con 0x7f57cc100840 2023-10-18T09:18:11.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.738+0000 7f57ca59c700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 0x7f57b40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:11.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.738+0000 7f57ca59c700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 0x7f57b40a39d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009c60 tx=0x7f57c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:11.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.740+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f57bc097560 con 0x7f57cc100840 2023-10-18T09:18:11.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.961+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f57cc060c30 con 0x7f57b40a1510 2023-10-18T09:18:11.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.968+0000 7f57b3fff700 1 -- 172.21.15.61:0/2122050514 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f57cc060c30 con 0x7f57b40a1510 2023-10-18T09:18:11.970 INFO:teuthology.orchestra.run.smithi061.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-18T09:18:11.970 INFO:teuthology.orchestra.run.smithi061.stdout:mds.all 12/12 39s ago 2m smithi061=a;smithi061=d;smithi061=g;smithi061=j;smithi110=b;smithi110=e;smithi110=h;smithi110=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2023-10-18T09:18:11.971 INFO:teuthology.orchestra.run.smithi061.stdout:mgr 3/3 39s ago 7m smithi061=x;smithi110=y;smithi170=z;count:3 2023-10-18T09:18:11.971 INFO:teuthology.orchestra.run.smithi061.stdout:mon 3/3 39s ago 8m smithi061:172.21.15.61=a;smithi110:172.21.15.110=b;smithi170:172.21.15.170=c;count:3 2023-10-18T09:18:11.971 INFO:teuthology.orchestra.run.smithi061.stdout:osd 12 39s ago - 2023-10-18T09:18:11.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 msgr2=0x7f57b40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:11.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 0x7f57b40a39d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009c60 tx=0x7f57c0009260 comp rx=0 tx=0).stop 2023-10-18T09:18:11.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 msgr2=0x7f57cc1967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:11.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0058e0 tx=0x7f57bc00f690 comp rx=0 tx=0).stop 2023-10-18T09:18:11.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 shutdown_connections 2023-10-18T09:18:11.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f57b40a1510 0x7f57b40a39d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f57cc1023e0 0x7f57cc19dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57cc100840 0x7f57cc1967a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 --2- 172.21.15.61:0/2122050514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f57cc101a40 0x7f57cc196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:11.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 >> 172.21.15.61:0/2122050514 conn(0x7f57cc0fbe00 msgr2=0x7f57cc104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:11.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 shutdown_connections 2023-10-18T09:18:11.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:11.970+0000 7f57d10e1700 1 -- 172.21.15.61:0/2122050514 wait complete. 2023-10-18T09:18:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:11 smithi110 ceph-mon[134818]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:11 smithi061 ceph-mon[98573]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:12.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:11 smithi170 ceph-mon[135150]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:12.492 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-18T09:18:13.243 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:14.076 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:13 smithi061 ceph-mon[98573]: from='client.25378 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:14.076 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:13 smithi061 ceph-mon[98573]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:13 smithi110 ceph-mon[134818]: from='client.25378 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:13 smithi110 ceph-mon[134818]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:14.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:13 smithi170 ceph-mon[135150]: from='client.25378 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:14.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:13 smithi170 ceph-mon[135150]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-10-18T09:18:14.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 -- 172.21.15.61:0/2059771983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490102200 msgr2=0x7ff490108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:14.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 --2- 172.21.15.61:0/2059771983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490102200 0x7ff490108ab0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b0d0 tx=0x7ff48800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 -- 172.21.15.61:0/2059771983 shutdown_connections 2023-10-18T09:18:14.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 --2- 172.21.15.61:0/2059771983 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490102200 0x7ff490108ab0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 --2- 172.21.15.61:0/2059771983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 0x7ff490100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.775 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 --2- 172.21.15.61:0/2059771983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490101860 0x7ff490101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 -- 172.21.15.61:0/2059771983 >> 172.21.15.61:0/2059771983 conn(0x7ff4900fbc00 msgr2=0x7ff4900fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:14.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.771+0000 7ff49541c700 1 -- 172.21.15.61:0/2059771983 shutdown_connections 2023-10-18T09:18:14.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.772+0000 7ff49541c700 1 -- 172.21.15.61:0/2059771983 wait complete. 2023-10-18T09:18:14.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.773+0000 7ff49541c700 1 Processor -- start 2023-10-18T09:18:14.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.773+0000 7ff49541c700 1 -- start start 2023-10-18T09:18:14.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 0x7ff4901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:14.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:14.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490102200 0x7ff49019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:14.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a03d0 con 0x7ff490101860 2023-10-18T09:18:14.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a0510 con 0x7ff490102200 2023-10-18T09:18:14.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff49541c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a06a0 con 0x7ff490100660 2023-10-18T09:18:14.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:14.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35874/0 (socket says 172.21.15.61:35874) 2023-10-18T09:18:14.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 -- 172.21.15.61:0/3485697246 learned_addr learned my addr 172.21.15.61:0/3485697246 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:14.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48ffff700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 0x7ff4901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:14.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff494c1b700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490102200 0x7ff49019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:14.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 -- 172.21.15.61:0/3485697246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 msgr2=0x7ff4901988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:14.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 0x7ff4901988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 -- 172.21.15.61:0/3485697246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490102200 msgr2=0x7ff49019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:14.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490102200 0x7ff49019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:14.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.774+0000 7ff48f7fe700 1 -- 172.21.15.61:0/3485697246 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484009520 con 0x7ff490101860 2023-10-18T09:18:14.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff48f7fe700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff48400eb60 tx=0x7ff48400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:14.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400cbd0 con 0x7ff490101860 2023-10-18T09:18:14.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4840044e0 con 0x7ff490101860 2023-10-18T09:18:14.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484003df0 con 0x7ff490101860 2023-10-18T09:18:14.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488009d00 con 0x7ff490101860 2023-10-18T09:18:14.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.775+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4901a0cf0 con 0x7ff490101860 2023-10-18T09:18:14.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.776+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49004e4c0 con 0x7ff490101860 2023-10-18T09:18:14.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.778+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7ff484003f50 con 0x7ff490101860 2023-10-18T09:18:14.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.780+0000 7ff48d7fa700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 0x7ff4780a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:14.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.780+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff484014070 con 0x7ff490101860 2023-10-18T09:18:14.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.780+0000 7ff48ffff700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 0x7ff4780a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:14.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.781+0000 7ff48ffff700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 0x7ff4780a3a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4901016c0 tx=0x7ff480009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:14.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:14.784+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff4840966d0 con 0x7ff490101860 2023-10-18T09:18:15.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.006+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff490074140 con 0x7ff4780a1590 2023-10-18T09:18:15.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.008+0000 7ff48d7fa700 1 -- 172.21.15.61:0/3485697246 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7ff490074140 con 0x7ff4780a1590 2023-10-18T09:18:15.011 INFO:teuthology.orchestra.run.smithi061.stdout:HOST ADDR LABELS STATUS 2023-10-18T09:18:15.011 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061 172.21.15.61 2023-10-18T09:18:15.011 INFO:teuthology.orchestra.run.smithi061.stdout:smithi110 172.21.15.110 2023-10-18T09:18:15.011 INFO:teuthology.orchestra.run.smithi061.stdout:smithi170 172.21.15.170 2023-10-18T09:18:15.012 INFO:teuthology.orchestra.run.smithi061.stdout:3 hosts in cluster 2023-10-18T09:18:15.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.010+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 msgr2=0x7ff4780a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:15.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.010+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 0x7ff4780a3a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4901016c0 tx=0x7ff480009260 comp rx=0 tx=0).stop 2023-10-18T09:18:15.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.010+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 msgr2=0x7ff490198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:15.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.010+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff48400eb60 tx=0x7ff48400ee70 comp rx=0 tx=0).stop 2023-10-18T09:18:15.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 shutdown_connections 2023-10-18T09:18:15.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff4780a1590 0x7ff4780a3a50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:15.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff490102200 0x7ff49019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:15.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff490100660 0x7ff4901988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:15.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 --2- 172.21.15.61:0/3485697246 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff490101860 0x7ff490198e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:15.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 >> 172.21.15.61:0/3485697246 conn(0x7ff4900fbc00 msgr2=0x7ff490072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:15.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 shutdown_connections 2023-10-18T09:18:15.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:15.011+0000 7ff49541c700 1 -- 172.21.15.61:0/3485697246 wait complete. 2023-10-18T09:18:15.540 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-18T09:18:16.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:15 smithi110 ceph-mon[134818]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:16.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:15 smithi061 ceph-mon[98573]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:16.288 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:16.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:15 smithi170 ceph-mon[135150]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-10-18T09:18:17.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:16 smithi110 ceph-mon[134818]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:17.229 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:16 smithi061 ceph-mon[98573]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:17.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:16 smithi170 ceph-mon[135150]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:17.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 -- 172.21.15.61:0/1447870501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 msgr2=0x7fef4c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:17.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 --2- 172.21.15.61:0/1447870501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c100c60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef400099c0 tx=0x7fef40009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 -- 172.21.15.61:0/1447870501 shutdown_connections 2023-10-18T09:18:17.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 --2- 172.21.15.61:0/1447870501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 0x7fef4c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 --2- 172.21.15.61:0/1447870501 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c100c60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 --2- 172.21.15.61:0/1447870501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 0x7fef4c101ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 -- 172.21.15.61:0/1447870501 >> 172.21.15.61:0/1447870501 conn(0x7fef4c0fbe00 msgr2=0x7fef4c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:17.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 -- 172.21.15.61:0/1447870501 shutdown_connections 2023-10-18T09:18:17.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.797+0000 7fef53064700 1 -- 172.21.15.61:0/1447870501 wait complete. 2023-10-18T09:18:17.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.798+0000 7fef53064700 1 Processor -- start 2023-10-18T09:18:17.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.798+0000 7fef53064700 1 -- start start 2023-10-18T09:18:17.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 0x7fef4c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 0x7fef4c19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1a04d0 con 0x7fef4c100840 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1a0610 con 0x7fef4c101a40 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef53064700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c1a07a0 con 0x7fef4c1023e0 2023-10-18T09:18:17.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:17.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35900/0 (socket says 172.21.15.61:35900) 2023-10-18T09:18:17.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 -- 172.21.15.61:0/2347480167 learned_addr learned my addr 172.21.15.61:0/2347480167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:17.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef4bfff700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 0x7fef4c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:17.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef51601700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 0x7fef4c19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:17.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 -- 172.21.15.61:0/2347480167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 msgr2=0x7fef4c19ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:17.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 0x7fef4c19ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 -- 172.21.15.61:0/2347480167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 msgr2=0x7fef4c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:17.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 0x7fef4c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:17.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.799+0000 7fef50e00700 1 -- 172.21.15.61:0/2347480167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400095f0 con 0x7fef4c100840 2023-10-18T09:18:17.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef50e00700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b580 tx=0x7fef4000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:17.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4001d070 con 0x7fef4c100840 2023-10-18T09:18:17.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef40005510 con 0x7fef4c100840 2023-10-18T09:18:17.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c1a0a20 con 0x7fef4c100840 2023-10-18T09:18:17.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40004b30 con 0x7fef4c100840 2023-10-18T09:18:17.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.800+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c1a0e90 con 0x7fef4c100840 2023-10-18T09:18:17.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.801+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c04e4c0 con 0x7fef4c100840 2023-10-18T09:18:17.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.803+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7fef40004c90 con 0x7fef4c100840 2023-10-18T09:18:17.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.804+0000 7fef49ffb700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 0x7fef3c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:17.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.805+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fef400cdfb0 con 0x7fef4c100840 2023-10-18T09:18:17.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.805+0000 7fef4bfff700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 0x7fef3c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:17.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.805+0000 7fef4bfff700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 0x7fef3c0a3a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fef38009c60 tx=0x7fef38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:17.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:17.809+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef400972f0 con 0x7fef4c100840 2023-10-18T09:18:18.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.031+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fef4c108950 con 0x7fef3c0a1590 2023-10-18T09:18:18.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.033+0000 7fef49ffb700 1 -- 172.21.15.61:0/2347480167 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fef4c108950 con 0x7fef3c0a1590 2023-10-18T09:18:18.035 INFO:teuthology.orchestra.run.smithi061.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-18T09:18:18.035 INFO:teuthology.orchestra.run.smithi061.stdout:smithi061 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534300NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-10-18T09:18:18.036 INFO:teuthology.orchestra.run.smithi061.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-10-18T09:18:18.036 INFO:teuthology.orchestra.run.smithi061.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-10-18T09:18:18.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 msgr2=0x7fef3c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:18.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 0x7fef3c0a3a50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fef38009c60 tx=0x7fef38009260 comp rx=0 tx=0).stop 2023-10-18T09:18:18.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 msgr2=0x7fef4c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:18.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fef4000b580 tx=0x7fef4000f690 comp rx=0 tx=0).stop 2023-10-18T09:18:18.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 shutdown_connections 2023-10-18T09:18:18.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef3c0a1590 0x7fef3c0a3a50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:18.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef4c1023e0 0x7fef4c19ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:18.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef4c100840 0x7fef4c198990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:18.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 --2- 172.21.15.61:0/2347480167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef4c101a40 0x7fef4c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:18.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 >> 172.21.15.61:0/2347480167 conn(0x7fef4c0fbe00 msgr2=0x7fef4c107230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:18.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 shutdown_connections 2023-10-18T09:18:18.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:18.035+0000 7fef53064700 1 -- 172.21.15.61:0/2347480167 wait complete. 2023-10-18T09:18:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:17 smithi110 ceph-mon[134818]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:18.213 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:17 smithi061 ceph-mon[98573]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:18.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:17 smithi170 ceph-mon[135150]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2023-10-18T09:18:18.595 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-18T09:18:18.605 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:18:18.605 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-18T09:18:19.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:18 smithi110 ceph-mon[134818]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:19.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:18 smithi110 ceph-mon[134818]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:19.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:18 smithi061 ceph-mon[98573]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:19.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:18 smithi061 ceph-mon[98573]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:19.356 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:18 smithi170 ceph-mon[135150]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-18T09:18:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:18 smithi170 ceph-mon[135150]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:18:20.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 -- 172.21.15.61:0/4107643720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c101a40 msgr2=0x7f619c101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:20.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 --2- 172.21.15.61:0/4107643720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c101a40 0x7f619c101ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f618c0099c0 tx=0x7f618c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 -- 172.21.15.61:0/4107643720 shutdown_connections 2023-10-18T09:18:20.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 --2- 172.21.15.61:0/4107643720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 0x7f619c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 --2- 172.21.15.61:0/4107643720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c100840 0x7f619c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 --2- 172.21.15.61:0/4107643720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c101a40 0x7f619c101ea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 -- 172.21.15.61:0/4107643720 >> 172.21.15.61:0/4107643720 conn(0x7f619c0fbe00 msgr2=0x7f619c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:20.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 -- 172.21.15.61:0/4107643720 shutdown_connections 2023-10-18T09:18:20.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.889+0000 7f61a46e5700 1 -- 172.21.15.61:0/4107643720 wait complete. 2023-10-18T09:18:20.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 Processor -- start 2023-10-18T09:18:20.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 -- start start 2023-10-18T09:18:20.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:20.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 0x7f619c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:20.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 0x7f619c19fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:20.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1a04c0 con 0x7f619c100840 2023-10-18T09:18:20.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1a0600 con 0x7f619c101a40 2023-10-18T09:18:20.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.890+0000 7f61a46e5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c1a0790 con 0x7f619c1023e0 2023-10-18T09:18:20.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:20.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37360/0 (socket says 172.21.15.61:37360) 2023-10-18T09:18:20.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 -- 172.21.15.61:0/3201003831 learned_addr learned my addr 172.21.15.61:0/3201003831 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:20.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a1c80700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 0x7f619c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:20.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2c82700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 0x7f619c19fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:20.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 -- 172.21.15.61:0/3201003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 msgr2=0x7f619c19fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:20.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 0x7f619c19fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 -- 172.21.15.61:0/3201003831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 msgr2=0x7f619c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:20.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 0x7f619c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:20.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a2481700 1 -- 172.21.15.61:0/3201003831 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0095f0 con 0x7f619c100840 2023-10-18T09:18:20.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.891+0000 7f61a1c80700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 0x7f619c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:20.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61a2481700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f619800d9b0 tx=0x7f619800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:20.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800b950 con 0x7f619c100840 2023-10-18T09:18:20.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6198007620 con 0x7f619c100840 2023-10-18T09:18:20.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c1a0a70 con 0x7f619c100840 2023-10-18T09:18:20.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800f3c0 con 0x7f619c100840 2023-10-18T09:18:20.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.892+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1a0f70 con 0x7f619c100840 2023-10-18T09:18:20.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.893+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c04e4c0 con 0x7f619c100840 2023-10-18T09:18:20.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.895+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 17) v1 ==== 136054+0+0 (secure 0 0 0) 0x7f6198010460 con 0x7f619c100840 2023-10-18T09:18:20.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.896+0000 7f61937fe700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 0x7f61880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:20.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.896+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f61980ccfe0 con 0x7f619c100840 2023-10-18T09:18:20.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.897+0000 7f61a1c80700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 0x7f61880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:20.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.897+0000 7f61a1c80700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 0x7f61880a39d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f618c003730 tx=0x7f618c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:20.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:20.900+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6198100100 con 0x7f619c100840 2023-10-18T09:18:21.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.161+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f619c10c340 con 0x7f619c100840 2023-10-18T09:18:21.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.162+0000 7f61937fe700 1 -- 172.21.15.61:0/3201003831 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2981 (secure 0 0 0) 0x7f61980961b0 con 0x7f619c100840 2023-10-18T09:18:21.165 INFO:teuthology.orchestra.run.smithi061.stdout:e28 2023-10-18T09:18:21.165 INFO:teuthology.orchestra.run.smithi061.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-18T09:18:21.165 INFO:teuthology.orchestra.run.smithi061.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T09:18:21.166 INFO:teuthology.orchestra.run.smithi061.stdout:legacy client fscid: 1 2023-10-18T09:18:21.166 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:21.166 INFO:teuthology.orchestra.run.smithi061.stdout:Filesystem 'cephfs' (1) 2023-10-18T09:18:21.166 INFO:teuthology.orchestra.run.smithi061.stdout:fs_name cephfs 2023-10-18T09:18:21.166 INFO:teuthology.orchestra.run.smithi061.stdout:epoch 28 2023-10-18T09:18:21.167 INFO:teuthology.orchestra.run.smithi061.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-18T09:18:21.167 INFO:teuthology.orchestra.run.smithi061.stdout:created 2023-10-18T09:16:26.396655+0000 2023-10-18T09:18:21.167 INFO:teuthology.orchestra.run.smithi061.stdout:modified 2023-10-18T09:16:59.542245+0000 2023-10-18T09:18:21.167 INFO:teuthology.orchestra.run.smithi061.stdout:tableserver 0 2023-10-18T09:18:21.168 INFO:teuthology.orchestra.run.smithi061.stdout:root 0 2023-10-18T09:18:21.168 INFO:teuthology.orchestra.run.smithi061.stdout:session_timeout 300 2023-10-18T09:18:21.168 INFO:teuthology.orchestra.run.smithi061.stdout:session_autoclose 300 2023-10-18T09:18:21.168 INFO:teuthology.orchestra.run.smithi061.stdout:max_file_size 1099511627776 2023-10-18T09:18:21.168 INFO:teuthology.orchestra.run.smithi061.stdout:max_xattr_size 65536 2023-10-18T09:18:21.169 INFO:teuthology.orchestra.run.smithi061.stdout:required_client_features {} 2023-10-18T09:18:21.169 INFO:teuthology.orchestra.run.smithi061.stdout:last_failure 0 2023-10-18T09:18:21.169 INFO:teuthology.orchestra.run.smithi061.stdout:last_failure_osd_epoch 0 2023-10-18T09:18:21.169 INFO:teuthology.orchestra.run.smithi061.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-18T09:18:21.169 INFO:teuthology.orchestra.run.smithi061.stdout:max_mds 5 2023-10-18T09:18:21.170 INFO:teuthology.orchestra.run.smithi061.stdout:in 0,1,2,3,4 2023-10-18T09:18:21.170 INFO:teuthology.orchestra.run.smithi061.stdout:up {0=24692,1=24676,2=24653,3=24646,4=24629} 2023-10-18T09:18:21.170 INFO:teuthology.orchestra.run.smithi061.stdout:failed 2023-10-18T09:18:21.170 INFO:teuthology.orchestra.run.smithi061.stdout:damaged 2023-10-18T09:18:21.171 INFO:teuthology.orchestra.run.smithi061.stdout:stopped 2023-10-18T09:18:21.171 INFO:teuthology.orchestra.run.smithi061.stdout:data_pools [3,4] 2023-10-18T09:18:21.171 INFO:teuthology.orchestra.run.smithi061.stdout:metadata_pool 2 2023-10-18T09:18:21.171 INFO:teuthology.orchestra.run.smithi061.stdout:inline_data disabled 2023-10-18T09:18:21.171 INFO:teuthology.orchestra.run.smithi061.stdout:balancer 2023-10-18T09:18:21.172 INFO:teuthology.orchestra.run.smithi061.stdout:bal_rank_mask -1 2023-10-18T09:18:21.172 INFO:teuthology.orchestra.run.smithi061.stdout:standby_count_wanted 0 2023-10-18T09:18:21.172 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.b{0:24692} state up:active seq 3 addr [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.172 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.g{0:24668} state up:standby-replay seq 1 addr [v2:172.21.15.61:6838/2550944546,v1:172.21.15.61:6839/2550944546] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.172 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.i{1:24676} state up:active seq 13 addr [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.173 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.c{1:24658} state up:standby-replay seq 1 addr [v2:172.21.15.170:6836/3025282224,v1:172.21.15.170:6837/3025282224] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.173 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.h{2:24653} state up:active seq 13 addr [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.173 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.k{2:24593} state up:standby-replay seq 1 addr [v2:172.21.15.110:6832/3590204423,v1:172.21.15.110:6833/3590204423] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.173 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.f{3:24646} state up:active seq 15 addr [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.174 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.l{3:24589} state up:standby-replay seq 1 addr [v2:172.21.15.170:6832/2686391817,v1:172.21.15.170:6833/2686391817] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.174 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.e{4:24629} state up:active seq 15 addr [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.174 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.a{4:14688} state up:standby-replay seq 1 addr [v2:172.21.15.61:6840/2114349313,v1:172.21.15.61:6841/2114349313] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.174 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:21.174 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:21.175 INFO:teuthology.orchestra.run.smithi061.stdout:Standby daemons: 2023-10-18T09:18:21.175 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:21.175 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.j{-1:14604} state up:standby seq 1 addr [v2:172.21.15.61:6834/2809482931,v1:172.21.15.61:6835/2809482931] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.175 INFO:teuthology.orchestra.run.smithi061.stdout:[mds.d{-1:14616} state up:standby seq 1 addr [v2:172.21.15.61:6836/1807421967,v1:172.21.15.61:6837/1807421967] compat {c=[1],r=[1],i=[fff]}] 2023-10-18T09:18:21.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 msgr2=0x7f61880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:21.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 0x7f61880a39d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f618c003730 tx=0x7f618c01a040 comp rx=0 tx=0).stop 2023-10-18T09:18:21.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 msgr2=0x7f619c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:21.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f619800d9b0 tx=0x7f619800dcc0 comp rx=0 tx=0).stop 2023-10-18T09:18:21.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 shutdown_connections 2023-10-18T09:18:21.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61880a1510 0x7f61880a39d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:21.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f619c1023e0 0x7f619c19fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:21.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f619c100840 0x7f619c198980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:21.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 --2- 172.21.15.61:0/3201003831 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f619c101a40 0x7f619c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:21.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.164+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 >> 172.21.15.61:0/3201003831 conn(0x7f619c0fbe00 msgr2=0x7f619c1071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:21.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.165+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 shutdown_connections 2023-10-18T09:18:21.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:21.165+0000 7f61a46e5700 1 -- 172.21.15.61:0/3201003831 wait complete. 2023-10-18T09:18:21.179 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:21.654 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-18T09:18:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:21 smithi110 ceph-mon[134818]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-18T09:18:22.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:21 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-18T09:18:22.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:21 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3201003831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:21 smithi061 ceph-mon[98573]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-18T09:18:22.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:21 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-18T09:18:22.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:21 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3201003831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T09:18:22.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:21 smithi170 ceph-mon[135150]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-10-18T09:18:22.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-10-18T09:18:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:21 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-10-18T09:18:22.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:21 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3201003831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-18T09:18:22.414 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:23.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-18T09:18:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:22 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-18T09:18:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:22 smithi061 ceph-mon[98573]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T09:18:23.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:22 smithi061 ceph-mon[98573]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-18T09:18:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-18T09:18:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:22 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-18T09:18:23.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:22 smithi110 ceph-mon[134818]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T09:18:23.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:22 smithi110 ceph-mon[134818]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-18T09:18:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-10-18T09:18:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:22 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-10-18T09:18:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:22 smithi170 ceph-mon[135150]: osdmap e91: 12 total, 12 up, 12 in 2023-10-18T09:18:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:22 smithi170 ceph-mon[135150]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-10-18T09:18:23.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 -- 172.21.15.61:0/1260792015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5298101a40 msgr2=0x7f5298101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:23.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 --2- 172.21.15.61:0/1260792015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5298101a40 0x7f5298101ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f528c0099c0 tx=0x7f528c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 -- 172.21.15.61:0/1260792015 shutdown_connections 2023-10-18T09:18:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 --2- 172.21.15.61:0/1260792015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52981023e0 0x7f5298108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 --2- 172.21.15.61:0/1260792015 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 0x7f5298100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 --2- 172.21.15.61:0/1260792015 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5298101a40 0x7f5298101ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 -- 172.21.15.61:0/1260792015 >> 172.21.15.61:0/1260792015 conn(0x7f52980fbe00 msgr2=0x7f52980fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:23.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 -- 172.21.15.61:0/1260792015 shutdown_connections 2023-10-18T09:18:23.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.946+0000 7f529e87f700 1 -- 172.21.15.61:0/1260792015 wait complete. 2023-10-18T09:18:23.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.947+0000 7f529e87f700 1 Processor -- start 2023-10-18T09:18:23.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.947+0000 7f529e87f700 1 -- start start 2023-10-18T09:18:23.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 0x7f5298198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:23.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101a40 0x7f5298198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:23.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:23.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52981a04c0 con 0x7f52981023e0 2023-10-18T09:18:23.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52981a0600 con 0x7f5298100840 2023-10-18T09:18:23.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529e87f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52981a0790 con 0x7f5298101a40 2023-10-18T09:18:23.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:23.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37378/0 (socket says 172.21.15.61:37378) 2023-10-18T09:18:23.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 -- 172.21.15.61:0/2440670273 learned_addr learned my addr 172.21.15.61:0/2440670273 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:23.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f5297fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 0x7f5298198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:23.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f52977fe700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101a40 0x7f5298198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:23.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 -- 172.21.15.61:0/2440670273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101a40 msgr2=0x7f5298198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:23.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101a40 0x7f5298198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 -- 172.21.15.61:0/2440670273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 msgr2=0x7f5298198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:23.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 0x7f5298198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:23.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.948+0000 7f529ce1c700 1 -- 172.21.15.61:0/2440670273 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c0095f0 con 0x7f52981023e0 2023-10-18T09:18:23.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f529ce1c700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f529000ef60 tx=0x7f529000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:23.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290018070 con 0x7f52981023e0 2023-10-18T09:18:23.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f529000f040 con 0x7f52981023e0 2023-10-18T09:18:23.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52981a0a10 con 0x7f52981023e0 2023-10-18T09:18:23.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000ccc0 con 0x7f52981023e0 2023-10-18T09:18:23.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.949+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52981a0f60 con 0x7f52981023e0 2023-10-18T09:18:23.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.950+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529804e4c0 con 0x7f52981023e0 2023-10-18T09:18:23.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.952+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5290003a00 con 0x7f52981023e0 2023-10-18T09:18:23.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.953+0000 7f52957fa700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 0x7f52800a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:23.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.953+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f52900cca70 con 0x7f52981023e0 2023-10-18T09:18:23.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.954+0000 7f5297fff700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 0x7f52800a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:23.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.954+0000 7f5297fff700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 0x7f52800a39c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52981018a0 tx=0x7f5288009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:23.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:23.957+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5290095db0 con 0x7f52981023e0 2023-10-18T09:18:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:23 smithi061 ceph-mon[98573]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:18:24.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:23 smithi061 ceph-mon[98573]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T09:18:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.168+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f529810c090 con 0x7f52981023e0 2023-10-18T09:18:24.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.169+0000 7f52957fa700 1 -- 172.21.15.61:0/2440670273 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5640 (secure 0 0 0) 0x7f52900993d0 con 0x7f52981023e0 2023-10-18T09:18:24.172 INFO:teuthology.orchestra.run.smithi061.stdout:epoch 92 2023-10-18T09:18:24.172 INFO:teuthology.orchestra.run.smithi061.stdout:fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:18:24.172 INFO:teuthology.orchestra.run.smithi061.stdout:created 2023-10-18T09:08:20.265998+0000 2023-10-18T09:18:24.172 INFO:teuthology.orchestra.run.smithi061.stdout:modified 2023-10-18T09:18:22.923624+0000 2023-10-18T09:18:24.173 INFO:teuthology.orchestra.run.smithi061.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-18T09:18:24.173 INFO:teuthology.orchestra.run.smithi061.stdout:crush_version 27 2023-10-18T09:18:24.173 INFO:teuthology.orchestra.run.smithi061.stdout:full_ratio 0.95 2023-10-18T09:18:24.173 INFO:teuthology.orchestra.run.smithi061.stdout:backfillfull_ratio 0.9 2023-10-18T09:18:24.173 INFO:teuthology.orchestra.run.smithi061.stdout:nearfull_ratio 0.85 2023-10-18T09:18:24.174 INFO:teuthology.orchestra.run.smithi061.stdout:require_min_compat_client luminous 2023-10-18T09:18:24.174 INFO:teuthology.orchestra.run.smithi061.stdout:min_compat_client luminous 2023-10-18T09:18:24.174 INFO:teuthology.orchestra.run.smithi061.stdout:require_osd_release reef 2023-10-18T09:18:24.174 INFO:teuthology.orchestra.run.smithi061.stdout:stretch_mode_enabled false 2023-10-18T09:18:24.175 INFO:teuthology.orchestra.run.smithi061.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-18T09:18:24.175 INFO:teuthology.orchestra.run.smithi061.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-10-18T09:18:24.175 INFO:teuthology.orchestra.run.smithi061.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-18T09:18:24.175 INFO:teuthology.orchestra.run.smithi061.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-18T09:18:24.176 INFO:teuthology.orchestra.run.smithi061.stdout:max_osd 12 2023-10-18T09:18:24.176 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.61:6802/3245715235,v1:172.21.15.61:6803/3245715235] [v2:172.21.15.61:6804/3245715235,v1:172.21.15.61:6805/3245715235] exists,up 1a451299-d658-4830-9f80-9ba0776f9122 2023-10-18T09:18:24.176 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/4194494003,v1:172.21.15.110:6801/4194494003] [v2:172.21.15.110:6802/4194494003,v1:172.21.15.110:6803/4194494003] exists,up 5535c5e8-804e-4d87-915d-00862ef1b386 2023-10-18T09:18:24.176 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/696261473,v1:172.21.15.170:6801/696261473] [v2:172.21.15.170:6802/696261473,v1:172.21.15.170:6803/696261473] exists,up 222ac61b-a988-44c1-b86e-b31756c1f539 2023-10-18T09:18:24.176 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.61:6810/671964640,v1:172.21.15.61:6811/671964640] [v2:172.21.15.61:6812/671964640,v1:172.21.15.61:6813/671964640] exists,up c79af743-8b1a-477c-a7ce-f4845ca1a61a 2023-10-18T09:18:24.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4 up in weight 1 up_from 34 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/113917828,v1:172.21.15.110:6809/113917828] [v2:172.21.15.110:6810/113917828,v1:172.21.15.110:6811/113917828] exists,up e542f6db-5212-4e76-b5d8-05029c585fd7 2023-10-18T09:18:24.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/1107914770,v1:172.21.15.170:6809/1107914770] [v2:172.21.15.170:6810/1107914770,v1:172.21.15.170:6811/1107914770] exists,up 2069c32b-e740-4ea4-b647-35be7d41f3df 2023-10-18T09:18:24.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.61:6818/2592494554,v1:172.21.15.61:6819/2592494554] [v2:172.21.15.61:6820/2592494554,v1:172.21.15.61:6821/2592494554] exists,up a152124f-a0de-47d0-97b3-b65467c83915 2023-10-18T09:18:24.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/3683648719,v1:172.21.15.110:6817/3683648719] [v2:172.21.15.110:6818/3683648719,v1:172.21.15.110:6819/3683648719] exists,up 4e523e57-7424-443a-99ba-a6becb55b91a 2023-10-18T09:18:24.178 INFO:teuthology.orchestra.run.smithi061.stdout:osd.8 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/3321062874,v1:172.21.15.170:6817/3321062874] [v2:172.21.15.170:6818/3321062874,v1:172.21.15.170:6819/3321062874] exists,up ff8f0e0e-869f-4e76-a3b5-7f6d43633dff 2023-10-18T09:18:24.178 INFO:teuthology.orchestra.run.smithi061.stdout:osd.9 up in weight 1 up_from 67 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.61:6826/65349699,v1:172.21.15.61:6827/65349699] [v2:172.21.15.61:6828/65349699,v1:172.21.15.61:6829/65349699] exists,up f66eb5b2-0e99-4f40-ad44-4d81f5a6c910 2023-10-18T09:18:24.178 INFO:teuthology.orchestra.run.smithi061.stdout:osd.10 up in weight 1 up_from 72 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/508893070,v1:172.21.15.110:6825/508893070] [v2:172.21.15.110:6826/508893070,v1:172.21.15.110:6827/508893070] exists,up 0cbca8f8-088f-43d3-bd0e-dcb212e8756e 2023-10-18T09:18:24.178 INFO:teuthology.orchestra.run.smithi061.stdout:osd.11 up in weight 1 up_from 79 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/4083010574,v1:172.21.15.170:6825/4083010574] [v2:172.21.15.170:6826/4083010574,v1:172.21.15.170:6827/4083010574] exists,up 4f2b592b-6b55-4c04-b153-da70a63b88f8 2023-10-18T09:18:24.179 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 2.6 [8,11] 2023-10-18T09:18:24.179 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 2.e [2,4] 2023-10-18T09:18:24.179 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 2.13 [2,11] 2023-10-18T09:18:24.180 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 3.3 [4,9] 2023-10-18T09:18:24.180 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 3.e [5,9] 2023-10-18T09:18:24.181 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 3.11 [4,8] 2023-10-18T09:18:24.181 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 3.14 [4,6] 2023-10-18T09:18:24.181 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 3.1e [3,9] 2023-10-18T09:18:24.182 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 4.c [9,10] 2023-10-18T09:18:24.182 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 4.f [5,8] 2023-10-18T09:18:24.183 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 4.16 [9,8] 2023-10-18T09:18:24.183 INFO:teuthology.orchestra.run.smithi061.stdout:pg_upmap_items 4.1f [9,8] 2023-10-18T09:18:24.183 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/3728584080 expires 2023-10-19T09:09:20.739745+0000 2023-10-18T09:18:24.184 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/1938489455 expires 2023-10-19T09:09:20.739745+0000 2023-10-18T09:18:24.184 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:6801/2589487616 expires 2023-10-19T09:09:20.739745+0000 2023-10-18T09:18:24.185 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:6801/1676472102 expires 2023-10-19T09:08:53.425206+0000 2023-10-18T09:18:24.185 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/2303013428 expires 2023-10-19T09:08:53.425206+0000 2023-10-18T09:18:24.185 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:6800/1676472102 expires 2023-10-19T09:08:53.425206+0000 2023-10-18T09:18:24.186 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/1518173664 expires 2023-10-19T09:09:20.739745+0000 2023-10-18T09:18:24.186 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:6800/2589487616 expires 2023-10-19T09:09:20.739745+0000 2023-10-18T09:18:24.186 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/2840133191 expires 2023-10-19T09:08:53.425206+0000 2023-10-18T09:18:24.186 INFO:teuthology.orchestra.run.smithi061.stdout:blocklist 172.21.15.61:0/623274229 expires 2023-10-19T09:08:53.425206+0000 2023-10-18T09:18:24.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.171+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 msgr2=0x7f52800a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:24.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.171+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 0x7f52800a39c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f52981018a0 tx=0x7f5288009260 comp rx=0 tx=0).stop 2023-10-18T09:18:24.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.171+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 msgr2=0x7f529819fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:24.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.171+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f529000ef60 tx=0x7f529000ef90 comp rx=0 tx=0).stop 2023-10-18T09:18:24.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 shutdown_connections 2023-10-18T09:18:24.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f52800a1500 0x7f52800a39c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f52981023e0 0x7f529819fef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5298100840 0x7f5298198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 --2- 172.21.15.61:0/2440670273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5298101a40 0x7f5298198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 >> 172.21.15.61:0/2440670273 conn(0x7f52980fbe00 msgr2=0x7f52981071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:24.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 shutdown_connections 2023-10-18T09:18:24.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.172+0000 7f529e87f700 1 -- 172.21.15.61:0/2440670273 wait complete. 2023-10-18T09:18:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:23 smithi110 ceph-mon[134818]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:18:24.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:23 smithi110 ceph-mon[134818]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T09:18:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:23 smithi170 ceph-mon[135150]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:18:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:23 smithi170 ceph-mon[135150]: osdmap e92: 12 total, 12 up, 12 in 2023-10-18T09:18:24.719 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-18T09:18:24.730 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:24.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 -- 172.21.15.61:0/487732858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2280101930 msgr2=0x7f2280101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:24.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 --2- 172.21.15.61:0/487732858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2280101930 0x7f2280101d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22740099c0 tx=0x7f2274009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 -- 172.21.15.61:0/487732858 shutdown_connections 2023-10-18T09:18:24.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 --2- 172.21.15.61:0/487732858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f22801022d0 0x7f2280108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 --2- 172.21.15.61:0/487732858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280100740 0x7f2280100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 --2- 172.21.15.61:0/487732858 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2280101930 0x7f2280101d90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 -- 172.21.15.61:0/487732858 >> 172.21.15.61:0/487732858 conn(0x7f22800fbc90 msgr2=0x7f22800fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:24.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 -- 172.21.15.61:0/487732858 shutdown_connections 2023-10-18T09:18:24.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.941+0000 7f22863ff700 1 -- 172.21.15.61:0/487732858 wait complete. 2023-10-18T09:18:24.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 Processor -- start 2023-10-18T09:18:24.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 -- start start 2023-10-18T09:18:24.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:24.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280101930 0x7f2280198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:24.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801022d0 0x7f228019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:24.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22801a0430 con 0x7f2280100740 2023-10-18T09:18:24.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22801a0570 con 0x7f22801022d0 2023-10-18T09:18:24.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.942+0000 7f22863ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22801a0700 con 0x7f2280101930 2023-10-18T09:18:24.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:24.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37386/0 (socket says 172.21.15.61:37386) 2023-10-18T09:18:24.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 -- 172.21.15.61:0/3061941753 learned_addr learned my addr 172.21.15.61:0/3061941753 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:24.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f228499c700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801022d0 0x7f228019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:24.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227f7fe700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280101930 0x7f2280198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:24.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 -- 172.21.15.61:0/3061941753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280101930 msgr2=0x7f2280198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:24.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280101930 0x7f2280198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 -- 172.21.15.61:0/3061941753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801022d0 msgr2=0x7f228019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:24.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801022d0 0x7f228019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:24.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 -- 172.21.15.61:0/3061941753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22740095f0 con 0x7f2280100740 2023-10-18T09:18:24.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.943+0000 7f227ffff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f227000d980 tx=0x7f227000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:24.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.944+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270004020 con 0x7f2280100740 2023-10-18T09:18:24.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.944+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f227000f690 con 0x7f2280100740 2023-10-18T09:18:24.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.944+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22801a09e0 con 0x7f2280100740 2023-10-18T09:18:24.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.944+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270003a60 con 0x7f2280100740 2023-10-18T09:18:24.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.944+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22801a0ef0 con 0x7f2280100740 2023-10-18T09:18:24.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.945+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2280067530 con 0x7f2280100740 2023-10-18T09:18:24.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.947+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f227000fb00 con 0x7f2280100740 2023-10-18T09:18:24.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.948+0000 7f227d7fa700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 0x7f22680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:24.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.948+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f22700ccc50 con 0x7f2280100740 2023-10-18T09:18:24.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.948+0000 7f227f7fe700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 0x7f22680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:24.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.949+0000 7f227f7fe700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 0x7f22680a3940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f227400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:24.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:24.952+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2270095f00 con 0x7f2280100740 2023-10-18T09:18:25.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.167+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f228010bfb0 con 0x7f2280100740 2023-10-18T09:18:25.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.169+0000 7f227d7fa700 1 -- 172.21.15.61:0/3061941753 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f2270099520 con 0x7f2280100740 2023-10-18T09:18:25.172 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:25.173 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:25.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 msgr2=0x7f22680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 0x7f22680a3940 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2274009990 tx=0x7f227400b540 comp rx=0 tx=0).stop 2023-10-18T09:18:25.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 msgr2=0x7f2280198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f227000d980 tx=0x7f227000dc90 comp rx=0 tx=0).stop 2023-10-18T09:18:25.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 shutdown_connections 2023-10-18T09:18:25.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f22680a1480 0x7f22680a3940 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801022d0 0x7f228019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2280100740 0x7f2280198950 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 --2- 172.21.15.61:0/3061941753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2280101930 0x7f2280198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 >> 172.21.15.61:0/3061941753 conn(0x7f22800fbc90 msgr2=0x7f2280072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:25.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 shutdown_connections 2023-10-18T09:18:25.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.171+0000 7f22863ff700 1 -- 172.21.15.61:0/3061941753 wait complete. 2023-10-18T09:18:25.191 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2440670273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T09:18:25.192 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:24 smithi061 ceph-mon[98573]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-18T09:18:25.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:18:25.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:18:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2440670273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T09:18:25.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:24 smithi110 ceph-mon[134818]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-18T09:18:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2440670273' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-18T09:18:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:24 smithi170 ceph-mon[135150]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 895 B/s wr, 8 op/s 2023-10-18T09:18:25.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 -- 172.21.15.61:0/139281506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910102330 msgr2=0x7f5910108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 --2- 172.21.15.61:0/139281506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910102330 0x7f5910108be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f590800b0d0 tx=0x7f590800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 -- 172.21.15.61:0/139281506 shutdown_connections 2023-10-18T09:18:25.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 --2- 172.21.15.61:0/139281506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910102330 0x7f5910108be0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 --2- 172.21.15.61:0/139281506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 0x7f5910100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 --2- 172.21.15.61:0/139281506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910101990 0x7f5910101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 -- 172.21.15.61:0/139281506 >> 172.21.15.61:0/139281506 conn(0x7f59100fbcf0 msgr2=0x7f59100fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:25.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 -- 172.21.15.61:0/139281506 shutdown_connections 2023-10-18T09:18:25.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.402+0000 7f59179ff700 1 -- 172.21.15.61:0/139281506 wait complete. 2023-10-18T09:18:25.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 Processor -- start 2023-10-18T09:18:25.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 -- start start 2023-10-18T09:18:25.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 0x7f5910196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019e370 con 0x7f5910102330 2023-10-18T09:18:25.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019e4b0 con 0x7f5910101990 2023-10-18T09:18:25.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.403+0000 7f59179ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019e640 con 0x7f59101007a0 2023-10-18T09:18:25.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5914f9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f591579b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 0x7f5910196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5914f9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46050/0 (socket says 172.21.15.61:46050) 2023-10-18T09:18:25.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5914f9a700 1 -- 172.21.15.61:0/846259455 learned_addr learned my addr 172.21.15.61:0/846259455 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:25.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37408/0 (socket says 172.21.15.61:37408) 2023-10-18T09:18:25.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 -- 172.21.15.61:0/846259455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 msgr2=0x7f5910196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 0x7f5910196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 -- 172.21.15.61:0/846259455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 msgr2=0x7f5910196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 -- 172.21.15.61:0/846259455 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009520 con 0x7f5910102330 2023-10-18T09:18:25.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5914f9a700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:25.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.404+0000 7f5915f9c700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5908007b60 tx=0x7f5908008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:25.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.405+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800e040 con 0x7f5910102330 2023-10-18T09:18:25.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.405+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5908009d00 con 0x7f5910102330 2023-10-18T09:18:25.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.405+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5908005070 con 0x7f5910102330 2023-10-18T09:18:25.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.405+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591019ebb0 con 0x7f5910102330 2023-10-18T09:18:25.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.405+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590801d670 con 0x7f5910102330 2023-10-18T09:18:25.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.406+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5910107b50 con 0x7f5910102330 2023-10-18T09:18:25.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.408+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f59080051e0 con 0x7f5910102330 2023-10-18T09:18:25.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.409+0000 7f59067fc700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 0x7f58fc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.409+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f59080cdc60 con 0x7f5910102330 2023-10-18T09:18:25.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.409+0000 7f591579b700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 0x7f58fc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.410+0000 7f591579b700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 0x7f58fc0a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f59101017f0 tx=0x7f590c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:25.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.413+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5908097030 con 0x7f5910102330 2023-10-18T09:18:25.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.628+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f591010c010 con 0x7f5910102330 2023-10-18T09:18:25.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.628+0000 7f59067fc700 1 -- 172.21.15.61:0/846259455 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f590809a650 con 0x7f5910102330 2023-10-18T09:18:25.631 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:18:25.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 msgr2=0x7f58fc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 0x7f58fc0a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f59101017f0 tx=0x7f590c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:25.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 msgr2=0x7f591019dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5908007b60 tx=0x7f5908008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:25.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 shutdown_connections 2023-10-18T09:18:25.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58fc0a1480 0x7f58fc0a3940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5910102330 0x7f591019dda0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59101007a0 0x7f5910196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 --2- 172.21.15.61:0/846259455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910101990 0x7f5910196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 >> 172.21.15.61:0/846259455 conn(0x7f59100fbcf0 msgr2=0x7f5910104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:25.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 shutdown_connections 2023-10-18T09:18:25.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.631+0000 7f59179ff700 1 -- 172.21.15.61:0/846259455 wait complete. 2023-10-18T09:18:25.652 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:18:25.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/3928475560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8101990 msgr2=0x7fc2a8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/3928475560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8101990 0x7fc2a8101df0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0099c0 tx=0x7fc29c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/3928475560 shutdown_connections 2023-10-18T09:18:25.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/3928475560 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a8102330 0x7fc2a8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/3928475560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a81007a0 0x7fc2a8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/3928475560 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8101990 0x7fc2a8101df0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/3928475560 >> 172.21.15.61:0/3928475560 conn(0x7fc2a80fbcf0 msgr2=0x7fc2a80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/3928475560 shutdown_connections 2023-10-18T09:18:25.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.860+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/3928475560 wait complete. 2023-10-18T09:18:25.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.861+0000 7fc2ad9ff700 1 Processor -- start 2023-10-18T09:18:25.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 -- start start 2023-10-18T09:18:25.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8101990 0x7fc2a8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a0420 con 0x7fc2a8102330 2023-10-18T09:18:25.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a0560 con 0x7fc2a81007a0 2023-10-18T09:18:25.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2ad9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a81a06f0 con 0x7fc2a8101990 2023-10-18T09:18:25.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46066/0 (socket says 172.21.15.61:46066) 2023-10-18T09:18:25.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37428/0 (socket says 172.21.15.61:37428) 2023-10-18T09:18:25.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a6ffd700 1 -- 172.21.15.61:0/2061318726 learned_addr learned my addr 172.21.15.61:0/2061318726 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:25.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.862+0000 7fc2a67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8101990 0x7fc2a8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 -- 172.21.15.61:0/2061318726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8101990 msgr2=0x7fc2a8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8101990 0x7fc2a8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 -- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 msgr2=0x7fc2a819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:25.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:25.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 -- 172.21.15.61:0/2061318726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0095f0 con 0x7fc2a81007a0 2023-10-18T09:18:25.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a77fe700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:25.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2a6ffd700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e970 tx=0x7fc29800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:25.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298009790 con 0x7fc2a81007a0 2023-10-18T09:18:25.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc29800c7e0 con 0x7fc2a81007a0 2023-10-18T09:18:25.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a81a09d0 con 0x7fc2a81007a0 2023-10-18T09:18:25.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298003c70 con 0x7fc2a81007a0 2023-10-18T09:18:25.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.863+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81a0f30 con 0x7fc2a81007a0 2023-10-18T09:18:25.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.864+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80675a0 con 0x7fc2a81007a0 2023-10-18T09:18:25.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.867+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc29800cc50 con 0x7fc2a81007a0 2023-10-18T09:18:25.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.868+0000 7fc2ac9fd700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 0x7fc2900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:25.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.868+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc298014070 con 0x7fc2a81007a0 2023-10-18T09:18:25.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.869+0000 7fc2a67fc700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 0x7fc2900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:25.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.869+0000 7fc2a67fc700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 0x7fc2900a39c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc29c003730 tx=0x7fc29c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:25.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:25.871+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc2980965e0 con 0x7fc2a81007a0 2023-10-18T09:18:26.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.086+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2a81a11e0 con 0x7fc2a81007a0 2023-10-18T09:18:26.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.086+0000 7fc2ac9fd700 1 -- 172.21.15.61:0/2061318726 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc298099c00 con 0x7fc2a81007a0 2023-10-18T09:18:26.089 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:26.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 msgr2=0x7fc2900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 0x7fc2900a39c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc29c003730 tx=0x7fc29c00b540 comp rx=0 tx=0).stop 2023-10-18T09:18:26.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 msgr2=0x7fc2a81988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e970 tx=0x7fc29800ec80 comp rx=0 tx=0).stop 2023-10-18T09:18:26.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 shutdown_connections 2023-10-18T09:18:26.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2900a1500 0x7fc2900a39c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2a8102330 0x7fc2a819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2a81007a0 0x7fc2a81988f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 --2- 172.21.15.61:0/2061318726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a8101990 0x7fc2a8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 >> 172.21.15.61:0/2061318726 conn(0x7fc2a80fbcf0 msgr2=0x7fc2a8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:26.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 shutdown_connections 2023-10-18T09:18:26.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.089+0000 7fc2ad9ff700 1 -- 172.21.15.61:0/2061318726 wait complete. 2023-10-18T09:18:26.111 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3061941753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:26.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/846259455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3061941753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:26.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/846259455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:26.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 -- 172.21.15.61:0/1089140129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 msgr2=0x7f1098100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 --2- 172.21.15.61:0/1089140129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098100bc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10880099c0 tx=0x7f1088009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 -- 172.21.15.61:0/1089140129 shutdown_connections 2023-10-18T09:18:26.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 --2- 172.21.15.61:0/1089140129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 0x7f1098108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 --2- 172.21.15.61:0/1089140129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098100bc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 --2- 172.21.15.61:0/1089140129 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 0x7f1098101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 -- 172.21.15.61:0/1089140129 >> 172.21.15.61:0/1089140129 conn(0x7f10980fbcf0 msgr2=0x7f10980fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:26.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 -- 172.21.15.61:0/1089140129 shutdown_connections 2023-10-18T09:18:26.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.318+0000 7f109cfff700 1 -- 172.21.15.61:0/1089140129 wait complete. 2023-10-18T09:18:26.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.320+0000 7f109cfff700 1 Processor -- start 2023-10-18T09:18:26.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.320+0000 7f109cfff700 1 -- start start 2023-10-18T09:18:26.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.320+0000 7f109cfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109cfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 0x7f1098072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109cfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 0x7f1098073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109cfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10980738c0 con 0x7f1098101990 2023-10-18T09:18:26.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109cfff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098073a30 con 0x7f10981007a0 2023-10-18T09:18:26.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109cfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10981a07b0 con 0x7f1098102330 2023-10-18T09:18:26.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46084/0 (socket says 172.21.15.61:46084) 2023-10-18T09:18:26.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f1095d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 0x7f1098072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 -- 172.21.15.61:0/1143603411 learned_addr learned my addr 172.21.15.61:0/1143603411 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f1096d9d700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 0x7f1098073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 -- 172.21.15.61:0/1143603411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 msgr2=0x7f1098073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 0x7f1098073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 -- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 msgr2=0x7f1098072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 0x7f1098072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 -- 172.21.15.61:0/1143603411 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880095f0 con 0x7f10981007a0 2023-10-18T09:18:26.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.321+0000 7f109659c700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1088003730 tx=0x7f108800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:26.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.322+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108801d070 con 0x7f10981007a0 2023-10-18T09:18:26.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.322+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1088005050 con 0x7f10981007a0 2023-10-18T09:18:26.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.322+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10981a09e0 con 0x7f10981007a0 2023-10-18T09:18:26.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.322+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088004af0 con 0x7f10981007a0 2023-10-18T09:18:26.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.322+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10981a0da0 con 0x7f10981007a0 2023-10-18T09:18:26.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.323+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098107c70 con 0x7f10981007a0 2023-10-18T09:18:26.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.325+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f10880051c0 con 0x7f10981007a0 2023-10-18T09:18:26.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.326+0000 7f10877fe700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 0x7f10800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.326+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f10880ce2f0 con 0x7f10981007a0 2023-10-18T09:18:26.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.327+0000 7f1095d9b700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 0x7f10800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.327+0000 7f1095d9b700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 0x7f10800a3940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:26.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.328+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1088097630 con 0x7f10981007a0 2023-10-18T09:18:26.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3061941753' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:26.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/846259455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:26.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.546+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f109810c110 con 0x7f10981007a0 2023-10-18T09:18:26.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.546+0000 7f10877fe700 1 -- 172.21.15.61:0/1143603411 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f108809ac50 con 0x7f10981007a0 2023-10-18T09:18:26.549 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:26.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 msgr2=0x7f10800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 0x7f10800a3940 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f108c009c60 tx=0x7f108c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:26.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 msgr2=0x7f1098074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1088003730 tx=0x7f108800f690 comp rx=0 tx=0).stop 2023-10-18T09:18:26.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 shutdown_connections 2023-10-18T09:18:26.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f10800a1480 0x7f10800a3940 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1098102330 0x7f1098073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10981007a0 0x7f1098074550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 --2- 172.21.15.61:0/1143603411 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1098101990 0x7f1098072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 >> 172.21.15.61:0/1143603411 conn(0x7f10980fbcf0 msgr2=0x7f1098104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:26.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 shutdown_connections 2023-10-18T09:18:26.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.549+0000 7f109cfff700 1 -- 172.21.15.61:0/1143603411 wait complete. 2023-10-18T09:18:26.571 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:26.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 -- 172.21.15.61:0/1315452045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 msgr2=0x7fd220108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 --2- 172.21.15.61:0/1315452045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd220108be0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd2180099c0 tx=0x7fd218009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 -- 172.21.15.61:0/1315452045 shutdown_connections 2023-10-18T09:18:26.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 --2- 172.21.15.61:0/1315452045 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd220108be0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 --2- 172.21.15.61:0/1315452045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2201007a0 0x7fd220100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 --2- 172.21.15.61:0/1315452045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd220101990 0x7fd220101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 -- 172.21.15.61:0/1315452045 >> 172.21.15.61:0/1315452045 conn(0x7fd2200fbcf0 msgr2=0x7fd2200fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:26.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 -- 172.21.15.61:0/1315452045 shutdown_connections 2023-10-18T09:18:26.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.777+0000 7fd226dff700 1 -- 172.21.15.61:0/1315452045 wait complete. 2023-10-18T09:18:26.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.778+0000 7fd226dff700 1 Processor -- start 2023-10-18T09:18:26.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 -- start start 2023-10-18T09:18:26.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd22019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201a03f0 con 0x7fd220102330 2023-10-18T09:18:26.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201a0530 con 0x7fd2201007a0 2023-10-18T09:18:26.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.779+0000 7fd226dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2201a06c0 con 0x7fd220101990 2023-10-18T09:18:26.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd21ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46092/0 (socket says 172.21.15.61:46092) 2023-10-18T09:18:26.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd21ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41618/0 (socket says 172.21.15.61:41618) 2023-10-18T09:18:26.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 -- 172.21.15.61:0/1681758241 learned_addr learned my addr 172.21.15.61:0/1681758241 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:26.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd22539c700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd22019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 -- 172.21.15.61:0/1681758241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 msgr2=0x7fd220198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 -- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 msgr2=0x7fd22019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd22019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 -- 172.21.15.61:0/1681758241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180095f0 con 0x7fd2201007a0 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd21ffff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.780+0000 7fd224b9b700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd21400efc0 tx=0x7fd21400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.781+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214018070 con 0x7fd2201007a0 2023-10-18T09:18:26.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.781+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd21400f040 con 0x7fd2201007a0 2023-10-18T09:18:26.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.781+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2201a0940 con 0x7fd2201007a0 2023-10-18T09:18:26.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.781+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214004280 con 0x7fd2201007a0 2023-10-18T09:18:26.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.781+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2201a0e50 con 0x7fd2201007a0 2023-10-18T09:18:26.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.782+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2200675a0 con 0x7fd2201007a0 2023-10-18T09:18:26.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.784+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd214003a00 con 0x7fd2201007a0 2023-10-18T09:18:26.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.785+0000 7fd21dffb700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 0x7fd2100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:26.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.785+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd2140cd880 con 0x7fd2201007a0 2023-10-18T09:18:26.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.786+0000 7fd21ffff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 0x7fd2100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:26.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.786+0000 7fd21ffff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 0x7fd2100a3940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd20c011490 tx=0x7fd20c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:26.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:26.787+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd214096bc0 con 0x7fd2201007a0 2023-10-18T09:18:27.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.004+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd22010c010 con 0x7fd2201007a0 2023-10-18T09:18:27.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.004+0000 7fd21dffb700 1 -- 172.21.15.61:0/1681758241 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd21409a1e0 con 0x7fd2201007a0 2023-10-18T09:18:27.008 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:27.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 msgr2=0x7fd2100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 0x7fd2100a3940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd20c011490 tx=0x7fd20c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:27.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 msgr2=0x7fd220198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd21400efc0 tx=0x7fd21400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 shutdown_connections 2023-10-18T09:18:27.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2100a1480 0x7fd2100a3940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd220102330 0x7fd22019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2201007a0 0x7fd220198960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 --2- 172.21.15.61:0/1681758241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd220101990 0x7fd220198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 >> 172.21.15.61:0/1681758241 conn(0x7fd2200fbcf0 msgr2=0x7fd220072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:27.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 shutdown_connections 2023-10-18T09:18:27.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.007+0000 7fd226dff700 1 -- 172.21.15.61:0/1681758241 wait complete. 2023-10-18T09:18:27.029 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:27.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:26 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2061318726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:26 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1143603411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:26 smithi110 ceph-mon[134818]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-10-18T09:18:27.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 -- 172.21.15.61:0/1392180097 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80747d0 msgr2=0x7fcca8072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/1392180097 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80747d0 0x7fcca8072c20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009990 tx=0x7fcca4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 -- 172.21.15.61:0/1392180097 shutdown_connections 2023-10-18T09:18:27.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/1392180097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 0x7fcca810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/1392180097 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80747d0 0x7fcca8072c20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/1392180097 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80731f0 0x7fcca8073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 -- 172.21.15.61:0/1392180097 >> 172.21.15.61:0/1392180097 conn(0x7fcca80fbc90 msgr2=0x7fcca80fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:27.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 -- 172.21.15.61:0/1392180097 shutdown_connections 2023-10-18T09:18:27.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.231+0000 7fccaf7ff700 1 -- 172.21.15.61:0/1392180097 wait complete. 2023-10-18T09:18:27.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.232+0000 7fccaf7ff700 1 Processor -- start 2023-10-18T09:18:27.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.232+0000 7fccaf7ff700 1 -- start start 2023-10-18T09:18:27.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80747d0 0x7fcca8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 0x7fcca819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a0400 con 0x7fcca80731f0 2023-10-18T09:18:27.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a0540 con 0x7fcca80747d0 2023-10-18T09:18:27.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccaf7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca81a06d0 con 0x7fcca8108610 2023-10-18T09:18:27.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccad59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccad59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37494/0 (socket says 172.21.15.61:37494) 2023-10-18T09:18:27.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccad59b700 1 -- 172.21.15.61:0/3034647331 learned_addr learned my addr 172.21.15.61:0/3034647331 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:27.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccadd9c700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 0x7fcca819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccacd9a700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80747d0 0x7fcca8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.233+0000 7fccad59b700 1 -- 172.21.15.61:0/3034647331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 msgr2=0x7fcca819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccad59b700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 0x7fcca819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccad59b700 1 -- 172.21.15.61:0/3034647331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80747d0 msgr2=0x7fcca8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccad59b700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80747d0 0x7fcca8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccad59b700 1 -- 172.21.15.61:0/3034647331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40095f0 con 0x7fcca80731f0 2023-10-18T09:18:27.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccad59b700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcca400b680 tx=0x7fcca4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:27.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca401d070 con 0x7fcca80731f0 2023-10-18T09:18:27.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcca4022470 con 0x7fcca80731f0 2023-10-18T09:18:27.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca81a0950 con 0x7fcca80731f0 2023-10-18T09:18:27.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4017650 con 0x7fcca80731f0 2023-10-18T09:18:27.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.234+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca81a0dd0 con 0x7fcca80731f0 2023-10-18T09:18:27.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.235+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8107420 con 0x7fcca80731f0 2023-10-18T09:18:27.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.237+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcca40225e0 con 0x7fcca80731f0 2023-10-18T09:18:27.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.238+0000 7fcc9e7fc700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 0x7fcc940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.238+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcca40ce2c0 con 0x7fcca80731f0 2023-10-18T09:18:27.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.239+0000 7fccacd9a700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 0x7fcc940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.239+0000 7fccacd9a700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 0x7fcc940a39c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009c60 tx=0x7fcc98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:27.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.243+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcca4097600 con 0x7fcca80731f0 2023-10-18T09:18:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:26 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2061318726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:26 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1143603411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:26 smithi061 ceph-mon[98573]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-10-18T09:18:27.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:26 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2061318726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:26 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1143603411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:27.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:26 smithi170 ceph-mon[135150]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2023-10-18T09:18:27.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.504+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcca8067530 con 0x7fcca80731f0 2023-10-18T09:18:27.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.505+0000 7fcc9e7fc700 1 -- 172.21.15.61:0/3034647331 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fcca409ac20 con 0x7fcca80731f0 2023-10-18T09:18:27.507 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:27.507 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:27.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 msgr2=0x7fcc940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 0x7fcc940a39c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009c60 tx=0x7fcc98009260 comp rx=0 tx=0).stop 2023-10-18T09:18:27.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 msgr2=0x7fcca81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcca400b680 tx=0x7fcca4009580 comp rx=0 tx=0).stop 2023-10-18T09:18:27.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 shutdown_connections 2023-10-18T09:18:27.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc940a1500 0x7fcc940a39c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca8108610 0x7fcca819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcca80731f0 0x7fcca81988d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 --2- 172.21.15.61:0/3034647331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcca80747d0 0x7fcca8198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 >> 172.21.15.61:0/3034647331 conn(0x7fcca80fbc90 msgr2=0x7fcca8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:27.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 shutdown_connections 2023-10-18T09:18:27.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.507+0000 7fccaf7ff700 1 -- 172.21.15.61:0/3034647331 wait complete. 2023-10-18T09:18:27.512 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:27.529 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:27.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3298428422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18101930 msgr2=0x7eff18101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3298428422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18101930 0x7eff18101d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0098f0 tx=0x7eff0c009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:27.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3298428422 shutdown_connections 2023-10-18T09:18:27.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3298428422 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff181022d0 0x7eff18108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3298428422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff18100740 0x7eff18100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3298428422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18101930 0x7eff18101d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3298428422 >> 172.21.15.61:0/3298428422 conn(0x7eff180fbc90 msgr2=0x7eff180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:27.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3298428422 shutdown_connections 2023-10-18T09:18:27.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.732+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3298428422 wait complete. 2023-10-18T09:18:27.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.733+0000 7eff1d1ff700 1 Processor -- start 2023-10-18T09:18:27.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.733+0000 7eff1d1ff700 1 -- start start 2023-10-18T09:18:27.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.733+0000 7eff1d1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18100740 0x7eff18198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.733+0000 7eff1d1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1d1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 0x7eff1819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1d1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a0460 con 0x7eff18101930 2023-10-18T09:18:27.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1d1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a05a0 con 0x7eff181022d0 2023-10-18T09:18:27.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1d1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181a0730 con 0x7eff18100740 2023-10-18T09:18:27.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37506/0 (socket says 172.21.15.61:37506) 2023-10-18T09:18:27.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 -- 172.21.15.61:0/3216334473 learned_addr learned my addr 172.21.15.61:0/3216334473 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:27.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff16d9d700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18100740 0x7eff18198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1759e700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 0x7eff1819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 -- 172.21.15.61:0/3216334473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18100740 msgr2=0x7eff18198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18100740 0x7eff18198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 -- 172.21.15.61:0/3216334473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 msgr2=0x7eff1819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 0x7eff1819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 -- 172.21.15.61:0/3216334473 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080095f0 con 0x7eff18101930 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1759e700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 0x7eff1819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.734+0000 7eff1659c700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009810 tx=0x7eff0c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.735+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c01c070 con 0x7eff18101930 2023-10-18T09:18:27.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.735+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c009520 con 0x7eff18101930 2023-10-18T09:18:27.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.735+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0c005040 con 0x7eff18101930 2023-10-18T09:18:27.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.735+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003a40 con 0x7eff18101930 2023-10-18T09:18:27.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.735+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff181a0c40 con 0x7eff18101930 2023-10-18T09:18:27.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.736+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff18067530 con 0x7eff18101930 2023-10-18T09:18:27.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.738+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7eff0c020030 con 0x7eff18101930 2023-10-18T09:18:27.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.739+0000 7efefffff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 0x7eff000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:27.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.739+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7eff0c0ce0e0 con 0x7eff18101930 2023-10-18T09:18:27.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.739+0000 7eff16d9d700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 0x7eff000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:27.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.740+0000 7eff16d9d700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 0x7eff000a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7eff18101790 tx=0x7eff08009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:27.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.743+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7eff0c097420 con 0x7eff18101930 2023-10-18T09:18:27.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.962+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff1810bfb0 con 0x7eff18101930 2023-10-18T09:18:27.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.964+0000 7efefffff700 1 -- 172.21.15.61:0/3216334473 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7eff0c09aa40 con 0x7eff18101930 2023-10-18T09:18:27.966 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:27.967 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:27.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.966+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 msgr2=0x7eff000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.966+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 0x7eff000a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7eff18101790 tx=0x7eff08009310 comp rx=0 tx=0).stop 2023-10-18T09:18:27.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.966+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 msgr2=0x7eff18198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:27.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.966+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009810 tx=0x7eff0c00b560 comp rx=0 tx=0).stop 2023-10-18T09:18:27.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 shutdown_connections 2023-10-18T09:18:27.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7eff000a1480 0x7eff000a3940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7eff181022d0 0x7eff1819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff18100740 0x7eff18198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 --2- 172.21.15.61:0/3216334473 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7eff18101930 0x7eff18198e70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:27.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 >> 172.21.15.61:0/3216334473 conn(0x7eff180fbc90 msgr2=0x7eff18072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:27.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 shutdown_connections 2023-10-18T09:18:27.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:27.967+0000 7eff1d1ff700 1 -- 172.21.15.61:0/3216334473 wait complete. 2023-10-18T09:18:27.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:18:27.989 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:18:28.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 -- 172.21.15.61:0/2189726825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d81007a0 msgr2=0x7f69d8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 --2- 172.21.15.61:0/2189726825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d81007a0 0x7f69d8100bc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f69d40098f0 tx=0x7f69d4009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:28.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 -- 172.21.15.61:0/2189726825 shutdown_connections 2023-10-18T09:18:28.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 --2- 172.21.15.61:0/2189726825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d8102330 0x7f69d8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 --2- 172.21.15.61:0/2189726825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d81007a0 0x7f69d8100bc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 --2- 172.21.15.61:0/2189726825 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8101990 0x7f69d8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.195+0000 7f69e03ff700 1 -- 172.21.15.61:0/2189726825 >> 172.21.15.61:0/2189726825 conn(0x7f69d80fbcf0 msgr2=0x7f69d80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:28.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.196+0000 7f69e03ff700 1 -- 172.21.15.61:0/2189726825 shutdown_connections 2023-10-18T09:18:28.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.196+0000 7f69e03ff700 1 -- 172.21.15.61:0/2189726825 wait complete. 2023-10-18T09:18:28.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.197+0000 7f69e03ff700 1 Processor -- start 2023-10-18T09:18:28.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.197+0000 7f69e03ff700 1 -- start start 2023-10-18T09:18:28.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d81007a0 0x7f69d8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d8101990 0x7f69d8196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d819e2b0 con 0x7f69d8102330 2023-10-18T09:18:28.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d819e3f0 con 0x7f69d81007a0 2023-10-18T09:18:28.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69e03ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d819e580 con 0x7f69d8101990 2023-10-18T09:18:28.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37528/0 (socket says 172.21.15.61:37528) 2023-10-18T09:18:28.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de19b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d81007a0 0x7f69d8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 -- 172.21.15.61:0/1218007461 learned_addr learned my addr 172.21.15.61:0/1218007461 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:28.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69dd99a700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d8101990 0x7f69d8196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 -- 172.21.15.61:0/1218007461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d8101990 msgr2=0x7f69d8196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d8101990 0x7f69d8196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 -- 172.21.15.61:0/1218007461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d81007a0 msgr2=0x7f69d8196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d81007a0 0x7f69d8196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.198+0000 7f69de99c700 1 -- 172.21.15.61:0/1218007461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80095f0 con 0x7f69d8102330 2023-10-18T09:18:28.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69de99c700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f69d000ef30 tx=0x7f69d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:28.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d0018070 con 0x7f69d8102330 2023-10-18T09:18:28.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d000f040 con 0x7f69d8102330 2023-10-18T09:18:28.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d4009520 con 0x7f69d8102330 2023-10-18T09:18:28.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d000cd00 con 0x7f69d8102330 2023-10-18T09:18:28.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.199+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d819eb30 con 0x7f69d8102330 2023-10-18T09:18:28.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.200+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d8107c70 con 0x7f69d8102330 2023-10-18T09:18:28.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.202+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f69d0003a00 con 0x7f69d8102330 2023-10-18T09:18:28.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.203+0000 7f69cf7fe700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 0x7f69c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.203+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f69d00ccbd0 con 0x7f69d8102330 2023-10-18T09:18:28.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.204+0000 7f69de19b700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 0x7f69c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.204+0000 7f69de19b700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 0x7f69c40a3940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f69d40098c0 tx=0x7f69d4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:28.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.207+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f69d0095f10 con 0x7f69d8102330 2023-10-18T09:18:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:27 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1681758241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:27 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3034647331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:27 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1681758241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:27 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3034647331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:27 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1681758241' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:27 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3034647331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:28.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.421+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69d802cf10 con 0x7f69d8102330 2023-10-18T09:18:28.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.421+0000 7f69cf7fe700 1 -- 172.21.15.61:0/1218007461 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69d0099530 con 0x7f69d8102330 2023-10-18T09:18:28.424 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:18:28.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 msgr2=0x7f69c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 0x7f69c40a3940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f69d40098c0 tx=0x7f69d4019040 comp rx=0 tx=0).stop 2023-10-18T09:18:28.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 msgr2=0x7f69d819dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f69d000ef30 tx=0x7f69d000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 shutdown_connections 2023-10-18T09:18:28.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f69c40a1480 0x7f69c40a3940 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f69d8102330 0x7f69d819dce0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69d81007a0 0x7f69d8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 --2- 172.21.15.61:0/1218007461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69d8101990 0x7f69d8196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 >> 172.21.15.61:0/1218007461 conn(0x7f69d80fbcf0 msgr2=0x7f69d8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:28.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 shutdown_connections 2023-10-18T09:18:28.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.424+0000 7f69e03ff700 1 -- 172.21.15.61:0/1218007461 wait complete. 2023-10-18T09:18:28.445 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:18:28.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 -- 172.21.15.61:0/1447682110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 msgr2=0x7f5a14108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1447682110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14108be0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00b0d0 tx=0x7f5a0c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 -- 172.21.15.61:0/1447682110 shutdown_connections 2023-10-18T09:18:28.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1447682110 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14108be0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1447682110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a141007a0 0x7f5a14100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1447682110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a14101990 0x7f5a14101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 -- 172.21.15.61:0/1447682110 >> 172.21.15.61:0/1447682110 conn(0x7f5a140fbcf0 msgr2=0x7f5a140fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:28.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 -- 172.21.15.61:0/1447682110 shutdown_connections 2023-10-18T09:18:28.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.652+0000 7f5a197ff700 1 -- 172.21.15.61:0/1447682110 wait complete. 2023-10-18T09:18:28.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.653+0000 7f5a197ff700 1 Processor -- start 2023-10-18T09:18:28.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.653+0000 7f5a197ff700 1 -- start start 2023-10-18T09:18:28.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 0x7f5a14074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a14101990 0x7f5a14072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140738c0 con 0x7f5a14102330 2023-10-18T09:18:28.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14073a30 con 0x7f5a14101990 2023-10-18T09:18:28.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a197ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141a0890 con 0x7f5a141007a0 2023-10-18T09:18:28.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:37540/0 (socket says 172.21.15.61:37540) 2023-10-18T09:18:28.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 -- 172.21.15.61:0/1923543133 learned_addr learned my addr 172.21.15.61:0/1923543133 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:28.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a0bfff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a14101990 0x7f5a14072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a12ffd700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 0x7f5a14074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 -- 172.21.15.61:0/1923543133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 msgr2=0x7f5a14074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 0x7f5a14074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 -- 172.21.15.61:0/1923543133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a14101990 msgr2=0x7f5a14072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a14101990 0x7f5a14072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a137fe700 1 -- 172.21.15.61:0/1923543133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a04009520 con 0x7f5a14102330 2023-10-18T09:18:28.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.654+0000 7f5a12ffd700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 0x7f5a14074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:28.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a137fe700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c007b60 tx=0x7f5a0c008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:28.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00e040 con 0x7f5a14102330 2023-10-18T09:18:28.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a0c01d930 con 0x7f5a14102330 2023-10-18T09:18:28.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c01ba40 con 0x7f5a14102330 2023-10-18T09:18:28.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c009d00 con 0x7f5a14102330 2023-10-18T09:18:28.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.655+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a141a0d00 con 0x7f5a14102330 2023-10-18T09:18:28.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.656+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14107b70 con 0x7f5a14102330 2023-10-18T09:18:28.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.658+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5a0c01daa0 con 0x7f5a14102330 2023-10-18T09:18:28.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.659+0000 7f5a10ff9700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 0x7f5a000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:28.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.659+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5a0c0ce570 con 0x7f5a14102330 2023-10-18T09:18:28.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.660+0000 7f5a12ffd700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 0x7f5a000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:28.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.660+0000 7f5a12ffd700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 0x7f5a000a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a141017f0 tx=0x7f5a04009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:28.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.663+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5a0c0978b0 con 0x7f5a14102330 2023-10-18T09:18:28.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.878+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a1410c010 con 0x7f5a14102330 2023-10-18T09:18:28.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.879+0000 7f5a10ff9700 1 -- 172.21.15.61:0/1923543133 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a0c09aed0 con 0x7f5a14102330 2023-10-18T09:18:28.882 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:28.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.881+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 msgr2=0x7f5a000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.881+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 0x7f5a000a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a141017f0 tx=0x7f5a04009260 comp rx=0 tx=0).stop 2023-10-18T09:18:28.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.881+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 msgr2=0x7f5a14073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:28.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.881+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c007b60 tx=0x7f5a0c008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:28.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 shutdown_connections 2023-10-18T09:18:28.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5a000a1480 0x7f5a000a3940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5a14102330 0x7f5a14073120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a141007a0 0x7f5a14074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 --2- 172.21.15.61:0/1923543133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a14101990 0x7f5a14072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:28.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 >> 172.21.15.61:0/1923543133 conn(0x7f5a140fbcf0 msgr2=0x7f5a14104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:28.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 shutdown_connections 2023-10-18T09:18:28.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:28.882+0000 7f5a197ff700 1 -- 172.21.15.61:0/1923543133 wait complete. 2023-10-18T09:18:28.903 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 -- 172.21.15.61:0/2796955835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e081007a0 msgr2=0x7f0e08100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/2796955835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e081007a0 0x7f0e08100bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0df80099c0 tx=0x7f0df8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 -- 172.21.15.61:0/2796955835 shutdown_connections 2023-10-18T09:18:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/2796955835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e08102330 0x7f0e08108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/2796955835 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e081007a0 0x7f0e08100bc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/2796955835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101990 0x7f0e08101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.110+0000 7f0e0cfff700 1 -- 172.21.15.61:0/2796955835 >> 172.21.15.61:0/2796955835 conn(0x7f0e080fbcf0 msgr2=0x7f0e080fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.111+0000 7f0e0cfff700 1 -- 172.21.15.61:0/2796955835 shutdown_connections 2023-10-18T09:18:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.111+0000 7f0e0cfff700 1 -- 172.21.15.61:0/2796955835 wait complete. 2023-10-18T09:18:29.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.111+0000 7f0e0cfff700 1 Processor -- start 2023-10-18T09:18:29.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 -- start start 2023-10-18T09:18:29.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 0x7f0e08196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 0x7f0e0819dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0819e280 con 0x7f0e08101990 2023-10-18T09:18:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0819e3c0 con 0x7f0e08102330 2023-10-18T09:18:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0cfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0819e550 con 0x7f0e081007a0 2023-10-18T09:18:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e05d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e05d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57458/0 (socket says 172.21.15.61:57458) 2023-10-18T09:18:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e05d9b700 1 -- 172.21.15.61:0/1248151735 learned_addr learned my addr 172.21.15.61:0/1248151735 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e0659c700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 0x7f0e08196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.112+0000 7f0e06d9d700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 0x7f0e0819dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 -- 172.21.15.61:0/1248151735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 msgr2=0x7f0e08196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 0x7f0e08196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 -- 172.21.15.61:0/1248151735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 msgr2=0x7f0e0819dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 0x7f0e0819dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 -- 172.21.15.61:0/1248151735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df80095f0 con 0x7f0e08101990 2023-10-18T09:18:29.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e0659c700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 0x7f0e08196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:29.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e06d9d700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 0x7f0e0819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:29.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e05d9b700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00e9a0 tx=0x7f0dfc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:29.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc009790 con 0x7f0e08101990 2023-10-18T09:18:29.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dfc00cbd0 con 0x7f0e08101990 2023-10-18T09:18:29.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc018680 con 0x7f0e08101990 2023-10-18T09:18:29.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0819e830 con 0x7f0e08101990 2023-10-18T09:18:29.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.113+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0819ecf0 con 0x7f0e08101990 2023-10-18T09:18:29.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.114+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08107bc0 con 0x7f0e08101990 2023-10-18T09:18:29.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.116+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0dfc00cd40 con 0x7f0e08101990 2023-10-18T09:18:29.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.118+0000 7f0df77fe700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 0x7f0df00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.118+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0dfc014070 con 0x7f0e08101990 2023-10-18T09:18:29.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.118+0000 7f0e0659c700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 0x7f0df00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.119+0000 7f0e0659c700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 0x7f0df00a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009990 tx=0x7f0df801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:29.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.121+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0dfc096680 con 0x7f0e08101990 2023-10-18T09:18:29.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3216334473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:29.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1218007461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:28 smithi110 ceph-mon[134818]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-10-18T09:18:29.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1923543133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3216334473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1218007461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:28 smithi061 ceph-mon[98573]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-10-18T09:18:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1923543133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.339+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e0802cf10 con 0x7f0e08101990 2023-10-18T09:18:29.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.339+0000 7f0df77fe700 1 -- 172.21.15.61:0/1248151735 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0dfc099ca0 con 0x7f0e08101990 2023-10-18T09:18:29.342 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 msgr2=0x7f0df00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 0x7f0df00a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009990 tx=0x7f0df801a040 comp rx=0 tx=0).stop 2023-10-18T09:18:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 msgr2=0x7f0e08196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00e9a0 tx=0x7f0dfc00ed60 comp rx=0 tx=0).stop 2023-10-18T09:18:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 shutdown_connections 2023-10-18T09:18:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0df00a1480 0x7f0df00a3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08102330 0x7f0e0819dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e081007a0 0x7f0e08196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 --2- 172.21.15.61:0/1248151735 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e08101990 0x7f0e08196c90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 >> 172.21.15.61:0/1248151735 conn(0x7f0e080fbcf0 msgr2=0x7f0e08104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:29.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 shutdown_connections 2023-10-18T09:18:29.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.342+0000 7f0e0cfff700 1 -- 172.21.15.61:0/1248151735 wait complete. 2023-10-18T09:18:29.363 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3216334473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1218007461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:28 smithi170 ceph-mon[135150]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 895 B/s wr, 11 op/s 2023-10-18T09:18:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1923543133' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 -- 172.21.15.61:0/3610841322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 msgr2=0x7f17d0108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 --2- 172.21.15.61:0/3610841322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0108be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f17c800b0d0 tx=0x7f17c800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 -- 172.21.15.61:0/3610841322 shutdown_connections 2023-10-18T09:18:29.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 --2- 172.21.15.61:0/3610841322 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0108be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 --2- 172.21.15.61:0/3610841322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d01007a0 0x7f17d0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 --2- 172.21.15.61:0/3610841322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d0101990 0x7f17d0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 -- 172.21.15.61:0/3610841322 >> 172.21.15.61:0/3610841322 conn(0x7f17d00fbcf0 msgr2=0x7f17d00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:29.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 -- 172.21.15.61:0/3610841322 shutdown_connections 2023-10-18T09:18:29.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.570+0000 7f17d71ff700 1 -- 172.21.15.61:0/3610841322 wait complete. 2023-10-18T09:18:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 Processor -- start 2023-10-18T09:18:29.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 -- start start 2023-10-18T09:18:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d01007a0 0x7f17d0074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d0101990 0x7f17d0072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d019a6d0 con 0x7f17d0102330 2023-10-18T09:18:29.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0073660 con 0x7f17d01007a0 2023-10-18T09:18:29.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.571+0000 7f17d71ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d0073820 con 0x7f17d0101990 2023-10-18T09:18:29.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57466/0 (socket says 172.21.15.61:57466) 2023-10-18T09:18:29.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 -- 172.21.15.61:0/511388355 learned_addr learned my addr 172.21.15.61:0/511388355 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:29.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d4f9b700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d01007a0 0x7f17d0074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17cffff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d0101990 0x7f17d0072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 -- 172.21.15.61:0/511388355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d0101990 msgr2=0x7f17d0072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d0101990 0x7f17d0072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 -- 172.21.15.61:0/511388355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d01007a0 msgr2=0x7f17d0074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d01007a0 0x7f17d0074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 -- 172.21.15.61:0/511388355 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009d00 con 0x7f17d0102330 2023-10-18T09:18:29.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17d579c700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17c8007b60 tx=0x7f17c8008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:29.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.572+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800e040 con 0x7f17d0102330 2023-10-18T09:18:29.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.573+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c801d930 con 0x7f17d0102330 2023-10-18T09:18:29.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.573+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d0073a40 con 0x7f17d0102330 2023-10-18T09:18:29.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.573+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c801b9f0 con 0x7f17d0102330 2023-10-18T09:18:29.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.573+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d01a5370 con 0x7f17d0102330 2023-10-18T09:18:29.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.574+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d0107b70 con 0x7f17d0102330 2023-10-18T09:18:29.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.576+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f17c801daa0 con 0x7f17d0102330 2023-10-18T09:18:29.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.577+0000 7f17cdffb700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 0x7f17c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:29.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.577+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f17c80ce510 con 0x7f17d0102330 2023-10-18T09:18:29.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.577+0000 7f17d4f9b700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 0x7f17c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:29.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.578+0000 7f17d4f9b700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 0x7f17c00a3940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f17d01017f0 tx=0x7f17c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:29.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.581+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f17c8097850 con 0x7f17d0102330 2023-10-18T09:18:29.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.794+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f17d0107550 con 0x7f17d0102330 2023-10-18T09:18:29.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.794+0000 7f17cdffb700 1 -- 172.21.15.61:0/511388355 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f17c809ae70 con 0x7f17d0102330 2023-10-18T09:18:29.797 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:29.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.796+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 msgr2=0x7f17c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.796+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 0x7f17c00a3940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f17d01017f0 tx=0x7f17c400b410 comp rx=0 tx=0).stop 2023-10-18T09:18:29.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 msgr2=0x7f17d0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:29.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f17c8007b60 tx=0x7f17c8008e90 comp rx=0 tx=0).stop 2023-10-18T09:18:29.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 shutdown_connections 2023-10-18T09:18:29.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f17c00a1480 0x7f17c00a3940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f17d0102330 0x7f17d0073120 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f17d01007a0 0x7f17d0074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 --2- 172.21.15.61:0/511388355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17d0101990 0x7f17d0072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:29.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 >> 172.21.15.61:0/511388355 conn(0x7f17d00fbcf0 msgr2=0x7f17d0104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:29.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 shutdown_connections 2023-10-18T09:18:29.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:29.797+0000 7f17d71ff700 1 -- 172.21.15.61:0/511388355 wait complete. 2023-10-18T09:18:29.819 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:30.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 -- 172.21.15.61:0/1918610165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 msgr2=0x7f0f34108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 --2- 172.21.15.61:0/1918610165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f34108b80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00b0d0 tx=0x7f0f2c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 -- 172.21.15.61:0/1918610165 shutdown_connections 2023-10-18T09:18:30.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 --2- 172.21.15.61:0/1918610165 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f34108b80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 --2- 172.21.15.61:0/1918610165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34100740 0x7f0f34100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 --2- 172.21.15.61:0/1918610165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34101930 0x7f0f34101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 -- 172.21.15.61:0/1918610165 >> 172.21.15.61:0/1918610165 conn(0x7f0f340fbc90 msgr2=0x7f0f340fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:30.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 -- 172.21.15.61:0/1918610165 shutdown_connections 2023-10-18T09:18:30.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.024+0000 7f0f3abff700 1 -- 172.21.15.61:0/1918610165 wait complete. 2023-10-18T09:18:30.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.025+0000 7f0f3abff700 1 Processor -- start 2023-10-18T09:18:30.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 -- start start 2023-10-18T09:18:30.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34100740 0x7f0f341989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34101930 0x7f0f34198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f341a0510 con 0x7f0f341022d0 2023-10-18T09:18:30.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f341a0650 con 0x7f0f34100740 2023-10-18T09:18:30.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3abff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f341a07e0 con 0x7f0f34101930 2023-10-18T09:18:30.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3919c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3919c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57478/0 (socket says 172.21.15.61:57478) 2023-10-18T09:18:30.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3919c700 1 -- 172.21.15.61:0/147363936 learned_addr learned my addr 172.21.15.61:0/147363936 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:30.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f33fff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34101930 0x7f0f34198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.026+0000 7f0f3899b700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34100740 0x7f0f341989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 -- 172.21.15.61:0/147363936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34101930 msgr2=0x7f0f34198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34101930 0x7f0f34198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 -- 172.21.15.61:0/147363936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34100740 msgr2=0x7f0f341989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34100740 0x7f0f341989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 -- 172.21.15.61:0/147363936 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c009d00 con 0x7f0f341022d0 2023-10-18T09:18:30.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3919c700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c007b60 tx=0x7f0f2c008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:30.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c00e040 con 0x7f0f341022d0 2023-10-18T09:18:30.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f2c01d930 con 0x7f0f341022d0 2023-10-18T09:18:30.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f341a0a60 con 0x7f0f341022d0 2023-10-18T09:18:30.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01b9f0 con 0x7f0f341022d0 2023-10-18T09:18:30.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.027+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f341a0f30 con 0x7f0f341022d0 2023-10-18T09:18:30.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.028+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f34067530 con 0x7f0f341022d0 2023-10-18T09:18:30.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.030+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0f2c01daa0 con 0x7f0f341022d0 2023-10-18T09:18:30.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.032+0000 7f0f31ffb700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 0x7f0f240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.032+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0f2c0ce390 con 0x7f0f341022d0 2023-10-18T09:18:30.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.032+0000 7f0f3899b700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 0x7f0f240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.033+0000 7f0f3899b700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 0x7f0f240a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f34101790 tx=0x7f0f28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:30.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.036+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f2c097730 con 0x7f0f341022d0 2023-10-18T09:18:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:29 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1248151735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:29 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/511388355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:29 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1248151735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:29 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/511388355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.294+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f3410bfb0 con 0x7f0f341022d0 2023-10-18T09:18:30.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.295+0000 7f0f31ffb700 1 -- 172.21.15.61:0/147363936 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f0f2c09ad50 con 0x7f0f341022d0 2023-10-18T09:18:30.297 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:30.298 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:30.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 msgr2=0x7f0f240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 0x7f0f240a39c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f34101790 tx=0x7f0f28009260 comp rx=0 tx=0).stop 2023-10-18T09:18:30.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 msgr2=0x7f0f3419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c007b60 tx=0x7f0f2c008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:30.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 shutdown_connections 2023-10-18T09:18:30.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0f240a1500 0x7f0f240a39c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0f341022d0 0x7f0f3419ff40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f34100740 0x7f0f341989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 --2- 172.21.15.61:0/147363936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f34101930 0x7f0f34198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.297+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 >> 172.21.15.61:0/147363936 conn(0x7f0f340fbc90 msgr2=0x7f0f34072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:30.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.298+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 shutdown_connections 2023-10-18T09:18:30.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.298+0000 7f0f3abff700 1 -- 172.21.15.61:0/147363936 wait complete. 2023-10-18T09:18:30.303 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:30.320 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:30.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:29 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1248151735' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:29 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/511388355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:30.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 -- 172.21.15.61:0/999008519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 msgr2=0x7f97e0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 --2- 172.21.15.61:0/999008519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0100b60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009990 tx=0x7f97dc009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 -- 172.21.15.61:0/999008519 shutdown_connections 2023-10-18T09:18:30.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 --2- 172.21.15.61:0/999008519 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 --2- 172.21.15.61:0/999008519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0100b60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 --2- 172.21.15.61:0/999008519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 0x7f97e0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 -- 172.21.15.61:0/999008519 >> 172.21.15.61:0/999008519 conn(0x7f97e00fbc90 msgr2=0x7f97e00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:30.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 -- 172.21.15.61:0/999008519 shutdown_connections 2023-10-18T09:18:30.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.521+0000 7f97e75ff700 1 -- 172.21.15.61:0/999008519 wait complete. 2023-10-18T09:18:30.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.522+0000 7f97e75ff700 1 Processor -- start 2023-10-18T09:18:30.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 -- start start 2023-10-18T09:18:30.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 0x7f97e0072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0073820 con 0x7f97e01022d0 2023-10-18T09:18:30.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0073990 con 0x7f97e0101930 2023-10-18T09:18:30.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e75ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e01a07d0 con 0x7f97e0100740 2023-10-18T09:18:30.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e5b9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e5b9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57496/0 (socket says 172.21.15.61:57496) 2023-10-18T09:18:30.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e5b9c700 1 -- 172.21.15.61:0/780325410 learned_addr learned my addr 172.21.15.61:0/780325410 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:30.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e4b9a700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 0x7f97e0072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.523+0000 7f97e539b700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 -- 172.21.15.61:0/780325410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 msgr2=0x7f97e0074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 -- 172.21.15.61:0/780325410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 msgr2=0x7f97e0072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 0x7f97e0072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 -- 172.21.15.61:0/780325410 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc0095f0 con 0x7f97e01022d0 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e5b9c700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97d800ef30 tx=0x7f97d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8018070 con 0x7f97e01022d0 2023-10-18T09:18:30.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e01a09d0 con 0x7f97e01022d0 2023-10-18T09:18:30.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d800f040 con 0x7f97e01022d0 2023-10-18T09:18:30.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800cd00 con 0x7f97e01022d0 2023-10-18T09:18:30.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.524+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e01a0e90 con 0x7f97e01022d0 2023-10-18T09:18:30.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.525+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e0107c80 con 0x7f97e01022d0 2023-10-18T09:18:30.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.527+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f97d8003a00 con 0x7f97e01022d0 2023-10-18T09:18:30.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.529+0000 7f97d67fc700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 0x7f97cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.529+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f97d80ccc20 con 0x7f97e01022d0 2023-10-18T09:18:30.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.529+0000 7f97e539b700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 0x7f97cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.529+0000 7f97e539b700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 0x7f97cc0a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b680 tx=0x7f97dc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:30.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.532+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f97d8095f60 con 0x7f97e01022d0 2023-10-18T09:18:30.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.746+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97e0107660 con 0x7f97e01022d0 2023-10-18T09:18:30.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.748+0000 7f97d67fc700 1 -- 172.21.15.61:0/780325410 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f97d8099580 con 0x7f97e01022d0 2023-10-18T09:18:30.750 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:30.751 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:30.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 msgr2=0x7f97cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 0x7f97cc0a3940 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00b680 tx=0x7f97dc005fd0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 msgr2=0x7f97e0073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f97d800ef30 tx=0x7f97d800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 shutdown_connections 2023-10-18T09:18:30.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f97cc0a1480 0x7f97cc0a3940 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f97e01022d0 0x7f97e0073120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97e0100740 0x7f97e0074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 --2- 172.21.15.61:0/780325410 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97e0101930 0x7f97e0072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 >> 172.21.15.61:0/780325410 conn(0x7f97e00fbc90 msgr2=0x7f97e0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:30.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.750+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 shutdown_connections 2023-10-18T09:18:30.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.751+0000 7f97e75ff700 1 -- 172.21.15.61:0/780325410 wait complete. 2023-10-18T09:18:30.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-18T09:16:26.396655+0000', 'modified': '2023-10-18T09:16:59.542245+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24692, 'mds_1': 24676, 'mds_2': 24653, 'mds_3': 24646, 'mds_4': 24629}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14688': {'gid': 14688, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.61:6841/2114349313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6840', 'nonce': 2114349313}, {'type': 'v1', 'addr': '172.21.15.61:6841', 'nonce': 2114349313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24589': {'gid': 24589, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6833/2686391817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6832', 'nonce': 2686391817}, {'type': 'v1', 'addr': '172.21.15.170:6833', 'nonce': 2686391817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6833/3590204423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6832', 'nonce': 3590204423}, {'type': 'v1', 'addr': '172.21.15.110:6833', 'nonce': 3590204423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24629': {'gid': 24629, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.110:6835/145516333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 145516333}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 145516333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24646': {'gid': 24646, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.170:6835/1203136193', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 1203136193}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 1203136193}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24653': {'gid': 24653, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.110:6837/2750873982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2750873982}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2750873982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24658': {'gid': 24658, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6837/3025282224', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 3025282224}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3025282224}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24668': {'gid': 24668, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.61:6839/2550944546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6838', 'nonce': 2550944546}, {'type': 'v1', 'addr': '172.21.15.61:6839', 'nonce': 2550944546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24676': {'gid': 24676, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.170:6839/2453939791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 2453939791}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 2453939791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24692': {'gid': 24692, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.110:6839/1483417844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 1483417844}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 1483417844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-18T09:18:30.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-10-18T09:18:30.773 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-10-18T09:18:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 -- 172.21.15.61:0/677836672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 msgr2=0x7fc2e8100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 --2- 172.21.15.61:0/677836672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e8100b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0099c0 tx=0x7fc2dc009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 -- 172.21.15.61:0/677836672 shutdown_connections 2023-10-18T09:18:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 --2- 172.21.15.61:0/677836672 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e8108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 --2- 172.21.15.61:0/677836672 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e8100b40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 --2- 172.21.15.61:0/677836672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 0x7fc2e8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 -- 172.21.15.61:0/677836672 >> 172.21.15.61:0/677836672 conn(0x7fc2e80fbc90 msgr2=0x7fc2e80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:30.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 -- 172.21.15.61:0/677836672 shutdown_connections 2023-10-18T09:18:30.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.982+0000 7fc2eedff700 1 -- 172.21.15.61:0/677836672 wait complete. 2023-10-18T09:18:30.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.983+0000 7fc2eedff700 1 Processor -- start 2023-10-18T09:18:30.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 -- start start 2023-10-18T09:18:30.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 0x7fc2e8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:30.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81a03f0 con 0x7fc2e8100720 2023-10-18T09:18:30.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81a0530 con 0x7fc2e81022b0 2023-10-18T09:18:30.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2eedff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81a06c0 con 0x7fc2e8101910 2023-10-18T09:18:30.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57512/0 (socket says 172.21.15.61:57512) 2023-10-18T09:18:30.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1354011023 learned_addr learned my addr 172.21.15.61:0/1354011023 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:30.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 0x7fc2e8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.984+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:30.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1354011023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 msgr2=0x7fc2e8198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 0x7fc2e8198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1354011023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 msgr2=0x7fc2e819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:30.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:30.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1354011023 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0095f0 con 0x7fc2e8100720 2023-10-18T09:18:30.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:30.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006010 tx=0x7fc2dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:30.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2e5ffb700 1 -- 172.21.15.61:0/1354011023 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc01d070 con 0x7fc2e8100720 2023-10-18T09:18:30.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2e5ffb700 1 -- 172.21.15.61:0/1354011023 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc005020 con 0x7fc2e8100720 2023-10-18T09:18:30.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e81a0940 con 0x7fc2e8100720 2023-10-18T09:18:30.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2e5ffb700 1 -- 172.21.15.61:0/1354011023 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc004bb0 con 0x7fc2e8100720 2023-10-18T09:18:30.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.985+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e81a0d00 con 0x7fc2e8100720 2023-10-18T09:18:31.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.988+0000 7fc2e5ffb700 1 -- 172.21.15.61:0/1354011023 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc2dc005190 con 0x7fc2e8100720 2023-10-18T09:18:31.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 Processor -- start 2023-10-18T09:18:31.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2e5ffb700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 0x7fc2d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 -- start start 2023-10-18T09:18:31.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2e5ffb700 1 -- 172.21.15.61:0/1354011023 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc2dc0cdb10 con 0x7fc2e8100720 2023-10-18T09:18:31.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 0x7fc2d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904ce580 0x5627904cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562790441690 con 0x5627904cf280 2023-10-18T09:18:31.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562790427c10 con 0x5627904cfbe0 2023-10-18T09:18:31.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc302776b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562790425530 con 0x5627904ce580 2023-10-18T09:18:31.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57526/0 (socket says 172.21.15.61:57526) 2023-10-18T09:18:31.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.990+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1558456354 learned_addr learned my addr 172.21.15.61:0/1558456354 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904ce580 0x5627904cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1558456354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904ce580 msgr2=0x5627904cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904ce580 0x5627904cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1558456354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 msgr2=0x5627904ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/1558456354 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627903de3a0 con 0x5627904cf280 2023-10-18T09:18:31.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 0x7fc2d80a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00093c0 tx=0x7fc2e0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0a28e0 tx=0x7fc2dc09a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b82018c9a60080a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2d2ffd700 1 -- 172.21.15.61:0/1558456354 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09ec60 con 0x5627904cf280 2023-10-18T09:18:31.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2d2ffd700 1 -- 172.21.15.61:0/1558456354 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc085040 con 0x5627904cf280 2023-10-18T09:18:31.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.991+0000 7fc2d2ffd700 1 -- 172.21.15.61:0/1558456354 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc09afd0 con 0x5627904cf280 2023-10-18T09:18:31.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 -- 172.21.15.61:0/1558456354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 msgr2=0x5627904cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0a28e0 tx=0x7fc2dc09a780 comp rx=0 tx=0).stop 2023-10-18T09:18:31.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 -- 172.21.15.61:0/1558456354 shutdown_connections 2023-10-18T09:18:31.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904ce580 0x5627904cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904cf280 0x5627904cf6a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 --2- 172.21.15.61:0/1558456354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 -- 172.21.15.61:0/1558456354 >> 172.21.15.61:0/1558456354 conn(0x5627904426d0 msgr2=0x5627904c8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 -- 172.21.15.61:0/1558456354 shutdown_connections 2023-10-18T09:18:31.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.992+0000 7fc302776b80 1 -- 172.21.15.61:0/1558456354 wait complete. 2023-10-18T09:18:31.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 -- start start 2023-10-18T09:18:31.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 Processor -- start 2023-10-18T09:18:31.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cf280 0x5627904d7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cfbe0 0x5627904dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562790420e20 con 0x5627904ce580 2023-10-18T09:18:31.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627903bd6c0 con 0x5627904cf280 2023-10-18T09:18:31.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.993+0000 7fc302776b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627903bd360 con 0x5627904cfbe0 2023-10-18T09:18:31.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc302776b80 10 client.0 _ll_register_callbacks cb 0x562790307a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57538/0 (socket says 172.21.15.61:57538) 2023-10-18T09:18:31.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/2484610510 learned_addr learned my addr 172.21.15.61:0/2484610510 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cf280 0x5627904d7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cfbe0 0x5627904dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/2484610510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cfbe0 msgr2=0x5627904dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cfbe0 0x5627904dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cf280 msgr2=0x5627904d7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cf280 0x5627904d7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/2484610510 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e81a0d00 con 0x5627904ce580 2023-10-18T09:18:31.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc003730 tx=0x7fc2dc096e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbb7a56dbcca6d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.994+0000 7fc2c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:31.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0b3370 con 0x5627904ce580 2023-10-18T09:18:31.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc2dc09be40 con 0x5627904ce580 2023-10-18T09:18:31.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562790425ea0 con 0x5627904ce580 2023-10-18T09:18:31.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2dc09ec60 con 0x5627904ce580 2023-10-18T09:18:31.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 10 client.15483 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56279024bd90 con 0x5627904ce580 2023-10-18T09:18:31.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc07d080 con 0x5627904ce580 2023-10-18T09:18:31.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2dc066170 con 0x5627904ce580 2023-10-18T09:18:31.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 10 client.15483 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-18T09:18:31.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 20 client.15483 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x5627904e2e00 0x5627904e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 4 client.15483 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:31.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.995+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56279049d860 con 0x5627904e2e00 2023-10-18T09:18:31.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.996+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x5627904e2e00 0x5627904e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.996+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x5627904e2e00 0x5627904e1c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e99ef443defa8c1 server_cookie=41785ab80337aaf1 in_seq=0 out_seq=0 2023-10-18T09:18:31.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.996+0000 7fc2c7fff700 10 client.15483 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:31.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc2c7fff700 1 -- 172.21.15.61:0/2484610510 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56279049d860 con 0x5627904e2e00 2023-10-18T09:18:31.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc2c7fff700 10 client.15483 handle_command_reply: tid=0 2023-10-18T09:18:31.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 client.15483 shutdown 2023-10-18T09:18:31.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 msgr2=0x5627904d7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc003730 tx=0x7fc2dc096e50 comp rx=0 tx=0).stop 2023-10-18T09:18:31.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 shutdown_connections 2023-10-18T09:18:31.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x5627904e2e00 0x5627904e1c50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cfbe0 0x5627904dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc2c7fff700 0 client.15483 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:31.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d7700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 --2- 172.21.15.61:0/2484610510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cf280 0x5627904d7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 >> 172.21.15.61:0/2484610510 conn(0x5627904426d0 msgr2=0x5627904cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.997+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 shutdown_connections 2023-10-18T09:18:31.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.998+0000 7fc302776b80 1 -- 172.21.15.61:0/2484610510 wait complete. 2023-10-18T09:18:31.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:30.998+0000 7fc302776b80 20 client.15483 trim_cache size 0 max 16384 2023-10-18T09:18:31.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.016+0000 7fc302776b80 1 Processor -- start 2023-10-18T09:18:31.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.016+0000 7fc302776b80 1 -- start start 2023-10-18T09:18:31.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627902a18d0 con 0x5627904ce580 2023-10-18T09:18:31.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627903bd6c0 con 0x5627904cfbe0 2023-10-18T09:18:31.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc302776b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627904df150 con 0x5627904cf280 2023-10-18T09:18:31.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57542/0 (socket says 172.21.15.61:57542) 2023-10-18T09:18:31.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/4113063709 learned_addr learned my addr 172.21.15.61:0/4113063709 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/4113063709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 msgr2=0x5627904dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904dec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/4113063709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 msgr2=0x5627904d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.017+0000 7fc2ecb9b700 1 -- 172.21.15.61:0/4113063709 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562790234c10 con 0x5627904ce580 2023-10-18T09:18:31.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc098e60 tx=0x7fc2dc0990c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7caaf13882db6bf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc2d37fe700 1 -- 172.21.15.61:0/4113063709 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0ba160 con 0x5627904ce580 2023-10-18T09:18:31.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc2d37fe700 1 -- 172.21.15.61:0/4113063709 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562790425ea0 con 0x5627904ce580 2023-10-18T09:18:31.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc2d37fe700 1 -- 172.21.15.61:0/4113063709 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0a2980 con 0x5627904ce580 2023-10-18T09:18:31.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 -- 172.21.15.61:0/4113063709 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 msgr2=0x5627904d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc098e60 tx=0x7fc2dc0990c0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 -- 172.21.15.61:0/4113063709 shutdown_connections 2023-10-18T09:18:31.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904d2990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 --2- 172.21.15.61:0/4113063709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.018+0000 7fc302776b80 1 -- 172.21.15.61:0/4113063709 >> 172.21.15.61:0/4113063709 conn(0x5627904426d0 msgr2=0x5627904d2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 -- 172.21.15.61:0/4113063709 shutdown_connections 2023-10-18T09:18:31.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 -- 172.21.15.61:0/4113063709 wait complete. 2023-10-18T09:18:31.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 -- start start 2023-10-18T09:18:31.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 Processor -- start 2023-10-18T09:18:31.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.019+0000 7fc302776b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc302776b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc302776b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627904dc1e0 con 0x5627904ce580 2023-10-18T09:18:31.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc302776b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627904dc320 con 0x5627904cfbe0 2023-10-18T09:18:31.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc302776b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627905367f0 con 0x5627904cf280 2023-10-18T09:18:31.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc302776b80 10 client.0 _ll_register_callbacks cb 0x562790307a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ecb9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43506/0 (socket says 172.21.15.61:43506) 2023-10-18T09:18:31.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 -- 172.21.15.61:0/3727999077 learned_addr learned my addr 172.21.15.61:0/3727999077 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2e7fff700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 -- 172.21.15.61:0/3727999077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 msgr2=0x5627904db680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904db680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 -- 172.21.15.61:0/3727999077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 msgr2=0x5627904de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.020+0000 7fc2ed39c700 1 -- 172.21.15.61:0/3727999077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56279024bd90 con 0x5627904cfbe0 2023-10-18T09:18:31.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2ed39c700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0017b90 tx=0x7fc2e0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3598c6052b7ec85e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:31.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000e050 con 0x5627904cfbe0 2023-10-18T09:18:31.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2dc085040 con 0x5627904cfbe0 2023-10-18T09:18:31.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc2e0018a20 con 0x5627904cfbe0 2023-10-18T09:18:31.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2e0018be0 con 0x5627904cfbe0 2023-10-18T09:18:31.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc302776b80 10 client.25472 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x562790272200 con 0x5627904cfbe0 2023-10-18T09:18:31.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2dc085040 con 0x5627904cfbe0 2023-10-18T09:18:31.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc2e002b020 con 0x5627904cfbe0 2023-10-18T09:18:31.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.021+0000 7fc302776b80 10 client.25472 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc302776b80 10 client.25472 resolve_mds: resolved name 'e' to daemon mds.e 2023-10-18T09:18:31.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc302776b80 20 client.25472 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x56279053bc00 0x562790538040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc302776b80 4 client.25472 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:31.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562790419e00 con 0x56279053bc00 2023-10-18T09:18:31.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x56279053bc00 0x562790538040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc2ecb9b700 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x56279053bc00 0x562790538040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a47f59283771f37c server_cookie=e8681628f1712361 in_seq=0 out_seq=0 2023-10-18T09:18:31.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc2c7fff700 10 client.25472 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:31.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc2c7fff700 1 -- 172.21.15.61:0/3727999077 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562790419e00 con 0x56279053bc00 2023-10-18T09:18:31.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.022+0000 7fc2c7fff700 10 client.25472 handle_command_reply: tid=0 2023-10-18T09:18:31.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 client.25472 shutdown 2023-10-18T09:18:31.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 msgr2=0x5627904dbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0017b90 tx=0x7fc2e0004b40 comp rx=0 tx=0).stop 2023-10-18T09:18:31.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 shutdown_connections 2023-10-18T09:18:31.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x56279053bc00 0x562790538040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5627904cfbe0 0x5627904dbbc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5627904ce580 0x5627904de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc2c7fff700 0 client.25472 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:18:31.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 --2- 172.21.15.61:0/3727999077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5627904cf280 0x5627904db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 >> 172.21.15.61:0/3727999077 conn(0x5627904426d0 msgr2=0x5627904d74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 shutdown_connections 2023-10-18T09:18:31.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 1 -- 172.21.15.61:0/3727999077 wait complete. 2023-10-18T09:18:31.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.023+0000 7fc302776b80 20 client.25472 trim_cache size 0 max 16384 2023-10-18T09:18:31.049 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:31.049 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:31.049 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 4, 2023-10-18T09:18:31.050 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24629, 2023-10-18T09:18:31.050 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:31.050 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:31.050 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 92.498692038000002, 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 92, 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 148.36678925499999 2023-10-18T09:18:31.051 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:31.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 msgr2=0x7fc2d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 0x7fc2d80a3940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00093c0 tx=0x7fc2e0009920 comp rx=0 tx=0).stop 2023-10-18T09:18:31.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 msgr2=0x7fc2e81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc006010 tx=0x7fc2dc00f690 comp rx=0 tx=0).stop 2023-10-18T09:18:31.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 shutdown_connections 2023-10-18T09:18:31.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc2d80a1480 0x7fc2d80a3940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2e81022b0 0x7fc2e819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc2e8100720 0x7fc2e81988c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 --2- 172.21.15.61:0/1354011023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2e8101910 0x7fc2e8198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 >> 172.21.15.61:0/1354011023 conn(0x7fc2e80fbc90 msgr2=0x7fc2e8107100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.054 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 shutdown_connections 2023-10-18T09:18:31.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.024+0000 7fc2eedff700 1 -- 172.21.15.61:0/1354011023 wait complete. 2023-10-18T09:18:31.055 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/147363936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.055 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:30 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/780325410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.055 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:30 smithi061 ceph-mon[98573]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 229 B/s wr, 8 op/s 2023-10-18T09:18:31.056 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-10-18T09:18:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:30 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/147363936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:30 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/780325410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:30 smithi110 ceph-mon[134818]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 229 B/s wr, 8 op/s 2023-10-18T09:18:31.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 -- 172.21.15.61:0/3948690484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 msgr2=0x7f27d8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 --2- 172.21.15.61:0/3948690484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d8072c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009990 tx=0x7f27d4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 -- 172.21.15.61:0/3948690484 shutdown_connections 2023-10-18T09:18:31.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 --2- 172.21.15.61:0/3948690484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d810ccf0 0x7f27d810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 --2- 172.21.15.61:0/3948690484 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d8072c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 --2- 172.21.15.61:0/3948690484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d8073250 0x7f27d80736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 -- 172.21.15.61:0/3948690484 >> 172.21.15.61:0/3948690484 conn(0x7f27d80fbc90 msgr2=0x7f27d80fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 -- 172.21.15.61:0/3948690484 shutdown_connections 2023-10-18T09:18:31.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.260+0000 7f27e07ff700 1 -- 172.21.15.61:0/3948690484 wait complete. 2023-10-18T09:18:31.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 Processor -- start 2023-10-18T09:18:31.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 -- start start 2023-10-18T09:18:31.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 0x7f27d819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d819d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810ccf0 0x7f27d81a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81a4790 con 0x7f27d8074830 2023-10-18T09:18:31.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81a4900 con 0x7f27d810ccf0 2023-10-18T09:18:31.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.261+0000 7f27e07ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d81a4ac0 con 0x7f27d8073250 2023-10-18T09:18:31.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 0x7f27d819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d819d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 0x7f27d819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:55622/0 (socket says 172.21.15.61:55622) 2023-10-18T09:18:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27de59b700 1 -- 172.21.15.61:0/165789753 learned_addr learned my addr 172.21.15.61:0/165789753 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ded9c700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810ccf0 0x7f27d81a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 -- 172.21.15.61:0/165789753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 msgr2=0x7f27d819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 0x7f27d819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 -- 172.21.15.61:0/165789753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810ccf0 msgr2=0x7f27d81a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810ccf0 0x7f27d81a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 -- 172.21.15.61:0/165789753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40095f0 con 0x7f27d8074830 2023-10-18T09:18:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.262+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d819d1e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9c0 tx=0x7f27c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.263+0000 7f27cf7fe700 1 -- 172.21.15.61:0/165789753 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8004220 con 0x7f27d8074830 2023-10-18T09:18:31.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.263+0000 7f27cf7fe700 1 -- 172.21.15.61:0/165789753 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c8004380 con 0x7f27d8074830 2023-10-18T09:18:31.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.263+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d81a4da0 con 0x7f27d8074830 2023-10-18T09:18:31.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.263+0000 7f27cf7fe700 1 -- 172.21.15.61:0/165789753 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c8003be0 con 0x7f27d8074830 2023-10-18T09:18:31.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.263+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a51f0 con 0x7f27d8074830 2023-10-18T09:18:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.266+0000 7f27cf7fe700 1 -- 172.21.15.61:0/165789753 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f27c8003d40 con 0x7f27d8074830 2023-10-18T09:18:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.267+0000 7f27f414bb80 1 Processor -- start 2023-10-18T09:18:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.267+0000 7f27cf7fe700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 0x7f27c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.267+0000 7f27f414bb80 1 -- start start 2023-10-18T09:18:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27cf7fe700 1 -- 172.21.15.61:0/165789753 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f27c8014070 con 0x7f27d8074830 2023-10-18T09:18:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27ded9c700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 0x7f27c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3355b690 con 0x557a335e9280 2023-10-18T09:18:31.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a33541c10 con 0x557a335e9be0 2023-10-18T09:18:31.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27f414bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3353f530 con 0x557a335e8580 2023-10-18T09:18:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27de59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27de59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57574/0 (socket says 172.21.15.61:57574) 2023-10-18T09:18:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27de59b700 1 -- 172.21.15.61:0/1892011922 learned_addr learned my addr 172.21.15.61:0/1892011922 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.268+0000 7f27ded9c700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 -- 172.21.15.61:0/1892011922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 msgr2=0x557a335e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 -- 172.21.15.61:0/1892011922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 msgr2=0x557a335e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 -- 172.21.15.61:0/1892011922 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a334f83a0 con 0x557a335e9280 2023-10-18T09:18:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27ded9c700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 0x7f27c40a3940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f27d000f540 tx=0x7f27d0009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27de59b700 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d4005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48db195f2d881e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27ccff9700 1 -- 172.21.15.61:0/1892011922 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d401d050 con 0x557a335e9280 2023-10-18T09:18:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27ccff9700 1 -- 172.21.15.61:0/1892011922 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d401e040 con 0x557a335e9280 2023-10-18T09:18:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.269+0000 7f27ccff9700 1 -- 172.21.15.61:0/1892011922 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4023dc0 con 0x557a335e9280 2023-10-18T09:18:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 -- 172.21.15.61:0/1892011922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 msgr2=0x557a335e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d4005ea0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 -- 172.21.15.61:0/1892011922 shutdown_connections 2023-10-18T09:18:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335e96a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1892011922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 -- 172.21.15.61:0/1892011922 >> 172.21.15.61:0/1892011922 conn(0x557a3355c6d0 msgr2=0x557a335e2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 -- 172.21.15.61:0/1892011922 shutdown_connections 2023-10-18T09:18:31.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.270+0000 7f27f414bb80 1 -- 172.21.15.61:0/1892011922 wait complete. 2023-10-18T09:18:31.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 -- start start 2023-10-18T09:18:31.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 Processor -- start 2023-10-18T09:18:31.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3353ae20 con 0x557a335e9280 2023-10-18T09:18:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a334d76c0 con 0x557a335e9be0 2023-10-18T09:18:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.271+0000 7f27f414bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a334d7360 con 0x557a335e8580 2023-10-18T09:18:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27f414bb80 10 client.0 _ll_register_callbacks cb 0x557a33421a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57590/0 (socket says 172.21.15.61:57590) 2023-10-18T09:18:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ded9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 -- 172.21.15.61:0/2655421258 learned_addr learned my addr 172.21.15.61:0/2655421258 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27de59b700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 -- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 msgr2=0x557a335f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 -- 172.21.15.61:0/2655421258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 msgr2=0x557a335f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 -- 172.21.15.61:0/2655421258 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d81a51f0 con 0x557a335e9280 2023-10-18T09:18:31.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003ff0 tx=0x7f27c8099f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be268565ad6891b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:31.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.272+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c809e410 con 0x557a335e9280 2023-10-18T09:18:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f27c80a2650 con 0x557a335e9280 2023-10-18T09:18:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a3353fea0 con 0x557a335e9280 2023-10-18T09:18:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c8098660 con 0x557a335e9280 2023-10-18T09:18:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 10 client.15501 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f27d810fa90 con 0x557a335e9280 2023-10-18T09:18:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c8063070 con 0x557a335e9280 2023-10-18T09:18:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27c80a21c0 con 0x557a335e9280 2023-10-18T09:18:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 10 client.15501 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-18T09:18:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 20 client.15501 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a335fd6d0 0x557a335fc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 4 client.15501 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.273+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a335b7860 con 0x557a335fd6d0 2023-10-18T09:18:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.274+0000 7f27de59b700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a335fd6d0 0x557a335fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.274+0000 7f27de59b700 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a335fd6d0 0x557a335fc210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=61c73433b34d1ded server_cookie=5e0795f90ff3bc9 in_seq=0 out_seq=0 2023-10-18T09:18:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.274+0000 7f27c1ffb700 10 client.15501 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27c1ffb700 1 -- 172.21.15.61:0/2655421258 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557a335b7860 con 0x557a335fd6d0 2023-10-18T09:18:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27c1ffb700 10 client.15501 handle_command_reply: tid=0 2023-10-18T09:18:31.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 client.15501 shutdown 2023-10-18T09:18:31.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 msgr2=0x557a335f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f27c8003ff0 tx=0x7f27c8099f00 comp rx=0 tx=0).stop 2023-10-18T09:18:31.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 shutdown_connections 2023-10-18T09:18:31.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a335fd6d0 0x557a335fc210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.275+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27f414bb80 1 --2- 172.21.15.61:0/2655421258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f1fd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27c1ffb700 0 client.15501 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 >> 172.21.15.61:0/2655421258 conn(0x557a3355c6d0 msgr2=0x557a335e7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 shutdown_connections 2023-10-18T09:18:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27f414bb80 1 -- 172.21.15.61:0/2655421258 wait complete. 2023-10-18T09:18:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.276+0000 7f27f414bb80 20 client.15501 trim_cache size 0 max 16384 2023-10-18T09:18:31.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.294+0000 7f27f414bb80 1 Processor -- start 2023-10-18T09:18:31.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.294+0000 7f27f414bb80 1 -- start start 2023-10-18T09:18:31.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a333bb8d0 con 0x557a335e9280 2023-10-18T09:18:31.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a3353c0b0 con 0x557a335e9be0 2023-10-18T09:18:31.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27f414bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a334f9590 con 0x557a335e8580 2023-10-18T09:18:31.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57600/0 (socket says 172.21.15.61:57600) 2023-10-18T09:18:31.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 -- 172.21.15.61:0/3703706181 learned_addr learned my addr 172.21.15.61:0/3703706181 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ded9c700 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27de59b700 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 -- 172.21.15.61:0/3703706181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 msgr2=0x557a335f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 -- 172.21.15.61:0/3703706181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 msgr2=0x557a335efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335efb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.295+0000 7f27ddd9a700 1 -- 172.21.15.61:0/3703706181 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a33365d90 con 0x557a335e9280 2023-10-18T09:18:31.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.296+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c5b0 tx=0x7f27c80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d13a8b47742136 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.296+0000 7f27cd7fa700 1 -- 172.21.15.61:0/3703706181 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80960b0 con 0x557a335e9280 2023-10-18T09:18:31.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.296+0000 7f27cd7fa700 1 -- 172.21.15.61:0/3703706181 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c8063070 con 0x557a335e9280 2023-10-18T09:18:31.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.296+0000 7f27cd7fa700 1 -- 172.21.15.61:0/3703706181 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a21c0 con 0x557a335e9280 2023-10-18T09:18:31.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.296+0000 7f27f414bb80 1 -- 172.21.15.61:0/3703706181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 msgr2=0x557a335f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c5b0 tx=0x7f27c80092d0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 -- 172.21.15.61:0/3703706181 shutdown_connections 2023-10-18T09:18:31.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 --2- 172.21.15.61:0/3703706181 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f8f30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 -- 172.21.15.61:0/3703706181 >> 172.21.15.61:0/3703706181 conn(0x557a3355c6d0 msgr2=0x557a335e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 -- 172.21.15.61:0/3703706181 shutdown_connections 2023-10-18T09:18:31.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 -- 172.21.15.61:0/3703706181 wait complete. 2023-10-18T09:18:31.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 -- start start 2023-10-18T09:18:31.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.297+0000 7f27f414bb80 1 Processor -- start 2023-10-18T09:18:31.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a335f8fc0 con 0x557a335e9280 2023-10-18T09:18:31.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a335f9130 con 0x557a335e9be0 2023-10-18T09:18:31.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a335f92f0 con 0x557a335e8580 2023-10-18T09:18:31.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27f414bb80 10 client.0 _ll_register_callbacks cb 0x557a33421a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27ddd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:55650/0 (socket says 172.21.15.61:55650) 2023-10-18T09:18:31.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 -- 172.21.15.61:0/1532164717 learned_addr learned my addr 172.21.15.61:0/1532164717 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27ded9c700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 -- 172.21.15.61:0/1532164717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 msgr2=0x557a335f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 -- 172.21.15.61:0/1532164717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 msgr2=0x557a335f84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27de59b700 1 -- 172.21.15.61:0/1532164717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d810fa90 con 0x557a335e8580 2023-10-18T09:18:31.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.298+0000 7f27ded9c700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:31.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27de59b700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b2b0 tx=0x7f27d40239d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f662f795dd754ab server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:18:31.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4033070 con 0x557a335e8580 2023-10-18T09:18:31.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d401e040 con 0x557a335e8580 2023-10-18T09:18:31.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f27d400ba60 con 0x557a335e8580 2023-10-18T09:18:31.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d401d040 con 0x557a335e8580 2023-10-18T09:18:31.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27f414bb80 10 client.25516 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.299+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557a3334ec10 con 0x557a335e8580 2023-10-18T09:18:31.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d401e040 con 0x557a335e8580 2023-10-18T09:18:31.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f27d4005040 con 0x557a335e8580 2023-10-18T09:18:31.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 10 client.25516 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 10 client.25516 resolve_mds: resolved name 'f' to daemon mds.f 2023-10-18T09:18:31.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 20 client.25516 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a33655c60 0x557a336520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 4 client.25516 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:31.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557a33533e00 con 0x557a33655c60 2023-10-18T09:18:31.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.300+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a33655c60 0x557a336520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27ddd9a700 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a33655c60 0x557a336520c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fbf3cc0415e31b61 server_cookie=94e42c404c9c306a in_seq=0 out_seq=0 2023-10-18T09:18:31.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27c1ffb700 10 client.25516 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:31.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27c1ffb700 1 -- 172.21.15.61:0/1532164717 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x557a33533e00 con 0x557a33655c60 2023-10-18T09:18:31.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27c1ffb700 10 client.25516 handle_command_reply: tid=0 2023-10-18T09:18:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 client.25516 shutdown 2023-10-18T09:18:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 msgr2=0x557a335f7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b2b0 tx=0x7f27d40239d0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 shutdown_connections 2023-10-18T09:18:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a335e9be0 0x557a335f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x557a33655c60 0x557a336520c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557a335e8580 0x557a335f7f70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 --2- 172.21.15.61:0/1532164717 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557a335e9280 0x557a335f84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27c1ffb700 0 client.25516 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:18:31.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 >> 172.21.15.61:0/1532164717 conn(0x557a3355c6d0 msgr2=0x557a335f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 shutdown_connections 2023-10-18T09:18:31.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.301+0000 7f27f414bb80 1 -- 172.21.15.61:0/1532164717 wait complete. 2023-10-18T09:18:31.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27f414bb80 20 client.25516 trim_cache size 0 max 16384 2023-10-18T09:18:31.327 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:31.327 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:31.327 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 3, 2023-10-18T09:18:31.328 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24646, 2023-10-18T09:18:31.328 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:31.328 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:31.328 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:31.329 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 93.785823932, 2023-10-18T09:18:31.329 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:31.329 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 92, 2023-10-18T09:18:31.329 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:31.329 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 146.444931255 2023-10-18T09:18:31.330 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:31.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 msgr2=0x7f27c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 0x7f27c40a3940 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f27d000f540 tx=0x7f27d0009920 comp rx=0 tx=0).stop 2023-10-18T09:18:31.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 msgr2=0x7f27d819d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d819d1e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27c800e9c0 tx=0x7f27c800ecd0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 shutdown_connections 2023-10-18T09:18:31.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f27c40a1480 0x7f27c40a3940 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27d810ccf0 0x7f27d81a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f27d8073250 0x7f27d819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 --2- 172.21.15.61:0/165789753 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f27d8074830 0x7f27d819d1e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 >> 172.21.15.61:0/165789753 conn(0x7f27d80fbc90 msgr2=0x7f27d8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 shutdown_connections 2023-10-18T09:18:31.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.302+0000 7f27e07ff700 1 -- 172.21.15.61:0/165789753 wait complete. 2023-10-18T09:18:31.334 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-18T09:18:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:30 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/147363936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:30 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/780325410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:30 smithi170 ceph-mon[135150]: pgmap v357: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 229 B/s wr, 8 op/s 2023-10-18T09:18:31.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.536+0000 7efff33ff700 1 -- 172.21.15.61:0/826821052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec101910 msgr2=0x7effec101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.536+0000 7efff33ff700 1 --2- 172.21.15.61:0/826821052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec101910 0x7effec101d70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7effdc0099c0 tx=0x7effdc009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 -- 172.21.15.61:0/826821052 shutdown_connections 2023-10-18T09:18:31.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 --2- 172.21.15.61:0/826821052 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec1022b0 0x7effec108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 --2- 172.21.15.61:0/826821052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec100720 0x7effec100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 --2- 172.21.15.61:0/826821052 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec101910 0x7effec101d70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 -- 172.21.15.61:0/826821052 >> 172.21.15.61:0/826821052 conn(0x7effec0fbc90 msgr2=0x7effec0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 -- 172.21.15.61:0/826821052 shutdown_connections 2023-10-18T09:18:31.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.537+0000 7efff33ff700 1 -- 172.21.15.61:0/826821052 wait complete. 2023-10-18T09:18:31.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 Processor -- start 2023-10-18T09:18:31.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 -- start start 2023-10-18T09:18:31.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec101910 0x7effec072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 0x7effec073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0738c0 con 0x7effec100720 2023-10-18T09:18:31.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec073a30 con 0x7effec1022b0 2023-10-18T09:18:31.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff33ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec1a0700 con 0x7effec101910 2023-10-18T09:18:31.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57634/0 (socket says 172.21.15.61:57634) 2023-10-18T09:18:31.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.538+0000 7efff119b700 1 -- 172.21.15.61:0/3518632631 learned_addr learned my addr 172.21.15.61:0/3518632631 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff099a700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec101910 0x7effec072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff199c700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 0x7effec073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 -- 172.21.15.61:0/3518632631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec101910 msgr2=0x7effec072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec101910 0x7effec072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 -- 172.21.15.61:0/3518632631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 msgr2=0x7effec073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 0x7effec073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 -- 172.21.15.61:0/3518632631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0095f0 con 0x7effec100720 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff199c700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 0x7effec073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff119b700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effe800e970 tx=0x7effe800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7effe27fc700 1 -- 172.21.15.61:0/3518632631 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8009790 con 0x7effec100720 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7effe27fc700 1 -- 172.21.15.61:0/3518632631 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe80044e0 con 0x7effec100720 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec1a0990 con 0x7effec100720 2023-10-18T09:18:31.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7effe27fc700 1 -- 172.21.15.61:0/3518632631 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8003d80 con 0x7effec100720 2023-10-18T09:18:31.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.539+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec1a0ea0 con 0x7effec100720 2023-10-18T09:18:31.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.542+0000 7effe27fc700 1 -- 172.21.15.61:0/3518632631 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7effe8003ee0 con 0x7effec100720 2023-10-18T09:18:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 Processor -- start 2023-10-18T09:18:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7effe27fc700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 0x7effd80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 -- start start 2023-10-18T09:18:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7effe27fc700 1 -- 172.21.15.61:0/3518632631 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7effe8014070 con 0x7effec100720 2023-10-18T09:18:31.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7efff199c700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 0x7effd80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91bbe0 0x56143f91a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 0x56143f91aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f88d690 con 0x56143f91b280 2023-10-18T09:18:31.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f873c10 con 0x56143f91bbe0 2023-10-18T09:18:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7f0006daeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f871530 con 0x56143f91a580 2023-10-18T09:18:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.544+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57650/0 (socket says 172.21.15.61:57650) 2023-10-18T09:18:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 -- 172.21.15.61:0/1860214638 learned_addr learned my addr 172.21.15.61:0/1860214638 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff099a700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91bbe0 0x56143f91a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff199c700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 0x56143f91aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 -- 172.21.15.61:0/1860214638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 msgr2=0x56143f91aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 0x56143f91aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 -- 172.21.15.61:0/1860214638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91bbe0 msgr2=0x56143f91a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91bbe0 0x56143f91a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 -- 172.21.15.61:0/1860214638 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56143f82a3a0 con 0x56143f91b280 2023-10-18T09:18:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff199c700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 0x56143f91aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:18:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff199c700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 0x7effd80a3940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effe40093f0 tx=0x7effe400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7efff119b700 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7effe8096800 tx=0x7effe8099ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9a1fb840cc6d27 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.545+0000 7effd77fe700 1 -- 172.21.15.61:0/1860214638 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe80cd160 con 0x56143f91b280 2023-10-18T09:18:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7effd77fe700 1 -- 172.21.15.61:0/1860214638 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe80b0070 con 0x56143f91b280 2023-10-18T09:18:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7effd77fe700 1 -- 172.21.15.61:0/1860214638 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe801fe00 con 0x56143f91b280 2023-10-18T09:18:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 -- 172.21.15.61:0/1860214638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 msgr2=0x56143f91b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7effe8096800 tx=0x7effe8099ec0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 -- 172.21.15.61:0/1860214638 shutdown_connections 2023-10-18T09:18:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91a580 0x56143f91aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f91b6a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 --2- 172.21.15.61:0/1860214638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91bbe0 0x56143f91a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 -- 172.21.15.61:0/1860214638 >> 172.21.15.61:0/1860214638 conn(0x56143f88e6d0 msgr2=0x56143f914b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 -- 172.21.15.61:0/1860214638 shutdown_connections 2023-10-18T09:18:31.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.546+0000 7f0006daeb80 1 -- 172.21.15.61:0/1860214638 wait complete. 2023-10-18T09:18:31.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.547+0000 7f0006daeb80 1 -- start start 2023-10-18T09:18:31.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.547+0000 7f0006daeb80 1 Processor -- start 2023-10-18T09:18:31.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91b280 0x56143f923f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f86ce20 con 0x56143f91a580 2023-10-18T09:18:31.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f8096c0 con 0x56143f91b280 2023-10-18T09:18:31.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f809360 con 0x56143f91bbe0 2023-10-18T09:18:31.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7f0006daeb80 10 client.0 _ll_register_callbacks cb 0x56143f753a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7efff119b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57660/0 (socket says 172.21.15.61:57660) 2023-10-18T09:18:31.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7efff119b700 1 -- 172.21.15.61:0/2624648292 learned_addr learned my addr 172.21.15.61:0/2624648292 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7efff199c700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.548+0000 7efff099a700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91b280 0x56143f923f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 -- 172.21.15.61:0/2624648292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 msgr2=0x56143f92af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 -- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91b280 msgr2=0x56143f923f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91b280 0x56143f923f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 -- 172.21.15.61:0/2624648292 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec1a0ea0 con 0x56143f91a580 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7efff119b700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7effe800c5b0 tx=0x7effe809a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddbff7bd555002f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7effd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8062070 con 0x56143f91a580 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7effe80a2690 con 0x56143f91a580 2023-10-18T09:18:31.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56143f871ea0 con 0x56143f91a580 2023-10-18T09:18:31.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe80cd160 con 0x56143f91a580 2023-10-18T09:18:31.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.549+0000 7f0006daeb80 10 client.15519 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56143f697d90 con 0x56143f91a580 2023-10-18T09:18:31.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56143f871ea0 con 0x56143f91a580 2023-10-18T09:18:31.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7effe8065020 con 0x56143f91a580 2023-10-18T09:18:31.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 10 client.15519 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 10 client.15519 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-18T09:18:31.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 20 client.15519 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f92f150 0x56143f92dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 4 client.15519 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:31.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56143f8e9860 con 0x56143f92f150 2023-10-18T09:18:31.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.550+0000 7efff099a700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f92f150 0x56143f92dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.551+0000 7efff099a700 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f92f150 0x56143f92dea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9dbe3ae4712260e6 server_cookie=9ecebe8d3347ebd8 in_seq=0 out_seq=0 2023-10-18T09:18:31.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.551+0000 7effd4ff9700 10 client.15519 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:31.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7effd4ff9700 1 -- 172.21.15.61:0/2624648292 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56143f8e9860 con 0x56143f92f150 2023-10-18T09:18:31.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7effd4ff9700 10 client.15519 handle_command_reply: tid=0 2023-10-18T09:18:31.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 client.15519 shutdown 2023-10-18T09:18:31.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 msgr2=0x56143f9239e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7effe800c5b0 tx=0x7effe809a370 comp rx=0 tx=0).stop 2023-10-18T09:18:31.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 shutdown_connections 2023-10-18T09:18:31.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f92f150 0x56143f92dea0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91a580 0x56143f9239e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2624648292 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91b280 0x56143f923f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 >> 172.21.15.61:0/2624648292 conn(0x56143f88e6d0 msgr2=0x56143f919030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7effd4ff9700 0 client.15519 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 shutdown_connections 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.552+0000 7f0006daeb80 1 -- 172.21.15.61:0/2624648292 wait complete. 2023-10-18T09:18:31.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.553+0000 7f0006daeb80 20 client.15519 trim_cache size 0 max 16384 2023-10-18T09:18:31.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.570+0000 7f0006daeb80 1 Processor -- start 2023-10-18T09:18:31.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 -- start start 2023-10-18T09:18:31.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f922be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f6ed8d0 con 0x56143f91b280 2023-10-18T09:18:31.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f871ea0 con 0x56143f91a580 2023-10-18T09:18:31.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7f0006daeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f824720 con 0x56143f91bbe0 2023-10-18T09:18:31.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff099a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff099a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57674/0 (socket says 172.21.15.61:57674) 2023-10-18T09:18:31.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff099a700 1 -- 172.21.15.61:0/2224935655 learned_addr learned my addr 172.21.15.61:0/2224935655 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff199c700 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff119b700 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f922be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.571+0000 7efff099a700 1 -- 172.21.15.61:0/2224935655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 msgr2=0x56143f92af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7efff099a700 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7efff099a700 1 -- 172.21.15.61:0/2224935655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 msgr2=0x56143f922be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7efff099a700 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f922be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7efff099a700 1 -- 172.21.15.61:0/2224935655 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56143f680c10 con 0x56143f91b280 2023-10-18T09:18:31.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7efff099a700 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7effdc0058e0 tx=0x7effdc005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec1a4a97b3dbd4a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7effd7fff700 1 -- 172.21.15.61:0/2224935655 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc01d040 con 0x56143f91b280 2023-10-18T09:18:31.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7effd7fff700 1 -- 172.21.15.61:0/2224935655 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effdc003c90 con 0x56143f91b280 2023-10-18T09:18:31.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.572+0000 7effd7fff700 1 -- 172.21.15.61:0/2224935655 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc0055e0 con 0x56143f91b280 2023-10-18T09:18:31.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- 172.21.15.61:0/2224935655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 msgr2=0x56143f923120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7effdc0058e0 tx=0x7effdc005ed0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- 172.21.15.61:0/2224935655 shutdown_connections 2023-10-18T09:18:31.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91bbe0 0x56143f92af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f922be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 --2- 172.21.15.61:0/2224935655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91b280 0x56143f923120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- 172.21.15.61:0/2224935655 >> 172.21.15.61:0/2224935655 conn(0x56143f88e6d0 msgr2=0x56143f914ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- 172.21.15.61:0/2224935655 shutdown_connections 2023-10-18T09:18:31.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- 172.21.15.61:0/2224935655 wait complete. 2023-10-18T09:18:31.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 -- start start 2023-10-18T09:18:31.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.573+0000 7f0006daeb80 1 Processor -- start 2023-10-18T09:18:31.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91b280 0x56143f9286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 0x56143f982b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f928c30 con 0x56143f91bbe0 2023-10-18T09:18:31.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f928da0 con 0x56143f91a580 2023-10-18T09:18:31.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56143f9830b0 con 0x56143f91b280 2023-10-18T09:18:31.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7f0006daeb80 10 client.0 _ll_register_callbacks cb 0x56143f753a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7efff199c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 0x56143f982b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7efff119b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7efff119b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43640/0 (socket says 172.21.15.61:43640) 2023-10-18T09:18:31.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7efff119b700 1 -- 172.21.15.61:0/4039454931 learned_addr learned my addr 172.21.15.61:0/4039454931 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.574+0000 7efff099a700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91b280 0x56143f9286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 -- 172.21.15.61:0/4039454931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91b280 msgr2=0x56143f9286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91b280 0x56143f9286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 -- 172.21.15.61:0/4039454931 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 msgr2=0x56143f982b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 0x56143f982b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 -- 172.21.15.61:0/4039454931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec1094b0 con 0x56143f91a580 2023-10-18T09:18:31.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff199c700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 0x56143f982b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:31.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7efff119b700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7effe8009230 tx=0x7effe80cab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8549c0f0c50f2b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7effd4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:31.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8062030 con 0x56143f91a580 2023-10-18T09:18:31.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7effe8063070 con 0x56143f91a580 2023-10-18T09:18:31.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effe80b0070 con 0x56143f91a580 2023-10-18T09:18:31.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7effe809e170 con 0x56143f91a580 2023-10-18T09:18:31.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7f0006daeb80 10 client.25508 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.575+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56143f6be200 con 0x56143f91a580 2023-10-18T09:18:31.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effe80b0070 con 0x56143f91a580 2023-10-18T09:18:31.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7effe8096a10 con 0x56143f91a580 2023-10-18T09:18:31.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 10 client.25508 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 10 client.25508 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-18T09:18:31.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 20 client.25508 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f987f20 0x56143f9843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 4 client.25508 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:31.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56143f865e00 con 0x56143f987f20 2023-10-18T09:18:31.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.576+0000 7efff099a700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f987f20 0x56143f9843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7efff099a700 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f987f20 0x56143f9843f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bda8921d158224c3 server_cookie=16eee5dd60af23b4 in_seq=0 out_seq=0 2023-10-18T09:18:31.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7effd4ff9700 10 client.25508 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7effd4ff9700 1 -- 172.21.15.61:0/4039454931 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56143f865e00 con 0x56143f987f20 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7effd4ff9700 10 client.25508 handle_command_reply: tid=0 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 client.25508 shutdown 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 msgr2=0x56143f928170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7effe8009230 tx=0x7effe80cab60 comp rx=0 tx=0).stop 2023-10-18T09:18:31.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 shutdown_connections 2023-10-18T09:18:31.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x56143f987f20 0x56143f9843f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56143f91bbe0 0x56143f982b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56143f91a580 0x56143f928170 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7effd4ff9700 0 client.25508 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:18:31.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 --2- 172.21.15.61:0/4039454931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56143f91b280 0x56143f9286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 >> 172.21.15.61:0/4039454931 conn(0x56143f88e6d0 msgr2=0x56143f922be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 shutdown_connections 2023-10-18T09:18:31.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 1 -- 172.21.15.61:0/4039454931 wait complete. 2023-10-18T09:18:31.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.577+0000 7f0006daeb80 20 client.25508 trim_cache size 0 max 16384 2023-10-18T09:18:31.595 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:31.595 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:31.595 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 2, 2023-10-18T09:18:31.595 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24653, 2023-10-18T09:18:31.596 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:31.596 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:31.596 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:31.596 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 95.068590639999996, 2023-10-18T09:18:31.596 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:31.597 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 92, 2023-10-18T09:18:31.597 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:31.597 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 142.27559366200001 2023-10-18T09:18:31.597 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:31.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 msgr2=0x7effd80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 0x7effd80a3940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7effe40093f0 tx=0x7effe400b040 comp rx=0 tx=0).stop 2023-10-18T09:18:31.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 msgr2=0x7effec074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effe800e970 tx=0x7effe800ed30 comp rx=0 tx=0).stop 2023-10-18T09:18:31.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 shutdown_connections 2023-10-18T09:18:31.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7effd80a1480 0x7effd80a3940 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7effec1022b0 0x7effec073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7effec100720 0x7effec074550 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 --2- 172.21.15.61:0/3518632631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effec101910 0x7effec072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 >> 172.21.15.61:0/3518632631 conn(0x7effec0fbc90 msgr2=0x7effec104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 shutdown_connections 2023-10-18T09:18:31.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.578+0000 7efff33ff700 1 -- 172.21.15.61:0/3518632631 wait complete. 2023-10-18T09:18:31.601 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-18T09:18:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 -- 172.21.15.61:0/1408379143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c101910 msgr2=0x7f753c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 --2- 172.21.15.61:0/1408379143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c101910 0x7f753c101d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f75300099c0 tx=0x7f7530009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 -- 172.21.15.61:0/1408379143 shutdown_connections 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 --2- 172.21.15.61:0/1408379143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 0x7f753c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 --2- 172.21.15.61:0/1408379143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c100720 0x7f753c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 --2- 172.21.15.61:0/1408379143 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c101910 0x7f753c101d70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 -- 172.21.15.61:0/1408379143 >> 172.21.15.61:0/1408379143 conn(0x7f753c0fbc90 msgr2=0x7f753c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 -- 172.21.15.61:0/1408379143 shutdown_connections 2023-10-18T09:18:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.804+0000 7f75421ff700 1 -- 172.21.15.61:0/1408379143 wait complete. 2023-10-18T09:18:31.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.805+0000 7f75421ff700 1 Processor -- start 2023-10-18T09:18:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.805+0000 7f75421ff700 1 -- start start 2023-10-18T09:18:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101910 0x7f753c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 0x7f753c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a0410 con 0x7f753c100720 2023-10-18T09:18:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a0550 con 0x7f753c1022b0 2023-10-18T09:18:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f75421ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c1a06e0 con 0x7f753c101910 2023-10-18T09:18:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57700/0 (socket says 172.21.15.61:57700) 2023-10-18T09:18:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 -- 172.21.15.61:0/2251475554 learned_addr learned my addr 172.21.15.61:0/2251475554 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753bfff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 0x7f753c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753affd700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101910 0x7f753c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 -- 172.21.15.61:0/2251475554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101910 msgr2=0x7f753c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101910 0x7f753c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 -- 172.21.15.61:0/2251475554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 msgr2=0x7f753c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 0x7f753c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.806+0000 7f753b7fe700 1 -- 172.21.15.61:0/2251475554 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300095f0 con 0x7f753c100720 2023-10-18T09:18:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e970 tx=0x7f752c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f7538ff9700 1 -- 172.21.15.61:0/2251475554 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c009790 con 0x7f753c100720 2023-10-18T09:18:31.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f7538ff9700 1 -- 172.21.15.61:0/2251475554 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752c00c7e0 con 0x7f753c100720 2023-10-18T09:18:31.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f7538ff9700 1 -- 172.21.15.61:0/2251475554 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c003c70 con 0x7f753c100720 2023-10-18T09:18:31.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c1a09c0 con 0x7f753c100720 2023-10-18T09:18:31.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.807+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c1a0e80 con 0x7f753c100720 2023-10-18T09:18:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.810+0000 7f7538ff9700 1 -- 172.21.15.61:0/2251475554 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f752c00cc50 con 0x7f753c100720 2023-10-18T09:18:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 Processor -- start 2023-10-18T09:18:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 -- start start 2023-10-18T09:18:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7538ff9700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 0x7f75240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59be0 0x5613f4d58040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7538ff9700 1 -- 172.21.15.61:0/2251475554 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f752c014070 con 0x7f753c100720 2023-10-18T09:18:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d58580 0x5613f4d58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4ccb690 con 0x5613f4d58580 2023-10-18T09:18:31.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4cb1c10 con 0x5613f4d59280 2023-10-18T09:18:31.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.812+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4caf530 con 0x5613f4d59be0 2023-10-18T09:18:31.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753bfff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 0x7f75240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d58580 0x5613f4d58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43666/0 (socket says 172.21.15.61:43666) 2023-10-18T09:18:31.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 -- 172.21.15.61:0/940536697 learned_addr learned my addr 172.21.15.61:0/940536697 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753affd700 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59be0 0x5613f4d58040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 -- 172.21.15.61:0/940536697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59be0 msgr2=0x5613f4d58040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59be0 0x5613f4d58040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 -- 172.21.15.61:0/940536697 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d58580 msgr2=0x5613f4d58a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d58580 0x5613f4d58a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 -- 172.21.15.61:0/940536697 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613f4c683a0 con 0x5613f4d59280 2023-10-18T09:18:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753bfff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 0x7f75240a39c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ec0 tx=0x7f75340092f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.813+0000 7f753b7fe700 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f752c09a0f0 tx=0x7f752c09a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2339bb66bc012 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7521ffb700 1 -- 172.21.15.61:0/940536697 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c09a160 con 0x5613f4d59280 2023-10-18T09:18:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7521ffb700 1 -- 172.21.15.61:0/940536697 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752c09e2b0 con 0x5613f4d59280 2023-10-18T09:18:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7521ffb700 1 -- 172.21.15.61:0/940536697 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c09a3c0 con 0x5613f4d59280 2023-10-18T09:18:31.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 -- 172.21.15.61:0/940536697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 msgr2=0x5613f4d596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f752c09a0f0 tx=0x7f752c09a060 comp rx=0 tx=0).stop 2023-10-18T09:18:31.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 -- 172.21.15.61:0/940536697 shutdown_connections 2023-10-18T09:18:31.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d58580 0x5613f4d58a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d596a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/940536697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59be0 0x5613f4d58040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 -- 172.21.15.61:0/940536697 >> 172.21.15.61:0/940536697 conn(0x5613f4ccc6d0 msgr2=0x5613f4d52b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.814+0000 7f7555bb8b80 1 -- 172.21.15.61:0/940536697 shutdown_connections 2023-10-18T09:18:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.815+0000 7f7555bb8b80 1 -- 172.21.15.61:0/940536697 wait complete. 2023-10-18T09:18:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.815+0000 7f7555bb8b80 1 -- start start 2023-10-18T09:18:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.815+0000 7f7555bb8b80 1 Processor -- start 2023-10-18T09:18:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d61af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d62030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d69020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4caae20 con 0x5613f4d59be0 2023-10-18T09:18:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4c476c0 con 0x5613f4d58580 2023-10-18T09:18:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4c47360 con 0x5613f4d59280 2023-10-18T09:18:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f7555bb8b80 10 client.0 _ll_register_callbacks cb 0x5613f4b91a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d69020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d69020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57720/0 (socket says 172.21.15.61:57720) 2023-10-18T09:18:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d61af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753bfff700 1 -- 172.21.15.61:0/2994586140 learned_addr learned my addr 172.21.15.61:0/2994586140 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753affd700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d62030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 -- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 msgr2=0x5613f4d62030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d62030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 -- 172.21.15.61:0/2994586140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 msgr2=0x5613f4d69020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d69020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.816+0000 7f753b7fe700 1 -- 172.21.15.61:0/2994586140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c1a0e80 con 0x5613f4d58580 2023-10-18T09:18:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f753b7fe700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d61af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f752c0a1fc0 tx=0x7f752c099950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c308321c120628a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c0a24f0 con 0x5613f4d58580 2023-10-18T09:18:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613f4cafea0 con 0x5613f4d58580 2023-10-18T09:18:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f752c09a160 con 0x5613f4d58580 2023-10-18T09:18:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f752c09a5d0 con 0x5613f4d58580 2023-10-18T09:18:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f7555bb8b80 10 client.25520 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5613f4ad5d90 con 0x5613f4d58580 2023-10-18T09:18:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613f4cafea0 con 0x5613f4d58580 2023-10-18T09:18:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.817+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f752c0832a0 con 0x5613f4d58580 2023-10-18T09:18:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 10 client.25520 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-18T09:18:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 20 client.25520 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4d6d240 0x5613f4d6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 4 client.25520 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613f4d27860 con 0x5613f4d6d240 2023-10-18T09:18:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f753affd700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4d6d240 0x5613f4d6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f753affd700 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4d6d240 0x5613f4d6bff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ecea2f3651f7f34 server_cookie=b965a13098df34e1 in_seq=0 out_seq=0 2023-10-18T09:18:31.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.818+0000 7f751affd700 10 client.25520 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:31.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.819+0000 7f751affd700 1 -- 172.21.15.61:0/2994586140 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5613f4d27860 con 0x5613f4d6d240 2023-10-18T09:18:31.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.819+0000 7f751affd700 10 client.25520 handle_command_reply: tid=0 2023-10-18T09:18:31.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.819+0000 7f7555bb8b80 1 client.25520 shutdown 2023-10-18T09:18:31.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.819+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 msgr2=0x5613f4d61af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.819+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d61af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f752c0a1fc0 tx=0x7f752c099950 comp rx=0 tx=0).stop 2023-10-18T09:18:31.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 shutdown_connections 2023-10-18T09:18:31.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4d6d240 0x5613f4d6bff0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d69020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d61af0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f751affd700 0 client.25520 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:31.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/2994586140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d62030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 >> 172.21.15.61:0/2994586140 conn(0x5613f4ccc6d0 msgr2=0x5613f4d57030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 shutdown_connections 2023-10-18T09:18:31.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 1 -- 172.21.15.61:0/2994586140 wait complete. 2023-10-18T09:18:31.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.820+0000 7f7555bb8b80 20 client.25520 trim_cache size 0 max 16384 2023-10-18T09:18:31.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 Processor -- start 2023-10-18T09:18:31.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 -- start start 2023-10-18T09:18:31.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d5c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d68fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4b2b8d0 con 0x5613f4d59be0 2023-10-18T09:18:31.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4d4f500 con 0x5613f4d58580 2023-10-18T09:18:31.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.838+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4d27860 con 0x5613f4d59280 2023-10-18T09:18:31.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57728/0 (socket says 172.21.15.61:57728) 2023-10-18T09:18:31.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753bfff700 1 -- 172.21.15.61:0/1763945319 learned_addr learned my addr 172.21.15.61:0/1763945319 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d5c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753affd700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d68fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 -- 172.21.15.61:0/1763945319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 msgr2=0x5613f4d68fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d68fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 -- 172.21.15.61:0/1763945319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 msgr2=0x5613f4d5c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 -- 172.21.15.61:0/1763945319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613f4abec10 con 0x5613f4d58580 2023-10-18T09:18:31.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753bfff700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:31.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.839+0000 7f753b7fe700 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d5c280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f752c09a0f0 tx=0x7f752c003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc91807616cee66f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f75227fc700 1 -- 172.21.15.61:0/1763945319 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c09a160 con 0x5613f4d58580 2023-10-18T09:18:31.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f75227fc700 1 -- 172.21.15.61:0/1763945319 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752c0a24f0 con 0x5613f4d58580 2023-10-18T09:18:31.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f75227fc700 1 -- 172.21.15.61:0/1763945319 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c098500 con 0x5613f4d58580 2023-10-18T09:18:31.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 -- 172.21.15.61:0/1763945319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 msgr2=0x5613f4d5c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d5c280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f752c09a0f0 tx=0x7f752c003680 comp rx=0 tx=0).stop 2023-10-18T09:18:31.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 -- 172.21.15.61:0/1763945319 shutdown_connections 2023-10-18T09:18:31.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4d5c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d58580 0x5613f4d5c280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/1763945319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d59280 0x5613f4d68fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 -- 172.21.15.61:0/1763945319 >> 172.21.15.61:0/1763945319 conn(0x5613f4ccc6d0 msgr2=0x5613f4d6bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 -- 172.21.15.61:0/1763945319 shutdown_connections 2023-10-18T09:18:31.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.840+0000 7f7555bb8b80 1 -- 172.21.15.61:0/1763945319 wait complete. 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 -- start start 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 Processor -- start 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 0x5613f4d66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4dc0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4d66e20 con 0x5613f4d59be0 2023-10-18T09:18:31.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4dc1170 con 0x5613f4d59280 2023-10-18T09:18:31.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613f4dc12b0 con 0x5613f4d58580 2023-10-18T09:18:31.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.841+0000 7f7555bb8b80 10 client.0 _ll_register_callbacks cb 0x5613f4b91a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:31.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753bfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4dc0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43698/0 (socket says 172.21.15.61:43698) 2023-10-18T09:18:31.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 -- 172.21.15.61:0/685882774 learned_addr learned my addr 172.21.15.61:0/685882774 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:31.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753b7fe700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 0x5613f4d66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 -- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 msgr2=0x5613f4d66260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 0x5613f4d66260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 -- 172.21.15.61:0/685882774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 msgr2=0x5613f4dc0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:31.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4dc0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 -- 172.21.15.61:0/685882774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7534008570 con 0x5613f4d59280 2023-10-18T09:18:31.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753b7fe700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 0x5613f4d66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f753affd700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7530003730 tx=0x7f7530005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a590f9030ca51e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f751affd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530028110 con 0x5613f4d59280 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f752c09e2b0 con 0x5613f4d59280 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.842+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7530028270 con 0x5613f4d59280 2023-10-18T09:18:31.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7530028430 con 0x5613f4d59280 2023-10-18T09:18:31.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 10 client.25526 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:31.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5613f4ad5d90 con 0x5613f4d59280 2023-10-18T09:18:31.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f752c09e2b0 con 0x5613f4d59280 2023-10-18T09:18:31.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7530003a70 con 0x5613f4d59280 2023-10-18T09:18:31.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 10 client.25526 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:31.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 10 client.25526 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-18T09:18:31.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 20 client.25526 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:31.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4dc6680 0x5613f4dc2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:31.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 4 client.25526 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:31.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.843+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5613f4ca3e00 con 0x5613f4dc6680 2023-10-18T09:18:31.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f753b7fe700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4dc6680 0x5613f4dc2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:31.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f753b7fe700 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4dc6680 0x5613f4dc2ae0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be0f55c1c953e039 server_cookie=e812ae092478a9c9 in_seq=0 out_seq=0 2023-10-18T09:18:31.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f751affd700 10 client.25526 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:31.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f751affd700 1 -- 172.21.15.61:0/685882774 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5613f4ca3e00 con 0x5613f4dc6680 2023-10-18T09:18:31.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f751affd700 10 client.25526 handle_command_reply: tid=0 2023-10-18T09:18:31.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 client.25526 shutdown 2023-10-18T09:18:31.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 msgr2=0x5613f4d667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7530003730 tx=0x7f7530005780 comp rx=0 tx=0).stop 2023-10-18T09:18:31.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 shutdown_connections 2023-10-18T09:18:31.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5613f4dc6680 0x5613f4dc2ae0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5613f4d59be0 0x5613f4dc0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5613f4d58580 0x5613f4d66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f751affd700 0 client.25526 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:18:31.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 --2- 172.21.15.61:0/685882774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5613f4d59280 0x5613f4d667a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.844+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 >> 172.21.15.61:0/685882774 conn(0x5613f4ccc6d0 msgr2=0x5613f4d5c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 shutdown_connections 2023-10-18T09:18:31.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f7555bb8b80 1 -- 172.21.15.61:0/685882774 wait complete. 2023-10-18T09:18:31.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f7555bb8b80 20 client.25526 trim_cache size 0 max 16384 2023-10-18T09:18:31.862 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:31.863 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:31.863 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 1, 2023-10-18T09:18:31.863 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24676, 2023-10-18T09:18:31.863 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:31.863 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:31.864 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:31.864 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 95.931214041000004, 2023-10-18T09:18:31.864 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:31.864 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 92, 2023-10-18T09:18:31.865 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:31.865 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 140.339549895 2023-10-18T09:18:31.865 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:31.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 msgr2=0x7f75240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 0x7f75240a39c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7534009ec0 tx=0x7f75340092f0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 msgr2=0x7f753c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:31.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e970 tx=0x7f752c00ec80 comp rx=0 tx=0).stop 2023-10-18T09:18:31.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 shutdown_connections 2023-10-18T09:18:31.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f75240a1500 0x7f75240a39c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f753c1022b0 0x7f753c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f753c100720 0x7f753c198930 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 --2- 172.21.15.61:0/2251475554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f753c101910 0x7f753c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.845+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 >> 172.21.15.61:0/2251475554 conn(0x7f753c0fbc90 msgr2=0x7f753c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.846+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 shutdown_connections 2023-10-18T09:18:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:31.846+0000 7f75421ff700 1 -- 172.21.15.61:0/2251475554 wait complete. 2023-10-18T09:18:31.869 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-18T09:18:32.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/435761732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf281085f0 msgr2=0x7fcf2810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/435761732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf281085f0 0x7fcf2810aa60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009f90 tx=0x7fcf2000b330 comp rx=0 tx=0).stop 2023-10-18T09:18:32.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/435761732 shutdown_connections 2023-10-18T09:18:32.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/435761732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf281085f0 0x7fcf2810aa60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/435761732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf280747d0 0x7fcf28072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/435761732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 0x7fcf28073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/435761732 >> 172.21.15.61:0/435761732 conn(0x7fcf280fbc90 msgr2=0x7fcf280fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/435761732 shutdown_connections 2023-10-18T09:18:32.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.072+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/435761732 wait complete. 2023-10-18T09:18:32.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.073+0000 7fcf2d7ff700 1 Processor -- start 2023-10-18T09:18:32.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.073+0000 7fcf2d7ff700 1 -- start start 2023-10-18T09:18:32.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.073+0000 7fcf2d7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 0x7fcf28198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.073+0000 7fcf2d7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf2d7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 0x7fcf2819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf2d7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf281a04a0 con 0x7fcf281085f0 2023-10-18T09:18:32.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf2d7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf281a05e0 con 0x7fcf280747d0 2023-10-18T09:18:32.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf2d7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf281a0770 con 0x7fcf280731f0 2023-10-18T09:18:32.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 0x7fcf2819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43724/0 (socket says 172.21.15.61:43724) 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2475018719 learned_addr learned my addr 172.21.15.61:0/2475018719 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 0x7fcf28198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2475018719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 msgr2=0x7fcf28198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 0x7fcf28198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 msgr2=0x7fcf2819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 0x7fcf2819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2475018719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf20009c40 con 0x7fcf280747d0 2023-10-18T09:18:32.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.074+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 0x7fcf2819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9a0 tx=0x7fcf1000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf24ff9700 1 -- 172.21.15.61:0/2475018719 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10009790 con 0x7fcf280747d0 2023-10-18T09:18:32.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf281a0a50 con 0x7fcf280747d0 2023-10-18T09:18:32.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf24ff9700 1 -- 172.21.15.61:0/2475018719 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1000ce80 con 0x7fcf280747d0 2023-10-18T09:18:32.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf24ff9700 1 -- 172.21.15.61:0/2475018719 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10018680 con 0x7fcf280747d0 2023-10-18T09:18:32.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.075+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf281a0f60 con 0x7fcf280747d0 2023-10-18T09:18:32.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.078+0000 7fcf24ff9700 1 -- 172.21.15.61:0/2475018719 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcf10010430 con 0x7fcf280747d0 2023-10-18T09:18:32.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.079+0000 7fcf41059b80 1 Processor -- start 2023-10-18T09:18:32.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.079+0000 7fcf41059b80 1 -- start start 2023-10-18T09:18:32.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.079+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.079+0000 7fcf24ff9700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 0x7fcf140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf41059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf939690 con 0x55ffdf9c7be0 2023-10-18T09:18:32.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf41059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf91fc10 con 0x55ffdf9c7280 2023-10-18T09:18:32.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf41059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf91d530 con 0x55ffdf9c6580 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf24ff9700 1 -- 172.21.15.61:0/2475018719 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcf10014070 con 0x7fcf280747d0 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43740/0 (socket says 172.21.15.61:43740) 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf277fe700 1 -- 172.21.15.61:0/2546203826 learned_addr learned my addr 172.21.15.61:0/2546203826 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:32.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 0x7fcf140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2546203826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 msgr2=0x55ffdf9c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2546203826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 msgr2=0x55ffdf9c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf1ffff700 1 -- 172.21.15.61:0/2546203826 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffdf8d63a0 con 0x55ffdf9c7be0 2023-10-18T09:18:32.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.080+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9c6040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c5b0 tx=0x7fcf1009a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cb71285bb88465 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf1d7fa700 1 -- 172.21.15.61:0/2546203826 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1009f080 con 0x55ffdf9c7be0 2023-10-18T09:18:32.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf1d7fa700 1 -- 172.21.15.61:0/2546203826 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf100cae90 con 0x55ffdf9c7be0 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf1d7fa700 1 -- 172.21.15.61:0/2546203826 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1009f8d0 con 0x55ffdf9c7be0 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf41059b80 1 -- 172.21.15.61:0/2546203826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 msgr2=0x55ffdf9c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 0x7fcf140a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005fd0 tx=0x7fcf18009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.081+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9c6040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c5b0 tx=0x7fcf1009a0f0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 -- 172.21.15.61:0/2546203826 shutdown_connections 2023-10-18T09:18:32.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2546203826 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9c6040 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 -- 172.21.15.61:0/2546203826 >> 172.21.15.61:0/2546203826 conn(0x55ffdf93a6d0 msgr2=0x55ffdf9c0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 -- 172.21.15.61:0/2546203826 shutdown_connections 2023-10-18T09:18:32.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 -- 172.21.15.61:0/2546203826 wait complete. 2023-10-18T09:18:32.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 -- start start 2023-10-18T09:18:32.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.082+0000 7fcf41059b80 1 Processor -- start 2023-10-18T09:18:32.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf918e20 con 0x55ffdf9c6580 2023-10-18T09:18:32.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf8b56c0 con 0x55ffdf9c7280 2023-10-18T09:18:32.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf8b5360 con 0x55ffdf9c7be0 2023-10-18T09:18:32.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf41059b80 10 client.0 _ll_register_callbacks cb 0x55ffdf7ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:32.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf26ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf26ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57784/0 (socket says 172.21.15.61:57784) 2023-10-18T09:18:32.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf26ffd700 1 -- 172.21.15.61:0/2163155511 learned_addr learned my addr 172.21.15.61:0/2163155511 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:32.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.083+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 -- 172.21.15.61:0/2163155511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 msgr2=0x55ffdf9d7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 -- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 msgr2=0x55ffdf9cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 -- 172.21.15.61:0/2163155511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2810b680 con 0x55ffdf9c6580 2023-10-18T09:18:32.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf18003900 tx=0x7fcf180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22995763780bd40d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf0e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1801e830 con 0x55ffdf9c6580 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffdf91dea0 con 0x55ffdf9c6580 2023-10-18T09:18:32.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.084+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcf1801e990 con 0x55ffdf9c6580 2023-10-18T09:18:32.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf18004580 con 0x55ffdf9c6580 2023-10-18T09:18:32.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 10 client.15555 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:32.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fcf281a0f60 con 0x55ffdf9c6580 2023-10-18T09:18:32.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffdf91dea0 con 0x55ffdf9c6580 2023-10-18T09:18:32.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf18024020 con 0x55ffdf9c6580 2023-10-18T09:18:32.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:32.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 10 client.15555 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:18:32.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 20 client.15555 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:32.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdf9db740 0x55ffdf9da200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 4 client.15555 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.085+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffdf995860 con 0x55ffdf9db740 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.086+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdf9db740 0x55ffdf9da200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.086+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdf9db740 0x55ffdf9da200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c103b5ceb142b82 server_cookie=fbc18346cc211be6 in_seq=0 out_seq=0 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.086+0000 7fcf0e7fc700 10 client.15555 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/2163155511 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ffdf995860 con 0x55ffdf9db740 2023-10-18T09:18:32.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf0e7fc700 10 client.15555 handle_command_reply: tid=0 2023-10-18T09:18:32.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 client.15555 shutdown 2023-10-18T09:18:32.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 msgr2=0x55ffdf9cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf18003900 tx=0x7fcf180047c0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 shutdown_connections 2023-10-18T09:18:32.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdf9db740 0x55ffdf9da200 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c6580 0x55ffdf9cfa80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf0e7fc700 0 client.15555 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:32.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2163155511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7280 0x55ffdf9cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 >> 172.21.15.61:0/2163155511 conn(0x55ffdf93a6d0 msgr2=0x55ffdf9c5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 shutdown_connections 2023-10-18T09:18:32.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 1 -- 172.21.15.61:0/2163155511 wait complete. 2023-10-18T09:18:32.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.087+0000 7fcf41059b80 20 client.15555 trim_cache size 0 max 16384 2023-10-18T09:18:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 Processor -- start 2023-10-18T09:18:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 -- start start 2023-10-18T09:18:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c6580 0x55ffdf9ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf7998d0 con 0x55ffdf9c7be0 2023-10-18T09:18:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf9d8d90 con 0x55ffdf9c6580 2023-10-18T09:18:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf41059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdf8d0720 con 0x55ffdf9c7280 2023-10-18T09:18:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57790/0 (socket says 172.21.15.61:57790) 2023-10-18T09:18:32.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf277fe700 1 -- 172.21.15.61:0/2059455216 learned_addr learned my addr 172.21.15.61:0/2059455216 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:32.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c6580 0x55ffdf9ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.106+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 -- 172.21.15.61:0/2059455216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 msgr2=0x55ffdf9d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 -- 172.21.15.61:0/2059455216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c6580 msgr2=0x55ffdf9ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c6580 0x55ffdf9ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 -- 172.21.15.61:0/2059455216 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffdf743d90 con 0x55ffdf9c7be0 2023-10-18T09:18:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf277fe700 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009f90 tx=0x7fcf2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a51838fd1e52176 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf1dffb700 1 -- 172.21.15.61:0/2059455216 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000e040 con 0x55ffdf9c7be0 2023-10-18T09:18:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf1dffb700 1 -- 172.21.15.61:0/2059455216 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2000bbf0 con 0x55ffdf9c7be0 2023-10-18T09:18:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.107+0000 7fcf1dffb700 1 -- 172.21.15.61:0/2059455216 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2001f6a0 con 0x55ffdf9c7be0 2023-10-18T09:18:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- 172.21.15.61:0/2059455216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 msgr2=0x55ffdf9cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009f90 tx=0x7fcf2000b330 comp rx=0 tx=0).stop 2023-10-18T09:18:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- 172.21.15.61:0/2059455216 shutdown_connections 2023-10-18T09:18:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9cf830 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c6580 0x55ffdf9ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 --2- 172.21.15.61:0/2059455216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- 172.21.15.61:0/2059455216 >> 172.21.15.61:0/2059455216 conn(0x55ffdf93a6d0 msgr2=0x55ffdf9ca280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- 172.21.15.61:0/2059455216 shutdown_connections 2023-10-18T09:18:32.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- 172.21.15.61:0/2059455216 wait complete. 2023-10-18T09:18:32.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.108+0000 7fcf41059b80 1 -- start start 2023-10-18T09:18:32.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 Processor -- start 2023-10-18T09:18:32.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdfa2f130 con 0x55ffdf9c7280 2023-10-18T09:18:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdfa2f270 con 0x55ffdf9c7be0 2023-10-18T09:18:32.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffdfa2f3b0 con 0x55ffdf9c6580 2023-10-18T09:18:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf41059b80 10 client.0 _ll_register_callbacks cb 0x55ffdf7ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf1ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf26ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf26ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:55824/0 (socket says 172.21.15.61:55824) 2023-10-18T09:18:32.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf26ffd700 1 -- 172.21.15.61:0/1643052167 learned_addr learned my addr 172.21.15.61:0/1643052167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57800/0 (socket says 172.21.15.61:57800) 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.109+0000 7fcf277fe700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 -- 172.21.15.61:0/1643052167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 msgr2=0x55ffdf9d6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 -- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 msgr2=0x55ffdf9d40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 -- 172.21.15.61:0/1643052167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf20008e90 con 0x55ffdf9c7280 2023-10-18T09:18:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf277fe700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf1ffff700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf100cd1f0 tx=0x7fcf100cd180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ad78dcf5017080 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf0e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1009f800 con 0x55ffdf9c7280 2023-10-18T09:18:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcf1009a8f0 con 0x55ffdf9c7280 2023-10-18T09:18:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf100cae90 con 0x55ffdf9c7280 2023-10-18T09:18:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf100a2580 con 0x55ffdf9c7280 2023-10-18T09:18:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf41059b80 10 client.15561 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.110+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ffdf72cc10 con 0x55ffdf9c7280 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf10063070 con 0x55ffdf9c7280 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcf100996e0 con 0x55ffdf9c7280 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 10 client.15561 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-18T09:18:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 20 client.15561 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdfa34420 0x55ffdfa30850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 4 client.15561 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-18T09:18:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ffdf911e00 con 0x55ffdfa34420 2023-10-18T09:18:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.111+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdfa34420 0x55ffdfa30850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf26ffd700 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdfa34420 0x55ffdfa30850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3575eb5ecbaf5b3 server_cookie=352996f2a644736d in_seq=0 out_seq=0 2023-10-18T09:18:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf0e7fc700 10 client.15561 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf0e7fc700 1 -- 172.21.15.61:0/1643052167 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ffdf911e00 con 0x55ffdfa34420 2023-10-18T09:18:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf0e7fc700 10 client.15561 handle_command_reply: tid=0 2023-10-18T09:18:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 client.15561 shutdown 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 msgr2=0x55ffdf9d3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcf100cd1f0 tx=0x7fcf100cd180 comp rx=0 tx=0).stop 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 shutdown_connections 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ffdfa34420 0x55ffdfa30850 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ffdf9c7be0 0x55ffdf9d40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffdf9c6580 0x55ffdf9d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 --2- 172.21.15.61:0/1643052167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ffdf9c7280 0x55ffdf9d3b20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf0e7fc700 0 client.15561 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 >> 172.21.15.61:0/1643052167 conn(0x55ffdf93a6d0 msgr2=0x55ffdf9cf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 shutdown_connections 2023-10-18T09:18:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.112+0000 7fcf41059b80 1 -- 172.21.15.61:0/1643052167 wait complete. 2023-10-18T09:18:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf41059b80 20 client.15561 trim_cache size 0 max 16384 2023-10-18T09:18:32.130 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:32.130 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_fsid": "c578e0ac-6d95-11ee-8db6-212e2dc638e7", 2023-10-18T09:18:32.130 INFO:teuthology.orchestra.run.smithi061.stdout: "whoami": 0, 2023-10-18T09:18:32.131 INFO:teuthology.orchestra.run.smithi061.stdout: "id": 24692, 2023-10-18T09:18:32.131 INFO:teuthology.orchestra.run.smithi061.stdout: "want_state": "up:active", 2023-10-18T09:18:32.131 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "up:active", 2023-10-18T09:18:32.131 INFO:teuthology.orchestra.run.smithi061.stdout: "fs_name": "cephfs", 2023-10-18T09:18:32.131 INFO:teuthology.orchestra.run.smithi061.stdout: "rank_uptime": 125.264421687, 2023-10-18T09:18:32.132 INFO:teuthology.orchestra.run.smithi061.stdout: "mdsmap_epoch": 28, 2023-10-18T09:18:32.132 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch": 92, 2023-10-18T09:18:32.132 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_epoch_barrier": 88, 2023-10-18T09:18:32.132 INFO:teuthology.orchestra.run.smithi061.stdout: "uptime": 131.90480134399999 2023-10-18T09:18:32.132 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 msgr2=0x7fcf140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 0x7fcf140a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf18005fd0 tx=0x7fcf18009310 comp rx=0 tx=0).stop 2023-10-18T09:18:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 msgr2=0x7fcf28198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000e9a0 tx=0x7fcf1000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 shutdown_connections 2023-10-18T09:18:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcf140a1480 0x7fcf140a3940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcf281085f0 0x7fcf2819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf280731f0 0x7fcf28198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 --2- 172.21.15.61:0/2475018719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcf280747d0 0x7fcf28198eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 >> 172.21.15.61:0/2475018719 conn(0x7fcf280fbc90 msgr2=0x7fcf28104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 shutdown_connections 2023-10-18T09:18:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:32.113+0000 7fcf2d7ff700 1 -- 172.21.15.61:0/2475018719 wait complete. 2023-10-18T09:18:32.136 INFO:teuthology.run_tasks:Running task kclient... 2023-10-18T09:18:32.151 INFO:tasks.kclient:Mounting kernel clients... 2023-10-18T09:18:32.151 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-18T09:18:32.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:32.152 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-10-18T09:18:32.198 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-18T09:18:32.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:32.199 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-18T09:18:32.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:32.309 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link delete ceph-brx 2023-10-18T09:18:32.425 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2023-10-18T09:18:32.425 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-18T09:18:32.426 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-10-18T09:18:32.426 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-10-18T09:18:32.426 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-18T09:18:33.177 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:18:34.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:33 smithi110 ceph-mon[134818]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2023-10-18T09:18:34.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:33 smithi061 ceph-mon[98573]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2023-10-18T09:18:34.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:33 smithi170 ceph-mon[135150]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 204 B/s wr, 10 op/s 2023-10-18T09:18:34.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 -- 172.21.15.61:0/1470135317 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e58101260 msgr2=0x7f9e5810d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:34.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/1470135317 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e58101260 0x7f9e5810d650 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b0a0 tx=0x7f9e5000b3b0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 -- 172.21.15.61:0/1470135317 shutdown_connections 2023-10-18T09:18:34.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/1470135317 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e58101260 0x7f9e5810d650 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/1470135317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e581057c0 0x7f9e58105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/1470135317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 -- 172.21.15.61:0/1470135317 >> 172.21.15.61:0/1470135317 conn(0x7f9e580779e0 msgr2=0x7f9e58077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:34.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 -- 172.21.15.61:0/1470135317 shutdown_connections 2023-10-18T09:18:34.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.701+0000 7f9e5cad9700 1 -- 172.21.15.61:0/1470135317 wait complete. 2023-10-18T09:18:34.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.702+0000 7f9e5cad9700 1 Processor -- start 2023-10-18T09:18:34.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.702+0000 7f9e5cad9700 1 -- start start 2023-10-18T09:18:34.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.702+0000 7f9e5cad9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:34.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.702+0000 7f9e5cad9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e58101260 0x7f9e58199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:34.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5cad9700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 0x7f9e5819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:34.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5cad9700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819de10 con 0x7f9e581057c0 2023-10-18T09:18:34.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5cad9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819df80 con 0x7f9e581008a0 2023-10-18T09:18:34.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5cad9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819e140 con 0x7f9e58101260 2023-10-18T09:18:34.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:34.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43784/0 (socket says 172.21.15.61:43784) 2023-10-18T09:18:34.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 -- 172.21.15.61:0/2778001607 learned_addr learned my addr 172.21.15.61:0/2778001607 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:34.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e56d9d700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 0x7f9e5819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:34.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e55d9b700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e58101260 0x7f9e58199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:34.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 -- 172.21.15.61:0/2778001607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e58101260 msgr2=0x7f9e58199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:34.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e58101260 0x7f9e58199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 -- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 msgr2=0x7f9e5819d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:34.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 0x7f9e5819d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e5659c700 1 -- 172.21.15.61:0/2778001607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50009d00 con 0x7f9e581008a0 2023-10-18T09:18:34.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.703+0000 7f9e56d9d700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 0x7f9e5819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:34.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e5659c700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e970 tx=0x7f9e4800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:34.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48009790 con 0x7f9e581008a0 2023-10-18T09:18:34.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e480044e0 con 0x7f9e581008a0 2023-10-18T09:18:34.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5819e420 con 0x7f9e581008a0 2023-10-18T09:18:34.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003e30 con 0x7f9e581008a0 2023-10-18T09:18:34.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.704+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e581a5c80 con 0x7f9e581008a0 2023-10-18T09:18:34.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.705+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e58101f30 con 0x7f9e581008a0 2023-10-18T09:18:34.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.707+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f9e48003f90 con 0x7f9e581008a0 2023-10-18T09:18:34.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.708+0000 7f9e477fe700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 0x7f9e400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:34.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.709+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9e48014070 con 0x7f9e581008a0 2023-10-18T09:18:34.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.709+0000 7f9e55d9b700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 0x7f9e400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:34.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.709+0000 7f9e55d9b700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 0x7f9e400a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9e5819a470 tx=0x7f9e4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:34.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.711+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9e48096780 con 0x7f9e581008a0 2023-10-18T09:18:34.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.970+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9e581107a0 con 0x7f9e581008a0 2023-10-18T09:18:34.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.970+0000 7f9e477fe700 1 -- 172.21.15.61:0/2778001607 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9e48099da0 con 0x7f9e581008a0 2023-10-18T09:18:34.973 INFO:teuthology.orchestra.run.smithi061.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-18T09:18:34.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 msgr2=0x7f9e400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:34.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 0x7f9e400a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9e5819a470 tx=0x7f9e4c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:34.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 msgr2=0x7f9e58198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:34.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800e970 tx=0x7f9e4800ed30 comp rx=0 tx=0).stop 2023-10-18T09:18:34.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 shutdown_connections 2023-10-18T09:18:34.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f9e400a1480 0x7f9e400a3940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9e581057c0 0x7f9e5819d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581008a0 0x7f9e58198f10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 --2- 172.21.15.61:0/2778001607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e58101260 0x7f9e58199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:34.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 >> 172.21.15.61:0/2778001607 conn(0x7f9e580779e0 msgr2=0x7f9e580fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:34.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 shutdown_connections 2023-10-18T09:18:34.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:34.973+0000 7f9e5cad9700 1 -- 172.21.15.61:0/2778001607 wait complete. 2023-10-18T09:18:35.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-18T09:18:35.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-18T09:18:35.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi061.front.sepia.ceph.com 2023-10-18T09:18:35.570 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-18T09:18:35.571 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-10-18T09:18:35.571 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-18T09:18:35.571 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-18T09:18:35.571 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-18T09:18:35.572 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-10-18T09:18:35.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:35.572 DEBUG:teuthology.orchestra.run.smithi061:> ip addr 2023-10-18T09:18:35.618 INFO:teuthology.orchestra.run.smithi061.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-18T09:18:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-18T09:18:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: inet 127.0.0.1/8 scope host lo 2023-10-18T09:18:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:18:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 ::1/128 scope host 2023-10-18T09:18:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:18:35.619 INFO:teuthology.orchestra.run.smithi061.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:18:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0c brd ff:ff:ff:ff:ff:ff 2023-10-18T09:18:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: altname enp7s0f0 2023-10-18T09:18:35.620 INFO:teuthology.orchestra.run.smithi061.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:18:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:a8:bb:0d brd ff:ff:ff:ff:ff:ff 2023-10-18T09:18:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: altname enp7s0f1 2023-10-18T09:18:35.620 INFO:teuthology.orchestra.run.smithi061.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-18T09:18:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b8 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:18:35.621 INFO:teuthology.orchestra.run.smithi061.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-18T09:18:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: link/ether 0c:c4:7a:b6:f8:b9 brd ff:ff:ff:ff:ff:ff 2023-10-18T09:18:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: inet 172.21.15.61/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-18T09:18:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft 41571sec preferred_lft 41571sec 2023-10-18T09:18:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: inet6 fe80::e081:b724:74b9:7104/64 scope link noprefixroute 2023-10-18T09:18:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: valid_lft forever preferred_lft forever 2023-10-18T09:18:35.623 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-18T09:18:35.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:35.624 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:18:35.624 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:18:35.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add name ceph-brx type bridge 2023-10-18T09:18:35.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr flush dev ceph-brx 2023-10-18T09:18:35.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set ceph-brx up 2023-10-18T09:18:35.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-18T09:18:35.625 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:18:35.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:35.874 DEBUG:teuthology.orchestra.run.smithi061:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-18T09:18:35.944 INFO:teuthology.orchestra.run.smithi061.stdout:1 2023-10-18T09:18:35.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:35.948 DEBUG:teuthology.orchestra.run.smithi061:> route 2023-10-18T09:18:35.995 INFO:teuthology.orchestra.run.smithi061.stdout:Kernel IP routing table 2023-10-18T09:18:35.996 INFO:teuthology.orchestra.run.smithi061.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-18T09:18:35.996 INFO:teuthology.orchestra.run.smithi061.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-18T09:18:35.996 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-18T09:18:35.997 INFO:teuthology.orchestra.run.smithi061.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-18T09:18:35.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:35.998 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:18:35.999 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:18:35.999 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-18T09:18:35.999 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-18T09:18:36.000 DEBUG:teuthology.orchestra.run.smithi061:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-18T09:18:36.000 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:18:36.104 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:35 smithi061 ceph-mon[98573]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 171 B/s wr, 9 op/s 2023-10-18T09:18:36.104 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:35 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2778001607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:18:36.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:36.176 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-10-18T09:18:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:35 smithi110 ceph-mon[134818]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 171 B/s wr, 9 op/s 2023-10-18T09:18:36.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:35 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2778001607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:18:36.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:36.222 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list-id 2023-10-18T09:18:36.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:36.306 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:18:36.306 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:18:36.306 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns add ceph-ns--tmp-cephfs 2023-10-18T09:18:36.307 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-10-18T09:18:36.307 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:18:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:35 smithi170 ceph-mon[135150]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 171 B/s wr, 9 op/s 2023-10-18T09:18:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:35 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2778001607' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-18T09:18:36.491 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-10-18T09:18:36.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:36.492 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:18:36.493 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:18:36.493 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-10-18T09:18:36.493 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-18T09:18:36.493 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-10-18T09:18:36.494 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-10-18T09:18:36.494 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-10-18T09:18:36.494 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:18:36.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:36.749 DEBUG:teuthology.orchestra.run.smithi061:> (cd / && exec bash -c ' 2023-10-18T09:18:36.749 DEBUG:teuthology.orchestra.run.smithi061:> set -e 2023-10-18T09:18:36.749 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set brx.0 up 2023-10-18T09:18:36.750 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link set dev brx.0 master ceph-brx 2023-10-18T09:18:36.750 DEBUG:teuthology.orchestra.run.smithi061:> ') 2023-10-18T09:18:36.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:18:36.928 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -v /tmp/cephfs 2023-10-18T09:18:36.972 INFO:teuthology.orchestra.run.smithi061.stdout:mkdir: created directory '/tmp/cephfs' 2023-10-18T09:18:36.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-18T09:18:36.973 DEBUG:teuthology.orchestra.run.smithi061:> chmod 0000 /tmp/cephfs 2023-10-18T09:18:37.056 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-18T09:18:37.057 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-18T09:18:37.057 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/916b255b-5c6f-497a-bbc4-d85b7266a4d4 2023-10-18T09:18:37.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:37.058 DEBUG:teuthology.orchestra.run.smithi061:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/916b255b-5c6f-497a-bbc4-d85b7266a4d4 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2023-10-18T09:18:37.204 INFO:teuthology.orchestra.run.smithi061.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-18T09:18:37.205 INFO:teuthology.orchestra.run.smithi061.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-18T09:18:37.668 INFO:teuthology.orchestra.run.smithi061.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2023-10-18T09:18:37.668 INFO:teuthology.orchestra.run.smithi061.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2023-10-18T09:18:37.669 INFO:teuthology.orchestra.run.smithi061.stdout:mount.ceph: resolved to: "172.21.15.61:6789,172.21.15.110:6789,172.21.15.170:6789" 2023-10-18T09:18:37.669 INFO:teuthology.orchestra.run.smithi061.stdout:mount.ceph: trying mount with new device syntax: 0@c578e0ac-6d95-11ee-8db6-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/916b255b-5c6f-497a-bbc4-d85b7266a4d4 2023-10-18T09:18:37.669 INFO:teuthology.orchestra.run.smithi061.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.61:6789/172.21.15.110:6789/172.21.15.170:6789" will pass to kernel 2023-10-18T09:18:37.675 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-18T09:18:37.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:37.676 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 1777 /tmp/cephfs 2023-10-18T09:18:37.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:37.750 DEBUG:teuthology.orchestra.run.smithi061:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:18:37.838 INFO:teuthology.orchestra.run.smithi061.stdout:client15570 2023-10-18T09:18:37.838 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:18:37.839 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:18:37.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:37.884 DEBUG:teuthology.orchestra.run.smithi061:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:18:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:37 smithi170 ceph-mon[135150]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-18T09:18:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:37 smithi110 ceph-mon[134818]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-18T09:18:38.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:37 smithi061 ceph-mon[98573]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 0 B/s wr, 11 op/s 2023-10-18T09:18:38.011 INFO:teuthology.orchestra.run.smithi061.stdout:client15570 2023-10-18T09:18:38.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:38.012 DEBUG:teuthology.orchestra.run.smithi061:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-18T09:18:38.099 INFO:teuthology.orchestra.run.smithi061.stdout:client15570 2023-10-18T09:18:38.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:18:38.100 DEBUG:teuthology.orchestra.run.smithi061:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7.client15570/status') 2023-10-18T09:18:38.209 INFO:teuthology.orchestra.run.smithi061.stdout:instance: client.15570 (3)192.168.0.1:0/3735001853 2023-10-18T09:18:38.210 INFO:teuthology.orchestra.run.smithi061.stdout:blocklisted: false 2023-10-18T09:18:38.210 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records in 2023-10-18T09:18:38.210 INFO:teuthology.orchestra.run.smithi061.stderr:0+1 records out 2023-10-18T09:18:38.211 INFO:teuthology.orchestra.run.smithi061.stderr:70 bytes copied, 5.8027e-05 s, 1.2 MB/s 2023-10-18T09:18:38.213 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-18T09:18:38.225 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-18T09:18:38.237 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:38.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.439+0000 7f05053ff700 1 -- 172.21.15.61:0/439309034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05001022d0 msgr2=0x7f0500108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.439+0000 7f05053ff700 1 --2- 172.21.15.61:0/439309034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05001022d0 0x7f0500108b80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b0d0 tx=0x7f04f800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 -- 172.21.15.61:0/439309034 shutdown_connections 2023-10-18T09:18:38.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 --2- 172.21.15.61:0/439309034 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f05001022d0 0x7f0500108b80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 --2- 172.21.15.61:0/439309034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0500100740 0x7f0500100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 --2- 172.21.15.61:0/439309034 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500101930 0x7f0500101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 -- 172.21.15.61:0/439309034 >> 172.21.15.61:0/439309034 conn(0x7f05000fbc90 msgr2=0x7f05000fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:38.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 -- 172.21.15.61:0/439309034 shutdown_connections 2023-10-18T09:18:38.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.440+0000 7f05053ff700 1 -- 172.21.15.61:0/439309034 wait complete. 2023-10-18T09:18:38.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 Processor -- start 2023-10-18T09:18:38.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 -- start start 2023-10-18T09:18:38.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500100740 0x7f05001967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 0x7f050019dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050019e2f0 con 0x7f0500101930 2023-10-18T09:18:38.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050019e430 con 0x7f0500100740 2023-10-18T09:18:38.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.441+0000 7f05053ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f050019e5c0 con 0x7f05001022d0 2023-10-18T09:18:38.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04fe7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04fe7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57836/0 (socket says 172.21.15.61:57836) 2023-10-18T09:18:38.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 0x7f050019dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04feffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500100740 0x7f05001967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04fe7fc700 1 -- 172.21.15.61:0/812416488 learned_addr learned my addr 172.21.15.61:0/812416488 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:38.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 -- 172.21.15.61:0/812416488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500100740 msgr2=0x7f05001967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500100740 0x7f05001967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 -- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 msgr2=0x7f0500196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 -- 172.21.15.61:0/812416488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f8009d00 con 0x7f05001022d0 2023-10-18T09:18:38.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04fe7fc700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:38.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.442+0000 7f04ff7fe700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 0x7f050019dd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f04f8007b60 tx=0x7f04f8008e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:38.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.443+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f800e040 con 0x7f05001022d0 2023-10-18T09:18:38.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.443+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050019e840 con 0x7f05001022d0 2023-10-18T09:18:38.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.443+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050019ecc0 con 0x7f05001022d0 2023-10-18T09:18:38.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.443+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04f8005070 con 0x7f05001022d0 2023-10-18T09:18:38.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.443+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f801d6d0 con 0x7f05001022d0 2023-10-18T09:18:38.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.444+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0500107af0 con 0x7f05001022d0 2023-10-18T09:18:38.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.446+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f04f80054a0 con 0x7f05001022d0 2023-10-18T09:18:38.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.447+0000 7f04e7fff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 0x7f04e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.448+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f04f80ce480 con 0x7f05001022d0 2023-10-18T09:18:38.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.448+0000 7f04feffd700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 0x7f04e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.448+0000 7f04feffd700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 0x7f04e80a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0500101790 tx=0x7f04f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:38.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.450+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f04f8097850 con 0x7f05001022d0 2023-10-18T09:18:38.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.661+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f050010bfb0 con 0x7f05001022d0 2023-10-18T09:18:38.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.663+0000 7f04e7fff700 1 -- 172.21.15.61:0/812416488 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f04f801a3c0 con 0x7f05001022d0 2023-10-18T09:18:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:38.666 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:38.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 msgr2=0x7f04e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 0x7f04e80a3940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0500101790 tx=0x7f04f0009260 comp rx=0 tx=0).stop 2023-10-18T09:18:38.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 msgr2=0x7f050019dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 0x7f050019dd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f04f8007b60 tx=0x7f04f8008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:38.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 shutdown_connections 2023-10-18T09:18:38.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f04e80a1480 0x7f04e80a3940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05001022d0 0x7f050019dd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0500100740 0x7f05001967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 --2- 172.21.15.61:0/812416488 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0500101930 0x7f0500196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 >> 172.21.15.61:0/812416488 conn(0x7f05000fbc90 msgr2=0x7f0500104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 shutdown_connections 2023-10-18T09:18:38.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.665+0000 7f05053ff700 1 -- 172.21.15.61:0/812416488 wait complete. 2023-10-18T09:18:38.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:18:38.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:18:38.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2751890600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 msgr2=0x7f1c14108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2751890600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c14108be0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00b0d0 tx=0x7f1c0c00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2751890600 shutdown_connections 2023-10-18T09:18:38.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2751890600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c14108be0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2751890600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c141007a0 0x7f1c14100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2751890600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c14101990 0x7f1c14101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.896+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2751890600 >> 172.21.15.61:0/2751890600 conn(0x7f1c140fbcf0 msgr2=0x7f1c140fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:38.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.897+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2751890600 shutdown_connections 2023-10-18T09:18:38.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.897+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2751890600 wait complete. 2023-10-18T09:18:38.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.897+0000 7f1c1b7ff700 1 Processor -- start 2023-10-18T09:18:38.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 -- start start 2023-10-18T09:18:38.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 0x7f1c14196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c14101990 0x7f1c14196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1419e360 con 0x7f1c14102330 2023-10-18T09:18:38.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1419e4a0 con 0x7f1c14101990 2023-10-18T09:18:38.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1b7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1419e630 con 0x7f1c141007a0 2023-10-18T09:18:38.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c19d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c1959b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 0x7f1c14196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c19d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34390/0 (socket says 172.21.15.61:34390) 2023-10-18T09:18:38.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c19d9c700 1 -- 172.21.15.61:0/2559086124 learned_addr learned my addr 172.21.15.61:0/2559086124 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:38.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.898+0000 7f1c18d9a700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c14101990 0x7f1c14196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 -- 172.21.15.61:0/2559086124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 msgr2=0x7f1c14196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 0x7f1c14196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 -- 172.21.15.61:0/2559086124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c14101990 msgr2=0x7f1c14196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:38.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c14101990 0x7f1c14196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:38.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 -- 172.21.15.61:0/2559086124 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009d00 con 0x7f1c14102330 2023-10-18T09:18:38.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c1959b700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 0x7f1c14196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:38.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c19d9c700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c007b60 tx=0x7f1c0c008e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:38.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00e040 con 0x7f1c14102330 2023-10-18T09:18:38.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c0c01d930 con 0x7f1c14102330 2023-10-18T09:18:38.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c01b9f0 con 0x7f1c14102330 2023-10-18T09:18:38.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1419e8b0 con 0x7f1c14102330 2023-10-18T09:18:38.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.899+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1419ece0 con 0x7f1c14102330 2023-10-18T09:18:38.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.900+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14107b70 con 0x7f1c14102330 2023-10-18T09:18:38.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.902+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1c0c01daa0 con 0x7f1c14102330 2023-10-18T09:18:38.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.904+0000 7f1c0a7fc700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 0x7f1c000a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:38.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.905+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1c0c0ce3b0 con 0x7f1c14102330 2023-10-18T09:18:38.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.905+0000 7f1c1959b700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 0x7f1c000a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:38.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.905+0000 7f1c1959b700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 0x7f1c000a39c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c141017f0 tx=0x7f1c10009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:38.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:38.909+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1c0c097750 con 0x7f1c14102330 2023-10-18T09:18:39.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:38 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/812416488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:39.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.125+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c1402cf10 con 0x7f1c14102330 2023-10-18T09:18:39.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.125+0000 7f1c0a7fc700 1 -- 172.21.15.61:0/2559086124 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c0c09ad70 con 0x7f1c14102330 2023-10-18T09:18:39.128 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:18:39.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 msgr2=0x7f1c000a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 0x7f1c000a39c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c141017f0 tx=0x7f1c10009260 comp rx=0 tx=0).stop 2023-10-18T09:18:39.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 msgr2=0x7f1c1419dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c007b60 tx=0x7f1c0c008e40 comp rx=0 tx=0).stop 2023-10-18T09:18:39.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 shutdown_connections 2023-10-18T09:18:39.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1c000a1500 0x7f1c000a39c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1c14102330 0x7f1c1419dd90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c141007a0 0x7f1c14196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 --2- 172.21.15.61:0/2559086124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c14101990 0x7f1c14196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.127+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 >> 172.21.15.61:0/2559086124 conn(0x7f1c140fbcf0 msgr2=0x7f1c14104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:39.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.128+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 shutdown_connections 2023-10-18T09:18:39.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.128+0000 7f1c1b7ff700 1 -- 172.21.15.61:0/2559086124 wait complete. 2023-10-18T09:18:39.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:38 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/812416488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:39.149 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:18:39.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:38 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/812416488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:39.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 -- 172.21.15.61:0/2700014612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c102330 msgr2=0x7fe19c108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/2700014612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c102330 0x7fe19c108be0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe19400b0d0 tx=0x7fe19400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 -- 172.21.15.61:0/2700014612 shutdown_connections 2023-10-18T09:18:39.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/2700014612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c102330 0x7fe19c108be0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/2700014612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 0x7fe19c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/2700014612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c101990 0x7fe19c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 -- 172.21.15.61:0/2700014612 >> 172.21.15.61:0/2700014612 conn(0x7fe19c0fbcf0 msgr2=0x7fe19c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:39.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 -- 172.21.15.61:0/2700014612 shutdown_connections 2023-10-18T09:18:39.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.354+0000 7fe1a1bff700 1 -- 172.21.15.61:0/2700014612 wait complete. 2023-10-18T09:18:39.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 Processor -- start 2023-10-18T09:18:39.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 -- start start 2023-10-18T09:18:39.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 0x7fe19c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c102330 0x7fe19c19fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1a05c0 con 0x7fe19c101990 2023-10-18T09:18:39.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1a0700 con 0x7fe19c102330 2023-10-18T09:18:39.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.355+0000 7fe1a1bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c1a0890 con 0x7fe19c1007a0 2023-10-18T09:18:39.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34406/0 (socket says 172.21.15.61:34406) 2023-10-18T09:18:39.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 -- 172.21.15.61:0/1116474789 learned_addr learned my addr 172.21.15.61:0/1116474789 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:39.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19b7fe700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 0x7fe19c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19bfff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c102330 0x7fe19c19fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 -- 172.21.15.61:0/1116474789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 msgr2=0x7fe19c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 0x7fe19c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 -- 172.21.15.61:0/1116474789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c102330 msgr2=0x7fe19c19fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c102330 0x7fe19c19fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 -- 172.21.15.61:0/1116474789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190009520 con 0x7fe19c101990 2023-10-18T09:18:39.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe19affd700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe19000eb30 tx=0x7fe19000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:39.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.356+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190009880 con 0x7fe19c101990 2023-10-18T09:18:39.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.357+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1900044e0 con 0x7fe19c101990 2023-10-18T09:18:39.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.357+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe194009d00 con 0x7fe19c101990 2023-10-18T09:18:39.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.357+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe190003d80 con 0x7fe19c101990 2023-10-18T09:18:39.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.357+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c1a0f40 con 0x7fe19c101990 2023-10-18T09:18:39.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.358+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c0675a0 con 0x7fe19c101990 2023-10-18T09:18:39.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.360+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe190003f20 con 0x7fe19c101990 2023-10-18T09:18:39.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.361+0000 7fe198ff9700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 0x7fe1840a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.361+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe190014070 con 0x7fe19c101990 2023-10-18T09:18:39.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.361+0000 7fe19b7fe700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 0x7fe1840a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.362+0000 7fe19b7fe700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 0x7fe1840a39c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe19c1017f0 tx=0x7fe18c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:39.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.365+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe190096820 con 0x7fe19c101990 2023-10-18T09:18:39.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.576+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe19c1a12f0 con 0x7fe19c101990 2023-10-18T09:18:39.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.577+0000 7fe198ff9700 1 -- 172.21.15.61:0/1116474789 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe190099e40 con 0x7fe19c101990 2023-10-18T09:18:39.580 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:39.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 msgr2=0x7fe1840a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 0x7fe1840a39c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe19c1017f0 tx=0x7fe18c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:39.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 msgr2=0x7fe19c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe19000eb30 tx=0x7fe19000ee40 comp rx=0 tx=0).stop 2023-10-18T09:18:39.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 shutdown_connections 2023-10-18T09:18:39.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe1840a1500 0x7fe1840a39c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe19c102330 0x7fe19c19fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1007a0 0x7fe19c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 --2- 172.21.15.61:0/1116474789 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe19c101990 0x7fe19c199020 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 >> 172.21.15.61:0/1116474789 conn(0x7fe19c0fbcf0 msgr2=0x7fe19c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:39.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 shutdown_connections 2023-10-18T09:18:39.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.579+0000 7fe1a1bff700 1 -- 172.21.15.61:0/1116474789 wait complete. 2023-10-18T09:18:39.600 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:39.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 -- 172.21.15.61:0/540159512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65401007a0 msgr2=0x7f6540100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 --2- 172.21.15.61:0/540159512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65401007a0 0x7f6540100bc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f653c0099c0 tx=0x7f653c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 -- 172.21.15.61:0/540159512 shutdown_connections 2023-10-18T09:18:39.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 --2- 172.21.15.61:0/540159512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 0x7f6540108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 --2- 172.21.15.61:0/540159512 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65401007a0 0x7f6540100bc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 --2- 172.21.15.61:0/540159512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6540101990 0x7f6540101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 -- 172.21.15.61:0/540159512 >> 172.21.15.61:0/540159512 conn(0x7f65400fbcf0 msgr2=0x7f65400fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:39.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 -- 172.21.15.61:0/540159512 shutdown_connections 2023-10-18T09:18:39.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.811+0000 7f6547bff700 1 -- 172.21.15.61:0/540159512 wait complete. 2023-10-18T09:18:39.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.812+0000 7f6547bff700 1 Processor -- start 2023-10-18T09:18:39.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.812+0000 7f6547bff700 1 -- start start 2023-10-18T09:18:39.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.812+0000 7f6547bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 0x7f6540196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.812+0000 7f6547bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.812+0000 7f6547bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 0x7f654019dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f6547bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654019e280 con 0x7f6540101990 2023-10-18T09:18:39.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f6547bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654019e3c0 con 0x7f6540102330 2023-10-18T09:18:39.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f6547bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654019e550 con 0x7f65401007a0 2023-10-18T09:18:39.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654619c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 0x7f654019dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34410/0 (socket says 172.21.15.61:34410) 2023-10-18T09:18:39.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 -- 172.21.15.61:0/3891744316 learned_addr learned my addr 172.21.15.61:0/3891744316 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:39.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654599b700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 0x7f6540196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 -- 172.21.15.61:0/3891744316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 msgr2=0x7f6540196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 0x7f6540196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 -- 172.21.15.61:0/3891744316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 msgr2=0x7f654019dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:39.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 0x7f654019dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:39.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 -- 172.21.15.61:0/3891744316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c0095f0 con 0x7f6540101990 2023-10-18T09:18:39.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654599b700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 0x7f6540196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:39.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.813+0000 7f654519a700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9a0 tx=0x7f653000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:39.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.814+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530009790 con 0x7f6540101990 2023-10-18T09:18:39.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.814+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f653000cbd0 con 0x7f6540101990 2023-10-18T09:18:39.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.814+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654019e830 con 0x7f6540101990 2023-10-18T09:18:39.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.814+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530018680 con 0x7f6540101990 2023-10-18T09:18:39.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.814+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654019ed40 con 0x7f6540101990 2023-10-18T09:18:39.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.815+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6540107bc0 con 0x7f6540101990 2023-10-18T09:18:39.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.817+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f653000cd40 con 0x7f6540101990 2023-10-18T09:18:39.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.818+0000 7f6536ffd700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 0x7f652c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:39.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.818+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6530014070 con 0x7f6540101990 2023-10-18T09:18:39.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.818+0000 7f654599b700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 0x7f652c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:39.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.819+0000 7f654599b700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 0x7f652c0a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f653c009990 tx=0x7f653c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:39.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:39.820+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65300968b0 con 0x7f6540101990 2023-10-18T09:18:40.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:39 smithi061 ceph-mon[98573]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-18T09:18:40.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:39 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2559086124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:39 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1116474789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.029+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f654010c110 con 0x7f6540101990 2023-10-18T09:18:40.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.030+0000 7f6536ffd700 1 -- 172.21.15.61:0/3891744316 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6530099ed0 con 0x7f6540101990 2023-10-18T09:18:40.033 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 msgr2=0x7f652c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 0x7f652c0a3940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f653c009990 tx=0x7f653c01a040 comp rx=0 tx=0).stop 2023-10-18T09:18:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 msgr2=0x7f6540196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f653000e9a0 tx=0x7f653000ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 shutdown_connections 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f652c0a1480 0x7f652c0a3940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6540102330 0x7f654019dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65401007a0 0x7f6540196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 --2- 172.21.15.61:0/3891744316 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6540101990 0x7f6540196c90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 >> 172.21.15.61:0/3891744316 conn(0x7f65400fbcf0 msgr2=0x7f6540104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:40.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 shutdown_connections 2023-10-18T09:18:40.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.032+0000 7f6547bff700 1 -- 172.21.15.61:0/3891744316 wait complete. 2023-10-18T09:18:40.054 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:39 smithi170 ceph-mon[135150]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-18T09:18:40.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:39 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2559086124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:39 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1116474789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:39 smithi110 ceph-mon[134818]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2023-10-18T09:18:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:39 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2559086124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:39 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1116474789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 -- 172.21.15.61:0/739267384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4881007a0 msgr2=0x7fc488100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 --2- 172.21.15.61:0/739267384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4881007a0 0x7fc488100bc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0099c0 tx=0x7fc47c009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 -- 172.21.15.61:0/739267384 shutdown_connections 2023-10-18T09:18:40.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 --2- 172.21.15.61:0/739267384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc488102330 0x7fc488108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 --2- 172.21.15.61:0/739267384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc4881007a0 0x7fc488100bc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 --2- 172.21.15.61:0/739267384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 -- 172.21.15.61:0/739267384 >> 172.21.15.61:0/739267384 conn(0x7fc4880fbcf0 msgr2=0x7fc4880fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:40.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 -- 172.21.15.61:0/739267384 shutdown_connections 2023-10-18T09:18:40.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.259+0000 7fc48edff700 1 -- 172.21.15.61:0/739267384 wait complete. 2023-10-18T09:18:40.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.260+0000 7fc48edff700 1 Processor -- start 2023-10-18T09:18:40.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 -- start start 2023-10-18T09:18:40.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 0x7fc48819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819e230 con 0x7fc488102330 2023-10-18T09:18:40.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819e370 con 0x7fc4881007a0 2023-10-18T09:18:40.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48edff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819e500 con 0x7fc488101990 2023-10-18T09:18:40.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48d39c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 0x7fc48819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48cb9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48cb9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50260/0 (socket says 172.21.15.61:50260) 2023-10-18T09:18:40.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc487fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.261+0000 7fc48cb9b700 1 -- 172.21.15.61:0/734369452 learned_addr learned my addr 172.21.15.61:0/734369452 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:40.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 -- 172.21.15.61:0/734369452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 msgr2=0x7fc488196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 -- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 msgr2=0x7fc48819dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 0x7fc48819dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 -- 172.21.15.61:0/734369452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0095f0 con 0x7fc488101990 2023-10-18T09:18:40.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc48cb9b700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc48d39c700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 0x7fc48819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc487fff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488196c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9f0 tx=0x7fc47400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:40.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47400c9c0 con 0x7fc488101990 2023-10-18T09:18:40.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48819e7e0 con 0x7fc488101990 2023-10-18T09:18:40.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc474010430 con 0x7fc488101990 2023-10-18T09:18:40.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474018680 con 0x7fc488101990 2023-10-18T09:18:40.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.262+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48819ed40 con 0x7fc488101990 2023-10-18T09:18:40.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.263+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc488107bf0 con 0x7fc488101990 2023-10-18T09:18:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.265+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc474010810 con 0x7fc488101990 2023-10-18T09:18:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.267+0000 7fc485ffb700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 0x7fc4780a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.267+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc474014070 con 0x7fc488101990 2023-10-18T09:18:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.267+0000 7fc48cb9b700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 0x7fc4780a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.268+0000 7fc48cb9b700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 0x7fc4780a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004800 tx=0x7fc47c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:40.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.269+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc474052100 con 0x7fc488101990 2023-10-18T09:18:40.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.480+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc48810bfc0 con 0x7fc488101990 2023-10-18T09:18:40.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.481+0000 7fc485ffb700 1 -- 172.21.15.61:0/734369452 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc474096530 con 0x7fc488101990 2023-10-18T09:18:40.483 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:40.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 msgr2=0x7fc4780a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 0x7fc4780a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc47c004800 tx=0x7fc47c01a040 comp rx=0 tx=0).stop 2023-10-18T09:18:40.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 msgr2=0x7fc488196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488196c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9f0 tx=0x7fc47400ed00 comp rx=0 tx=0).stop 2023-10-18T09:18:40.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 shutdown_connections 2023-10-18T09:18:40.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc4780a1480 0x7fc4780a3940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc488102330 0x7fc48819dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4881007a0 0x7fc488196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 --2- 172.21.15.61:0/734369452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc488101990 0x7fc488196c40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 >> 172.21.15.61:0/734369452 conn(0x7fc4880fbcf0 msgr2=0x7fc488104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:40.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 shutdown_connections 2023-10-18T09:18:40.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.483+0000 7fc48edff700 1 -- 172.21.15.61:0/734369452 wait complete. 2023-10-18T09:18:40.505 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:40.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 -- 172.21.15.61:0/2558782687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c0731f0 msgr2=0x7f899c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 --2- 172.21.15.61:0/2558782687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c0731f0 0x7f899c073670 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8990009990 tx=0x7f8990009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 -- 172.21.15.61:0/2558782687 shutdown_connections 2023-10-18T09:18:40.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 --2- 172.21.15.61:0/2558782687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c108610 0x7f899c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 --2- 172.21.15.61:0/2558782687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0747d0 0x7f899c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 --2- 172.21.15.61:0/2558782687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c0731f0 0x7f899c073670 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 -- 172.21.15.61:0/2558782687 >> 172.21.15.61:0/2558782687 conn(0x7f899c0fbc90 msgr2=0x7f899c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:40.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 -- 172.21.15.61:0/2558782687 shutdown_connections 2023-10-18T09:18:40.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.709+0000 7f89a25ff700 1 -- 172.21.15.61:0/2558782687 wait complete. 2023-10-18T09:18:40.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 Processor -- start 2023-10-18T09:18:40.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 -- start start 2023-10-18T09:18:40.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0731f0 0x7f899c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 0x7f899c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1a0420 con 0x7f899c0747d0 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1a0560 con 0x7f899c108610 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.710+0000 7f89a25ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c1a06f0 con 0x7f899c0731f0 2023-10-18T09:18:40.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f899b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f899b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34448/0 (socket says 172.21.15.61:34448) 2023-10-18T09:18:40.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 0x7f899c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f899b7fe700 1 -- 172.21.15.61:0/652036232 learned_addr learned my addr 172.21.15.61:0/652036232 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:40.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f899bfff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0731f0 0x7f899c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 -- 172.21.15.61:0/652036232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0731f0 msgr2=0x7f899c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0731f0 0x7f899c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 -- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 msgr2=0x7f899c198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 -- 172.21.15.61:0/652036232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900095f0 con 0x7f899c108610 2023-10-18T09:18:40.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f899b7fe700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:40.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.711+0000 7f89a0b9c700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 0x7f899c19fe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f899400efc0 tx=0x7f899400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:40.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.712+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994018070 con 0x7f899c108610 2023-10-18T09:18:40.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.712+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f899400f040 con 0x7f899c108610 2023-10-18T09:18:40.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.712+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400cd40 con 0x7f899c108610 2023-10-18T09:18:40.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.712+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c1a0970 con 0x7f899c108610 2023-10-18T09:18:40.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.712+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c1a0e30 con 0x7f899c108610 2023-10-18T09:18:40.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.713+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c107420 con 0x7f899c108610 2023-10-18T09:18:40.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.715+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8994003a00 con 0x7f899c108610 2023-10-18T09:18:40.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.716+0000 7f89997fa700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 0x7f89840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:40.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.717+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f89940cd610 con 0x7f899c108610 2023-10-18T09:18:40.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.717+0000 7f899bfff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 0x7f89840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:40.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.717+0000 7f899bfff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 0x7f89840a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c60 tx=0x7f898c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:40.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.719+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8994096950 con 0x7f899c108610 2023-10-18T09:18:40.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:40 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3891744316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:40 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/734369452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:40.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.935+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f899c10e180 con 0x7f899c108610 2023-10-18T09:18:40.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.937+0000 7f89997fa700 1 -- 172.21.15.61:0/652036232 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f8994099f70 con 0x7f899c108610 2023-10-18T09:18:40.940 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:40.940 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:40.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 msgr2=0x7f89840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 0x7f89840a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c60 tx=0x7f898c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:40.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 msgr2=0x7f899c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:40.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 0x7f899c19fe50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f899400efc0 tx=0x7f899400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 shutdown_connections 2023-10-18T09:18:40.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f89840a1480 0x7f89840a3940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f899c108610 0x7f899c19fe50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f899c0731f0 0x7f899c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 --2- 172.21.15.61:0/652036232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f899c0747d0 0x7f899c198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:40.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 >> 172.21.15.61:0/652036232 conn(0x7f899c0fbc90 msgr2=0x7f899c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:40.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 shutdown_connections 2023-10-18T09:18:40.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:40.940+0000 7f89a25ff700 1 -- 172.21.15.61:0/652036232 wait complete. 2023-10-18T09:18:40.962 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:18:40.962 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:18:41.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:40 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3891744316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:41.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:40 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/734369452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:41.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 -- 172.21.15.61:0/2302354700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c1012b0 msgr2=0x7f0e7c1036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 --2- 172.21.15.61:0/2302354700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c1036a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009990 tx=0x7f0e78009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 -- 172.21.15.61:0/2302354700 shutdown_connections 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 --2- 172.21.15.61:0/2302354700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c106510 0x7f0e7c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 --2- 172.21.15.61:0/2302354700 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c1036a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 --2- 172.21.15.61:0/2302354700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 0x7f0e7c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 -- 172.21.15.61:0/2302354700 >> 172.21.15.61:0/2302354700 conn(0x7f0e7c0fac00 msgr2=0x7f0e7c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 -- 172.21.15.61:0/2302354700 shutdown_connections 2023-10-18T09:18:41.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.169+0000 7f0e83fff700 1 -- 172.21.15.61:0/2302354700 wait complete. 2023-10-18T09:18:41.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.170+0000 7f0e83fff700 1 Processor -- start 2023-10-18T09:18:41.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 -- start start 2023-10-18T09:18:41.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 0x7f0e7c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a0480 con 0x7f0e7c106510 2023-10-18T09:18:41.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a05c0 con 0x7f0e7c103be0 2023-10-18T09:18:41.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e83fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c1a0750 con 0x7f0e7c1012b0 2023-10-18T09:18:41.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e8259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34470/0 (socket says 172.21.15.61:34470) 2023-10-18T09:18:41.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e8259c700 1 -- 172.21.15.61:0/1361360974 learned_addr learned my addr 172.21.15.61:0/1361360974 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:41.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e81d9b700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.171+0000 7f0e8159a700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 0x7f0e7c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 -- 172.21.15.61:0/1361360974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 msgr2=0x7f0e7c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 -- 172.21.15.61:0/1361360974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 msgr2=0x7f0e7c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 0x7f0e7c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 -- 172.21.15.61:0/1361360974 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780095f0 con 0x7f0e7c106510 2023-10-18T09:18:41.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e81d9b700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:41.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e8259c700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400ef30 tx=0x7f0e7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:41.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74018070 con 0x7f0e7c106510 2023-10-18T09:18:41.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7400f040 con 0x7f0e7c106510 2023-10-18T09:18:41.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c1a09d0 con 0x7f0e7c106510 2023-10-18T09:18:41.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400cd00 con 0x7f0e7c106510 2023-10-18T09:18:41.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.172+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c1a0ee0 con 0x7f0e7c106510 2023-10-18T09:18:41.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.173+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c108090 con 0x7f0e7c106510 2023-10-18T09:18:41.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.175+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0e74003a00 con 0x7f0e7c106510 2023-10-18T09:18:41.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.177+0000 7f0e72ffd700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 0x7f0e680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.177+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0e740ccb50 con 0x7f0e7c106510 2023-10-18T09:18:41.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.177+0000 7f0e81d9b700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 0x7f0e680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.177+0000 7f0e81d9b700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 0x7f0e680a39c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b2e0 tx=0x7f0e78005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:41.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.180+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0e74095e90 con 0x7f0e7c106510 2023-10-18T09:18:41.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:40 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3891744316' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:41.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:40 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/734369452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:41.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.389+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e7c10bfd0 con 0x7f0e7c106510 2023-10-18T09:18:41.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.390+0000 7f0e72ffd700 1 -- 172.21.15.61:0/1361360974 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e740994b0 con 0x7f0e7c106510 2023-10-18T09:18:41.393 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:18:41.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 msgr2=0x7f0e680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 0x7f0e680a39c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b2e0 tx=0x7f0e78005fd0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 msgr2=0x7f0e7c19feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400ef30 tx=0x7f0e7400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 shutdown_connections 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0e680a1500 0x7f0e680a39c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0e7c106510 0x7f0e7c19feb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e7c1012b0 0x7f0e7c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 --2- 172.21.15.61:0/1361360974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e7c103be0 0x7f0e7c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.392+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 >> 172.21.15.61:0/1361360974 conn(0x7f0e7c0fac00 msgr2=0x7f0e7c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:41.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.393+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 shutdown_connections 2023-10-18T09:18:41.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.393+0000 7f0e83fff700 1 -- 172.21.15.61:0/1361360974 wait complete. 2023-10-18T09:18:41.414 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:18:41.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 -- 172.21.15.61:0/2827758354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd0101990 msgr2=0x7f8bd0101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/2827758354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd0101990 0x7f8bd0101df0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40099c0 tx=0x7f8bc4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 -- 172.21.15.61:0/2827758354 shutdown_connections 2023-10-18T09:18:41.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/2827758354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd0108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/2827758354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd01007a0 0x7f8bd0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/2827758354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd0101990 0x7f8bd0101df0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 -- 172.21.15.61:0/2827758354 >> 172.21.15.61:0/2827758354 conn(0x7f8bd00fbcf0 msgr2=0x7f8bd00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:41.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 -- 172.21.15.61:0/2827758354 shutdown_connections 2023-10-18T09:18:41.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.622+0000 7f8bd55ff700 1 -- 172.21.15.61:0/2827758354 wait complete. 2023-10-18T09:18:41.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 Processor -- start 2023-10-18T09:18:41.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 -- start start 2023-10-18T09:18:41.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd0101990 0x7f8bd0198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01a04b0 con 0x7f8bd01007a0 2023-10-18T09:18:41.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01a05f0 con 0x7f8bd0102330 2023-10-18T09:18:41.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.623+0000 7f8bd55ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd01a0780 con 0x7f8bd0101990 2023-10-18T09:18:41.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34476/0 (socket says 172.21.15.61:34476) 2023-10-18T09:18:41.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 -- 172.21.15.61:0/1830103362 learned_addr learned my addr 172.21.15.61:0/1830103362 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:41.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bce7fc700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd0101990 0x7f8bd0198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bcf7fe700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 -- 172.21.15.61:0/1830103362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd0101990 msgr2=0x7f8bd0198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd0101990 0x7f8bd0198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 -- 172.21.15.61:0/1830103362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 msgr2=0x7f8bd019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 -- 172.21.15.61:0/1830103362 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40095f0 con 0x7f8bd01007a0 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bcf7fe700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.624+0000 7f8bceffd700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000d980 tx=0x7f8bc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.625+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000b950 con 0x7f8bd01007a0 2023-10-18T09:18:41.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.625+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc0007620 con 0x7f8bd01007a0 2023-10-18T09:18:41.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.625+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000f3c0 con 0x7f8bd01007a0 2023-10-18T09:18:41.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.625+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd01a0a60 con 0x7f8bd01007a0 2023-10-18T09:18:41.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.625+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd01a0f20 con 0x7f8bd01007a0 2023-10-18T09:18:41.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.626+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd00675a0 con 0x7f8bd01007a0 2023-10-18T09:18:41.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.628+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8bc0010640 con 0x7f8bd01007a0 2023-10-18T09:18:41.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.629+0000 7f8bb7fff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 0x7f8bb80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:41.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.629+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8bc00cd830 con 0x7f8bd01007a0 2023-10-18T09:18:41.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.629+0000 7f8bce7fc700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 0x7f8bb80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:41.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.630+0000 7f8bce7fc700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 0x7f8bb80a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006010 tx=0x7f8bc400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:41.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.632+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8bc0096ae0 con 0x7f8bd01007a0 2023-10-18T09:18:41.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.852+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bd01a1170 con 0x7f8bd01007a0 2023-10-18T09:18:41.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.853+0000 7f8bb7fff700 1 -- 172.21.15.61:0/1830103362 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8bc009a100 con 0x7f8bd01007a0 2023-10-18T09:18:41.855 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:41.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 msgr2=0x7f8bb80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 0x7f8bb80a3940 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006010 tx=0x7f8bc400b540 comp rx=0 tx=0).stop 2023-10-18T09:18:41.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 msgr2=0x7f8bd01989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:41.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000d980 tx=0x7f8bc000dc90 comp rx=0 tx=0).stop 2023-10-18T09:18:41.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 shutdown_connections 2023-10-18T09:18:41.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8bb80a1480 0x7f8bb80a3940 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8bd0102330 0x7f8bd019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8bd01007a0 0x7f8bd01989d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 --2- 172.21.15.61:0/1830103362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bd0101990 0x7f8bd0198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:41.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 >> 172.21.15.61:0/1830103362 conn(0x7f8bd00fbcf0 msgr2=0x7f8bd0072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:41.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 shutdown_connections 2023-10-18T09:18:41.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:41.855+0000 7f8bd55ff700 1 -- 172.21.15.61:0/1830103362 wait complete. 2023-10-18T09:18:41.878 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:42.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:41 smithi061 ceph-mon[98573]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:18:42.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/652036232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:42.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:41 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1361360974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:42.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 -- 172.21.15.61:0/752790555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 msgr2=0x7f1240100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 --2- 172.21.15.61:0/752790555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240100bc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f12300099c0 tx=0x7f1230009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 -- 172.21.15.61:0/752790555 shutdown_connections 2023-10-18T09:18:42.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 --2- 172.21.15.61:0/752790555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240102330 0x7f1240108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 --2- 172.21.15.61:0/752790555 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240100bc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 --2- 172.21.15.61:0/752790555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240101990 0x7f1240101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 -- 172.21.15.61:0/752790555 >> 172.21.15.61:0/752790555 conn(0x7f12400fbcf0 msgr2=0x7f12400fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:42.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 -- 172.21.15.61:0/752790555 shutdown_connections 2023-10-18T09:18:42.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.087+0000 7f12451ff700 1 -- 172.21.15.61:0/752790555 wait complete. 2023-10-18T09:18:42.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.088+0000 7f12451ff700 1 Processor -- start 2023-10-18T09:18:42.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.088+0000 7f12451ff700 1 -- start start 2023-10-18T09:18:42.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240101990 0x7f1240072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240102330 0x7f1240073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12400738c0 con 0x7f12401007a0 2023-10-18T09:18:42.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240073a30 con 0x7f1240102330 2023-10-18T09:18:42.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f12451ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12401a07b0 con 0x7f1240101990 2023-10-18T09:18:42.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34500/0 (socket says 172.21.15.61:34500) 2023-10-18T09:18:42.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 -- 172.21.15.61:0/2315258537 learned_addr learned my addr 172.21.15.61:0/2315258537 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:42.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f1237fff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240101990 0x7f1240072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123f59e700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240102330 0x7f1240073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 -- 172.21.15.61:0/2315258537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240101990 msgr2=0x7f1240072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240101990 0x7f1240072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 -- 172.21.15.61:0/2315258537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240102330 msgr2=0x7f1240073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240102330 0x7f1240073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.089+0000 7f123ed9d700 1 -- 172.21.15.61:0/2315258537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12300095f0 con 0x7f12401007a0 2023-10-18T09:18:42.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f123ed9d700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1230003730 tx=0x7f123000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:42.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123001d070 con 0x7f12401007a0 2023-10-18T09:18:42.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123000bda0 con 0x7f12401007a0 2023-10-18T09:18:42.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12401a09e0 con 0x7f12401007a0 2023-10-18T09:18:42.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1230004b30 con 0x7f12401007a0 2023-10-18T09:18:42.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.090+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a0da0 con 0x7f12401007a0 2023-10-18T09:18:42.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.091+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1240107bc0 con 0x7f12401007a0 2023-10-18T09:18:42.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.093+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1230004c90 con 0x7f12401007a0 2023-10-18T09:18:42.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.095+0000 7f123cd99700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 0x7f122c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.095+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f12300ce2b0 con 0x7f12401007a0 2023-10-18T09:18:42.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.096+0000 7f1237fff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 0x7f122c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.096+0000 7f1237fff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 0x7f122c0a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1228009c60 tx=0x7f1228009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:42.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.098+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1230097650 con 0x7f12401007a0 2023-10-18T09:18:42.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:41 smithi170 ceph-mon[135150]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:18:42.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/652036232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:42.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:41 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1361360974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:41 smithi110 ceph-mon[134818]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:18:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/652036232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:42.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:41 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1361360974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:42.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.312+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f124010c110 con 0x7f12401007a0 2023-10-18T09:18:42.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.313+0000 7f123cd99700 1 -- 172.21.15.61:0/2315258537 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f123009ac70 con 0x7f12401007a0 2023-10-18T09:18:42.316 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:42.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 msgr2=0x7f122c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 0x7f122c0a3940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1228009c60 tx=0x7f1228009260 comp rx=0 tx=0).stop 2023-10-18T09:18:42.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 msgr2=0x7f1240074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1230003730 tx=0x7f123000fa00 comp rx=0 tx=0).stop 2023-10-18T09:18:42.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 shutdown_connections 2023-10-18T09:18:42.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f122c0a1480 0x7f122c0a3940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1240102330 0x7f1240073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f12401007a0 0x7f1240074550 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 --2- 172.21.15.61:0/2315258537 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1240101990 0x7f1240072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.315+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 >> 172.21.15.61:0/2315258537 conn(0x7f12400fbcf0 msgr2=0x7f1240104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:42.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.316+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 shutdown_connections 2023-10-18T09:18:42.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.316+0000 7f12451ff700 1 -- 172.21.15.61:0/2315258537 wait complete. 2023-10-18T09:18:42.338 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:42.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 -- 172.21.15.61:0/2864590235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 msgr2=0x7fda50100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 --2- 172.21.15.61:0/2864590235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50100bc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fda44009920 tx=0x7fda44009c30 comp rx=0 tx=0).stop 2023-10-18T09:18:42.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 -- 172.21.15.61:0/2864590235 shutdown_connections 2023-10-18T09:18:42.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 --2- 172.21.15.61:0/2864590235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50102330 0x7fda50108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 --2- 172.21.15.61:0/2864590235 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50100bc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 --2- 172.21.15.61:0/2864590235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50101990 0x7fda50101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 -- 172.21.15.61:0/2864590235 >> 172.21.15.61:0/2864590235 conn(0x7fda500fbcf0 msgr2=0x7fda500fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:42.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 -- 172.21.15.61:0/2864590235 shutdown_connections 2023-10-18T09:18:42.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.547+0000 7fda571ff700 1 -- 172.21.15.61:0/2864590235 wait complete. 2023-10-18T09:18:42.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 Processor -- start 2023-10-18T09:18:42.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 -- start start 2023-10-18T09:18:42.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50101990 0x7fda50199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50102330 0x7fda5019ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda501a0550 con 0x7fda501007a0 2023-10-18T09:18:42.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda501a0690 con 0x7fda50101990 2023-10-18T09:18:42.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.548+0000 7fda571ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda501a0820 con 0x7fda50102330 2023-10-18T09:18:42.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34522/0 (socket says 172.21.15.61:34522) 2023-10-18T09:18:42.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 -- 172.21.15.61:0/2621057285 learned_addr learned my addr 172.21.15.61:0/2621057285 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:42.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda4ffff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50101990 0x7fda50199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda5579c700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50102330 0x7fda5019ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 -- 172.21.15.61:0/2621057285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50102330 msgr2=0x7fda5019ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50102330 0x7fda5019ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 -- 172.21.15.61:0/2621057285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50101990 msgr2=0x7fda50199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50101990 0x7fda50199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 -- 172.21.15.61:0/2621057285 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44009520 con 0x7fda501007a0 2023-10-18T09:18:42.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.549+0000 7fda54f9b700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fda44009230 tx=0x7fda4400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:42.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.550+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4401c070 con 0x7fda501007a0 2023-10-18T09:18:42.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.550+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda4400bba0 con 0x7fda501007a0 2023-10-18T09:18:42.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.550+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440049c0 con 0x7fda501007a0 2023-10-18T09:18:42.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.550+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda501a0aa0 con 0x7fda501007a0 2023-10-18T09:18:42.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.550+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda501a0e70 con 0x7fda501007a0 2023-10-18T09:18:42.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.551+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda500675a0 con 0x7fda501007a0 2023-10-18T09:18:42.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.553+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fda44020070 con 0x7fda501007a0 2023-10-18T09:18:42.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.554+0000 7fda4dffb700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 0x7fda400a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:42.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.554+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fda440cdd30 con 0x7fda501007a0 2023-10-18T09:18:42.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.555+0000 7fda4ffff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 0x7fda400a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:42.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.555+0000 7fda4ffff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 0x7fda400a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009c60 tx=0x7fda3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:42.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.558+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fda44097070 con 0x7fda501007a0 2023-10-18T09:18:42.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.774+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda501a11a0 con 0x7fda501007a0 2023-10-18T09:18:42.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.774+0000 7fda4dffb700 1 -- 172.21.15.61:0/2621057285 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda4409a690 con 0x7fda501007a0 2023-10-18T09:18:42.777 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:42.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 msgr2=0x7fda400a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 0x7fda400a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009c60 tx=0x7fda3c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:42.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 msgr2=0x7fda50198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:42.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fda44009230 tx=0x7fda4400bef0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 shutdown_connections 2023-10-18T09:18:42.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fda400a1500 0x7fda400a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda50102330 0x7fda5019ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fda501007a0 0x7fda50198ac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 --2- 172.21.15.61:0/2621057285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda50101990 0x7fda50199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:42.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 >> 172.21.15.61:0/2621057285 conn(0x7fda500fbcf0 msgr2=0x7fda50107150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:42.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 shutdown_connections 2023-10-18T09:18:42.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:42.777+0000 7fda571ff700 1 -- 172.21.15.61:0/2621057285 wait complete. 2023-10-18T09:18:42.798 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-18T09:18:42.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:43.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1830103362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:42 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2315258537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 -- 172.21.15.61:0/2597040450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140103ac0 msgr2=0x7f7140105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 --2- 172.21.15.61:0/2597040450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140103ac0 0x7f7140105eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f712c009990 tx=0x7f712c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 -- 172.21.15.61:0/2597040450 shutdown_connections 2023-10-18T09:18:43.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 --2- 172.21.15.61:0/2597040450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71401063f0 0x7f7140108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 --2- 172.21.15.61:0/2597040450 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7140101190 0x7f7140103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 --2- 172.21.15.61:0/2597040450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140103ac0 0x7f7140105eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 -- 172.21.15.61:0/2597040450 >> 172.21.15.61:0/2597040450 conn(0x7f71400faba0 msgr2=0x7f71400fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:43.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 -- 172.21.15.61:0/2597040450 shutdown_connections 2023-10-18T09:18:43.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.002+0000 7f7146bff700 1 -- 172.21.15.61:0/2597040450 wait complete. 2023-10-18T09:18:43.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.003+0000 7f7146bff700 1 Processor -- start 2023-10-18T09:18:43.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.003+0000 7f7146bff700 1 -- start start 2023-10-18T09:18:43.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7140103ac0 0x7f7140196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 0x7f714019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714019e160 con 0x7f71401063f0 2023-10-18T09:18:43.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714019e2a0 con 0x7f7140101190 2023-10-18T09:18:43.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f7146bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714019e430 con 0x7f7140103ac0 2023-10-18T09:18:43.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714499b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714499b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50334/0 (socket says 172.21.15.61:50334) 2023-10-18T09:18:43.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714499b700 1 -- 172.21.15.61:0/1431264395 learned_addr learned my addr 172.21.15.61:0/1431264395 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:43.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 0x7f714019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f713ffff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7140103ac0 0x7f7140196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 -- 172.21.15.61:0/1431264395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7140103ac0 msgr2=0x7f7140196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7140103ac0 0x7f7140196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 -- 172.21.15.61:0/1431264395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 msgr2=0x7f7140196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714519c700 1 -- 172.21.15.61:0/1431264395 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7134009520 con 0x7f71401063f0 2023-10-18T09:18:43.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.004+0000 7f714499b700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:43.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f714519c700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 0x7f714019db90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f713800efc0 tx=0x7f713800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:43.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138018070 con 0x7f71401063f0 2023-10-18T09:18:43.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f713800f040 con 0x7f71401063f0 2023-10-18T09:18:43.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0095f0 con 0x7f71401063f0 2023-10-18T09:18:43.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800cd40 con 0x7f71401063f0 2023-10-18T09:18:43.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.005+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714019e9e0 con 0x7f71401063f0 2023-10-18T09:18:43.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.006+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400fc720 con 0x7f71401063f0 2023-10-18T09:18:43.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.008+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7138003a00 con 0x7f71401063f0 2023-10-18T09:18:43.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.009+0000 7f713dffb700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 0x7f71300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.010+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f71380ccc90 con 0x7f71401063f0 2023-10-18T09:18:43.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.010+0000 7f714499b700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 0x7f71300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.010+0000 7f714499b700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 0x7f71300a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7134009fd0 tx=0x7f7134009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:43.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.013+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7138095fd0 con 0x7f71401063f0 2023-10-18T09:18:43.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1830103362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:42 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2315258537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1830103362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:42 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2315258537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:43.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.271+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7140067530 con 0x7f71401063f0 2023-10-18T09:18:43.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.272+0000 7f713dffb700 1 -- 172.21.15.61:0/1431264395 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f71380995f0 con 0x7f71401063f0 2023-10-18T09:18:43.275 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:43.275 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:43.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.274+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 msgr2=0x7f71300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.274+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 0x7f71300a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7134009fd0 tx=0x7f7134009260 comp rx=0 tx=0).stop 2023-10-18T09:18:43.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.274+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 msgr2=0x7f714019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.274+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 0x7f714019db90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f713800efc0 tx=0x7f713800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 shutdown_connections 2023-10-18T09:18:43.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71300a1480 0x7f71300a3940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71401063f0 0x7f714019db90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7140101190 0x7f7140196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 --2- 172.21.15.61:0/1431264395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7140103ac0 0x7f7140196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 >> 172.21.15.61:0/1431264395 conn(0x7f71400faba0 msgr2=0x7f7140101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:43.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 shutdown_connections 2023-10-18T09:18:43.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.275+0000 7f7146bff700 1 -- 172.21.15.61:0/1431264395 wait complete. 2023-10-18T09:18:43.280 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:43.296 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-18T09:18:43.297 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-18T09:18:43.297 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:18:43.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.498+0000 7f67f63ff700 1 -- 172.21.15.61:0/2640584221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0100760 msgr2=0x7f67f0100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.498+0000 7f67f63ff700 1 --2- 172.21.15.61:0/2640584221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0100760 0x7f67f0100b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67e00099c0 tx=0x7f67e0009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 -- 172.21.15.61:0/2640584221 shutdown_connections 2023-10-18T09:18:43.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 --2- 172.21.15.61:0/2640584221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f0108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 --2- 172.21.15.61:0/2640584221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0100760 0x7f67f0100b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 --2- 172.21.15.61:0/2640584221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0101950 0x7f67f0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 -- 172.21.15.61:0/2640584221 >> 172.21.15.61:0/2640584221 conn(0x7f67f00fbcf0 msgr2=0x7f67f00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:43.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 -- 172.21.15.61:0/2640584221 shutdown_connections 2023-10-18T09:18:43.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.499+0000 7f67f63ff700 1 -- 172.21.15.61:0/2640584221 wait complete. 2023-10-18T09:18:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 Processor -- start 2023-10-18T09:18:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 -- start start 2023-10-18T09:18:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 0x7f67f01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f019dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f019e210 con 0x7f67f0101950 2023-10-18T09:18:43.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f019e350 con 0x7f67f0100760 2023-10-18T09:18:43.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67f63ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f019e4e0 con 0x7f67f01022f0 2023-10-18T09:18:43.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.500+0000 7f67effff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 0x7f67f01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67ef7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34560/0 (socket says 172.21.15.61:34560) 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67ef7fe700 1 -- 172.21.15.61:0/1627247131 learned_addr learned my addr 172.21.15.61:0/1627247131 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67f499c700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f019dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 -- 172.21.15.61:0/1627247131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 msgr2=0x7f67f019dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f019dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 -- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 msgr2=0x7f67f0196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 -- 172.21.15.61:0/1627247131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00095f0 con 0x7f67f0100760 2023-10-18T09:18:43.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67f499c700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:43.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67ef7fe700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:43.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67effff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 0x7f67f01966e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67e00098e0 tx=0x7f67e000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.501+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e001d070 con 0x7f67f0100760 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.502+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f019e760 con 0x7f67f0100760 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.502+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67e0005050 con 0x7f67f0100760 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.502+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0004a50 con 0x7f67f0100760 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.502+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f019ebe0 con 0x7f67f0100760 2023-10-18T09:18:43.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.503+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0107be0 con 0x7f67f0100760 2023-10-18T09:18:43.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.505+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f67e00051c0 con 0x7f67f0100760 2023-10-18T09:18:43.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.506+0000 7f67ed7fa700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 0x7f67d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:43.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.506+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f67e00cdef0 con 0x7f67f0100760 2023-10-18T09:18:43.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.506+0000 7f67ef7fe700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 0x7f67d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:43.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.507+0000 7f67ef7fe700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 0x7f67d80a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67e4011550 tx=0x7f67e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:43.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.508+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f67e0097230 con 0x7f67f0100760 2023-10-18T09:18:43.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.719+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67f0067500 con 0x7f67d80a1480 2023-10-18T09:18:43.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.733+0000 7f67ed7fa700 1 -- 172.21.15.61:0/1627247131 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427392 (secure 0 0 0) 0x7f67f0067500 con 0x7f67d80a1480 2023-10-18T09:18:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:43.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 msgr2=0x7f67d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 0x7f67d80a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f67e4011550 tx=0x7f67e4009260 comp rx=0 tx=0).stop 2023-10-18T09:18:43.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 msgr2=0x7f67f01966e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:43.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 0x7f67f01966e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f67e00098e0 tx=0x7f67e000f690 comp rx=0 tx=0).stop 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 shutdown_connections 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f67d80a1480 0x7f67d80a3940 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67f01022f0 0x7f67f019dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f67f0100760 0x7f67f01966e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 --2- 172.21.15.61:0/1627247131 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f67f0101950 0x7f67f0196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:43.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 >> 172.21.15.61:0/1627247131 conn(0x7f67f00fbcf0 msgr2=0x7f67f0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:43.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 shutdown_connections 2023-10-18T09:18:43.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:43.739+0000 7f67f63ff700 1 -- 172.21.15.61:0/1627247131 wait complete. 2023-10-18T09:18:43.747 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:18:43.761 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2023-10-18T09:18:42.866940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664799,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1198,"num_read_kb":730,"num_write":436,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180927,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338612,"kb_used_data":14376,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603212,"statfs":{"total":1151940427776,"available":1151593689088,"internally_reserved":0,"allocated":14721024,"data_stored":8085621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6011,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931324+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:18:22.931324+0000","last_peered":"2023-10-18T09:18:22.931324+0000","last_clean":"2023-10-18T09:18:22.931324+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:18:22.931324+0000","last_undegraded":"2023-10-18T09:18:22.931324+0000","last_fullsized":"2023-10-18T09:18:22.931324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299959+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:18:23.299959+0000","last_peered":"2023-10-18T09:18:23.299959+0000","last_clean":"2023-10-18T09:18:23.299959+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:18:23.299959+0000","last_undegraded":"2023-10-18T09:18:23.299959+0000","last_fullsized":"2023-10-18T09:18:23.299959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300618+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:18:23.300618+0000","last_peered":"2023-10-18T09:18:23.300618+0000","last_clean":"2023-10-18T09:18:23.300618+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:18:23.300618+0000","last_undegraded":"2023-10-18T09:18:23.300618+0000","last_fullsized":"2023-10-18T09:18:23.300618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300357+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:18:23.300357+0000","last_peered":"2023-10-18T09:18:23.300357+0000","last_clean":"2023-10-18T09:18:23.300357+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:18:23.300357+0000","last_undegraded":"2023-10-18T09:18:23.300357+0000","last_fullsized":"2023-10-18T09:18:23.300357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934459+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:18:21.934459+0000","last_peered":"2023-10-18T09:18:21.934459+0000","last_clean":"2023-10-18T09:18:21.934459+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:18:21.934459+0000","last_undegraded":"2023-10-18T09:18:21.934459+0000","last_fullsized":"2023-10-18T09:18:21.934459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574887+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:18:22.574887+0000","last_peered":"2023-10-18T09:18:22.574887+0000","last_clean":"2023-10-18T09:18:22.574887+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:18:22.574887+0000","last_undegraded":"2023-10-18T09:18:22.574887+0000","last_fullsized":"2023-10-18T09:18:22.574887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574959+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:18:22.574959+0000","last_peered":"2023-10-18T09:18:22.574959+0000","last_clean":"2023-10-18T09:18:22.574959+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:18:22.574959+0000","last_undegraded":"2023-10-18T09:18:22.574959+0000","last_fullsized":"2023-10-18T09:18:22.574959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926842+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:18:21.926842+0000","last_peered":"2023-10-18T09:18:21.926842+0000","last_clean":"2023-10-18T09:18:21.926842+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:18:21.926842+0000","last_undegraded":"2023-10-18T09:18:21.926842+0000","last_fullsized":"2023-10-18T09:18:21.926842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934246+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:18:21.934246+0000","last_peered":"2023-10-18T09:18:21.934246+0000","last_clean":"2023-10-18T09:18:21.934246+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:18:21.934246+0000","last_undegraded":"2023-10-18T09:18:21.934246+0000","last_fullsized":"2023-10-18T09:18:21.934246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.575106+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:18:22.575106+0000","last_peered":"2023-10-18T09:18:22.575106+0000","last_clean":"2023-10-18T09:18:22.575106+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:18:22.575106+0000","last_undegraded":"2023-10-18T09:18:22.575106+0000","last_fullsized":"2023-10-18T09:18:22.575106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574171+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:18:22.574171+0000","last_peered":"2023-10-18T09:18:22.574171+0000","last_clean":"2023-10-18T09:18:22.574171+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:18:22.574171+0000","last_undegraded":"2023-10-18T09:18:22.574171+0000","last_fullsized":"2023-10-18T09:18:22.574171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927896+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:18:21.927896+0000","last_peered":"2023-10-18T09:18:21.927896+0000","last_clean":"2023-10-18T09:18:21.927896+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:18:21.927896+0000","last_undegraded":"2023-10-18T09:18:21.927896+0000","last_fullsized":"2023-10-18T09:18:21.927896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573750+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:18:22.573750+0000","last_peered":"2023-10-18T09:18:22.573750+0000","last_clean":"2023-10-18T09:18:22.573750+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:18:22.573750+0000","last_undegraded":"2023-10-18T09:18:22.573750+0000","last_fullsized":"2023-10-18T09:18:22.573750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781028+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:18:22.781028+0000","last_peered":"2023-10-18T09:18:22.781028+0000","last_clean":"2023-10-18T09:18:22.781028+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:18:22.781028+0000","last_undegraded":"2023-10-18T09:18:22.781028+0000","last_fullsized":"2023-10-18T09:18:22.781028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781064+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:18:22.781064+0000","last_peered":"2023-10-18T09:18:22.781064+0000","last_clean":"2023-10-18T09:18:22.781064+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:18:22.781064+0000","last_undegraded":"2023-10-18T09:18:22.781064+0000","last_fullsized":"2023-10-18T09:18:22.781064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933702+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:18:21.933702+0000","last_peered":"2023-10-18T09:18:21.933702+0000","last_clean":"2023-10-18T09:18:21.933702+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:18:21.933702+0000","last_undegraded":"2023-10-18T09:18:21.933702+0000","last_fullsized":"2023-10-18T09:18:21.933702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574196+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:18:22.574196+0000","last_peered":"2023-10-18T09:18:22.574196+0000","last_clean":"2023-10-18T09:18:22.574196+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:18:22.574196+0000","last_undegraded":"2023-10-18T09:18:22.574196+0000","last_fullsized":"2023-10-18T09:18:22.574196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936722+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:18:21.936722+0000","last_peered":"2023-10-18T09:18:21.936722+0000","last_clean":"2023-10-18T09:18:21.936722+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:18:21.936722+0000","last_undegraded":"2023-10-18T09:18:21.936722+0000","last_fullsized":"2023-10-18T09:18:21.936722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926926+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:18:21.926926+0000","last_peered":"2023-10-18T09:18:21.926926+0000","last_clean":"2023-10-18T09:18:21.926926+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:18:21.926926+0000","last_undegraded":"2023-10-18T09:18:21.926926+0000","last_fullsized":"2023-10-18T09:18:21.926926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926410+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:18:21.926410+0000","last_peered":"2023-10-18T09:18:21.926410+0000","last_clean":"2023-10-18T09:18:21.926410+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:18:21.926410+0000","last_undegraded":"2023-10-18T09:18:21.926410+0000","last_fullsized":"2023-10-18T09:18:21.926410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":136,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:38.684407+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:18:38.684407+0000","last_peered":"2023-10-18T09:18:38.684407+0000","last_clean":"2023-10-18T09:18:38.684407+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:18:38.684407+0000","last_undegraded":"2023-10-18T09:18:38.684407+0000","last_fullsized":"2023-10-18T09:18:38.684407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.936374+0000","last_change":"2023-10-18T09:18:22.935678+0000","last_active":"2023-10-18T09:18:22.936374+0000","last_peered":"2023-10-18T09:18:22.936374+0000","last_clean":"2023-10-18T09:18:22.936374+0000","last_became_active":"2023-10-18T09:18:22.935128+0000","last_became_peered":"2023-10-18T09:18:22.935128+0000","last_unstale":"2023-10-18T09:18:22.936374+0000","last_undegraded":"2023-10-18T09:18:22.936374+0000","last_fullsized":"2023-10-18T09:18:22.936374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:23.547809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926645+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:18:21.926645+0000","last_peered":"2023-10-18T09:18:21.926645+0000","last_clean":"2023-10-18T09:18:21.926645+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:18:21.926645+0000","last_undegraded":"2023-10-18T09:18:21.926645+0000","last_fullsized":"2023-10-18T09:18:21.926645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282769+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:18:23.282769+0000","last_peered":"2023-10-18T09:18:23.282769+0000","last_clean":"2023-10-18T09:18:23.282769+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:18:23.282769+0000","last_undegraded":"2023-10-18T09:18:23.282769+0000","last_fullsized":"2023-10-18T09:18:23.282769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300027+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:18:23.300027+0000","last_peered":"2023-10-18T09:18:23.300027+0000","last_clean":"2023-10-18T09:18:23.300027+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:18:23.300027+0000","last_undegraded":"2023-10-18T09:18:23.300027+0000","last_fullsized":"2023-10-18T09:18:23.300027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926915+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:18:21.926915+0000","last_peered":"2023-10-18T09:18:21.926915+0000","last_clean":"2023-10-18T09:18:21.926915+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:18:21.926915+0000","last_undegraded":"2023-10-18T09:18:21.926915+0000","last_fullsized":"2023-10-18T09:18:21.926915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935048+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:18:21.935048+0000","last_peered":"2023-10-18T09:18:21.935048+0000","last_clean":"2023-10-18T09:18:21.935048+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:18:21.935048+0000","last_undegraded":"2023-10-18T09:18:21.935048+0000","last_fullsized":"2023-10-18T09:18:21.935048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934483+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:18:21.934483+0000","last_peered":"2023-10-18T09:18:21.934483+0000","last_clean":"2023-10-18T09:18:21.934483+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:18:21.934483+0000","last_undegraded":"2023-10-18T09:18:21.934483+0000","last_fullsized":"2023-10-18T09:18:21.934483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926826+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:18:21.926826+0000","last_peered":"2023-10-18T09:18:21.926826+0000","last_clean":"2023-10-18T09:18:21.926826+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:18:21.926826+0000","last_undegraded":"2023-10-18T09:18:21.926826+0000","last_fullsized":"2023-10-18T09:18:21.926826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934579+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:18:21.934579+0000","last_peered":"2023-10-18T09:18:21.934579+0000","last_clean":"2023-10-18T09:18:21.934579+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:18:21.934579+0000","last_undegraded":"2023-10-18T09:18:21.934579+0000","last_fullsized":"2023-10-18T09:18:21.934579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930047+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:18:21.930047+0000","last_peered":"2023-10-18T09:18:21.930047+0000","last_clean":"2023-10-18T09:18:21.930047+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:18:21.930047+0000","last_undegraded":"2023-10-18T09:18:21.930047+0000","last_fullsized":"2023-10-18T09:18:21.930047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927100+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:18:21.927100+0000","last_peered":"2023-10-18T09:18:21.927100+0000","last_clean":"2023-10-18T09:18:21.927100+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:18:21.927100+0000","last_undegraded":"2023-10-18T09:18:21.927100+0000","last_fullsized":"2023-10-18T09:18:21.927100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684206+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:18:37.684206+0000","last_peered":"2023-10-18T09:18:37.684206+0000","last_clean":"2023-10-18T09:18:37.684206+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:18:37.684206+0000","last_undegraded":"2023-10-18T09:18:37.684206+0000","last_fullsized":"2023-10-18T09:18:37.684206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934348+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:18:21.934348+0000","last_peered":"2023-10-18T09:18:21.934348+0000","last_clean":"2023-10-18T09:18:21.934348+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:18:21.934348+0000","last_undegraded":"2023-10-18T09:18:21.934348+0000","last_fullsized":"2023-10-18T09:18:21.934348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926656+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:18:21.926656+0000","last_peered":"2023-10-18T09:18:21.926656+0000","last_clean":"2023-10-18T09:18:21.926656+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:18:21.926656+0000","last_undegraded":"2023-10-18T09:18:21.926656+0000","last_fullsized":"2023-10-18T09:18:21.926656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574771+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:18:22.574771+0000","last_peered":"2023-10-18T09:18:22.574771+0000","last_clean":"2023-10-18T09:18:22.574771+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:18:22.574771+0000","last_undegraded":"2023-10-18T09:18:22.574771+0000","last_fullsized":"2023-10-18T09:18:22.574771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930860+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:18:21.930860+0000","last_peered":"2023-10-18T09:18:21.930860+0000","last_clean":"2023-10-18T09:18:21.930860+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:18:21.930860+0000","last_undegraded":"2023-10-18T09:18:21.930860+0000","last_fullsized":"2023-10-18T09:18:21.930860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931037+0000","last_change":"2023-10-18T09:17:21.918712+0000","last_active":"2023-10-18T09:18:22.931037+0000","last_peered":"2023-10-18T09:18:22.931037+0000","last_clean":"2023-10-18T09:18:22.931037+0000","last_became_active":"2023-10-18T09:17:21.918229+0000","last_became_peered":"2023-10-18T09:17:21.918229+0000","last_unstale":"2023-10-18T09:18:22.931037+0000","last_undegraded":"2023-10-18T09:18:22.931037+0000","last_fullsized":"2023-10-18T09:18:22.931037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:31.948581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573599+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:18:22.573599+0000","last_peered":"2023-10-18T09:18:22.573599+0000","last_clean":"2023-10-18T09:18:22.573599+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:18:22.573599+0000","last_undegraded":"2023-10-18T09:18:22.573599+0000","last_fullsized":"2023-10-18T09:18:22.573599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926937+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:18:21.926937+0000","last_peered":"2023-10-18T09:18:21.926937+0000","last_clean":"2023-10-18T09:18:21.926937+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:18:21.926937+0000","last_undegraded":"2023-10-18T09:18:21.926937+0000","last_fullsized":"2023-10-18T09:18:21.926937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299862+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:18:23.299862+0000","last_peered":"2023-10-18T09:18:23.299862+0000","last_clean":"2023-10-18T09:18:23.299862+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:18:23.299862+0000","last_undegraded":"2023-10-18T09:18:23.299862+0000","last_fullsized":"2023-10-18T09:18:23.299862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930648+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:18:21.930648+0000","last_peered":"2023-10-18T09:18:21.930648+0000","last_clean":"2023-10-18T09:18:21.930648+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:18:21.930648+0000","last_undegraded":"2023-10-18T09:18:21.930648+0000","last_fullsized":"2023-10-18T09:18:21.930648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933262+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:18:21.933262+0000","last_peered":"2023-10-18T09:18:21.933262+0000","last_clean":"2023-10-18T09:18:21.933262+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:18:21.933262+0000","last_undegraded":"2023-10-18T09:18:21.933262+0000","last_fullsized":"2023-10-18T09:18:21.933262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926979+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:18:21.926979+0000","last_peered":"2023-10-18T09:18:21.926979+0000","last_clean":"2023-10-18T09:18:21.926979+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:18:21.926979+0000","last_undegraded":"2023-10-18T09:18:21.926979+0000","last_fullsized":"2023-10-18T09:18:21.926979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":232,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688656+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:18:40.688656+0000","last_peered":"2023-10-18T09:18:40.688656+0000","last_clean":"2023-10-18T09:18:40.688656+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:18:40.688656+0000","last_undegraded":"2023-10-18T09:18:40.688656+0000","last_fullsized":"2023-10-18T09:18:40.688656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934250+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:18:21.934250+0000","last_peered":"2023-10-18T09:18:21.934250+0000","last_clean":"2023-10-18T09:18:21.934250+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:18:21.934250+0000","last_undegraded":"2023-10-18T09:18:21.934250+0000","last_fullsized":"2023-10-18T09:18:21.934250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932772+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:18:21.932772+0000","last_peered":"2023-10-18T09:18:21.932772+0000","last_clean":"2023-10-18T09:18:21.932772+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:18:21.932772+0000","last_undegraded":"2023-10-18T09:18:21.932772+0000","last_fullsized":"2023-10-18T09:18:21.932772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574023+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:18:22.574023+0000","last_peered":"2023-10-18T09:18:22.574023+0000","last_clean":"2023-10-18T09:18:22.574023+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:18:22.574023+0000","last_undegraded":"2023-10-18T09:18:22.574023+0000","last_fullsized":"2023-10-18T09:18:22.574023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781543+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:18:22.781543+0000","last_peered":"2023-10-18T09:18:22.781543+0000","last_clean":"2023-10-18T09:18:22.781543+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:18:22.781543+0000","last_undegraded":"2023-10-18T09:18:22.781543+0000","last_fullsized":"2023-10-18T09:18:22.781543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927375+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:18:21.927375+0000","last_peered":"2023-10-18T09:18:21.927375+0000","last_clean":"2023-10-18T09:18:21.927375+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:18:21.927375+0000","last_undegraded":"2023-10-18T09:18:21.927375+0000","last_fullsized":"2023-10-18T09:18:21.927375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.929940+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:18:21.929940+0000","last_peered":"2023-10-18T09:18:21.929940+0000","last_clean":"2023-10-18T09:18:21.929940+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:18:21.929940+0000","last_undegraded":"2023-10-18T09:18:21.929940+0000","last_fullsized":"2023-10-18T09:18:21.929940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934572+0000","last_change":"2023-10-18T09:17:21.921019+0000","last_active":"2023-10-18T09:18:21.934572+0000","last_peered":"2023-10-18T09:18:21.934572+0000","last_clean":"2023-10-18T09:18:21.934572+0000","last_became_active":"2023-10-18T09:17:21.920611+0000","last_became_peered":"2023-10-18T09:17:21.920611+0000","last_unstale":"2023-10-18T09:18:21.934572+0000","last_undegraded":"2023-10-18T09:18:21.934572+0000","last_fullsized":"2023-10-18T09:18:21.934572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:44.317766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932701+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:18:21.932701+0000","last_peered":"2023-10-18T09:18:21.932701+0000","last_clean":"2023-10-18T09:18:21.932701+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:18:21.932701+0000","last_undegraded":"2023-10-18T09:18:21.932701+0000","last_fullsized":"2023-10-18T09:18:21.932701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":282,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688670+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:18:40.688670+0000","last_peered":"2023-10-18T09:18:40.688670+0000","last_clean":"2023-10-18T09:18:40.688670+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:18:40.688670+0000","last_undegraded":"2023-10-18T09:18:40.688670+0000","last_fullsized":"2023-10-18T09:18:40.688670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":221,"num_read_kb":177,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934847+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:18:21.934847+0000","last_peered":"2023-10-18T09:18:21.934847+0000","last_clean":"2023-10-18T09:18:21.934847+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:18:21.934847+0000","last_undegraded":"2023-10-18T09:18:21.934847+0000","last_fullsized":"2023-10-18T09:18:21.934847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930547+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:18:21.930547+0000","last_peered":"2023-10-18T09:18:21.930547+0000","last_clean":"2023-10-18T09:18:21.930547+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:18:21.930547+0000","last_undegraded":"2023-10-18T09:18:21.930547+0000","last_fullsized":"2023-10-18T09:18:21.930547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573816+0000","last_change":"2023-10-18T09:17:21.956449+0000","last_active":"2023-10-18T09:18:22.573816+0000","last_peered":"2023-10-18T09:18:22.573816+0000","last_clean":"2023-10-18T09:18:22.573816+0000","last_became_active":"2023-10-18T09:17:21.946784+0000","last_became_peered":"2023-10-18T09:17:21.946784+0000","last_unstale":"2023-10-18T09:18:22.573816+0000","last_undegraded":"2023-10-18T09:18:22.573816+0000","last_fullsized":"2023-10-18T09:18:22.573816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:49:21.415006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573678+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:18:22.573678+0000","last_peered":"2023-10-18T09:18:22.573678+0000","last_clean":"2023-10-18T09:18:22.573678+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:18:22.573678+0000","last_undegraded":"2023-10-18T09:18:22.573678+0000","last_fullsized":"2023-10-18T09:18:22.573678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573914+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:18:22.573914+0000","last_peered":"2023-10-18T09:18:22.573914+0000","last_clean":"2023-10-18T09:18:22.573914+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:18:22.573914+0000","last_undegraded":"2023-10-18T09:18:22.573914+0000","last_fullsized":"2023-10-18T09:18:22.573914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930527+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:18:21.930527+0000","last_peered":"2023-10-18T09:18:21.930527+0000","last_clean":"2023-10-18T09:18:21.930527+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:18:21.930527+0000","last_undegraded":"2023-10-18T09:18:21.930527+0000","last_fullsized":"2023-10-18T09:18:21.930527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574246+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:18:22.574246+0000","last_peered":"2023-10-18T09:18:22.574246+0000","last_clean":"2023-10-18T09:18:22.574246+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:18:22.574246+0000","last_undegraded":"2023-10-18T09:18:22.574246+0000","last_fullsized":"2023-10-18T09:18:22.574246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936027+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:18:21.936027+0000","last_peered":"2023-10-18T09:18:21.936027+0000","last_clean":"2023-10-18T09:18:21.936027+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:18:21.936027+0000","last_undegraded":"2023-10-18T09:18:21.936027+0000","last_fullsized":"2023-10-18T09:18:21.936027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684689+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:18:37.684689+0000","last_peered":"2023-10-18T09:18:37.684689+0000","last_clean":"2023-10-18T09:18:37.684689+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:18:37.684689+0000","last_undegraded":"2023-10-18T09:18:37.684689+0000","last_fullsized":"2023-10-18T09:18:37.684689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574313+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:18:22.574313+0000","last_peered":"2023-10-18T09:18:22.574313+0000","last_clean":"2023-10-18T09:18:22.574313+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:18:22.574313+0000","last_undegraded":"2023-10-18T09:18:22.574313+0000","last_fullsized":"2023-10-18T09:18:22.574313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574038+0000","last_change":"2023-10-18T09:17:21.948883+0000","last_active":"2023-10-18T09:18:22.574038+0000","last_peered":"2023-10-18T09:18:22.574038+0000","last_clean":"2023-10-18T09:18:22.574038+0000","last_became_active":"2023-10-18T09:17:21.948451+0000","last_became_peered":"2023-10-18T09:17:21.948451+0000","last_unstale":"2023-10-18T09:18:22.574038+0000","last_undegraded":"2023-10-18T09:18:22.574038+0000","last_fullsized":"2023-10-18T09:18:22.574038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:09:25.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933791+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:18:21.933791+0000","last_peered":"2023-10-18T09:18:21.933791+0000","last_clean":"2023-10-18T09:18:21.933791+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:18:21.933791+0000","last_undegraded":"2023-10-18T09:18:21.933791+0000","last_fullsized":"2023-10-18T09:18:21.933791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301638+0000","last_change":"2023-10-18T09:18:23.301638+0000","last_active":"2023-10-18T09:18:23.301638+0000","last_peered":"2023-10-18T09:18:23.301638+0000","last_clean":"2023-10-18T09:18:23.301638+0000","last_became_active":"2023-10-18T09:18:23.301186+0000","last_became_peered":"2023-10-18T09:18:23.301186+0000","last_unstale":"2023-10-18T09:18:23.301638+0000","last_undegraded":"2023-10-18T09:18:23.301638+0000","last_fullsized":"2023-10-18T09:18:23.301638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:34.532983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932148+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:18:21.932148+0000","last_peered":"2023-10-18T09:18:21.932148+0000","last_clean":"2023-10-18T09:18:21.932148+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:18:21.932148+0000","last_undegraded":"2023-10-18T09:18:21.932148+0000","last_fullsized":"2023-10-18T09:18:21.932148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935119+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:18:21.935119+0000","last_peered":"2023-10-18T09:18:21.935119+0000","last_clean":"2023-10-18T09:18:21.935119+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:18:21.935119+0000","last_undegraded":"2023-10-18T09:18:21.935119+0000","last_fullsized":"2023-10-18T09:18:21.935119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574093+0000","last_change":"2023-10-18T09:17:21.919795+0000","last_active":"2023-10-18T09:18:22.574093+0000","last_peered":"2023-10-18T09:18:22.574093+0000","last_clean":"2023-10-18T09:18:22.574093+0000","last_became_active":"2023-10-18T09:17:21.919419+0000","last_became_peered":"2023-10-18T09:17:21.919419+0000","last_unstale":"2023-10-18T09:18:22.574093+0000","last_undegraded":"2023-10-18T09:18:22.574093+0000","last_fullsized":"2023-10-18T09:18:22.574093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:45.138506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300194+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:18:23.300194+0000","last_peered":"2023-10-18T09:18:23.300194+0000","last_clean":"2023-10-18T09:18:23.300194+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:18:23.300194+0000","last_undegraded":"2023-10-18T09:18:23.300194+0000","last_fullsized":"2023-10-18T09:18:23.300194+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931248+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:18:22.931248+0000","last_peered":"2023-10-18T09:18:22.931248+0000","last_clean":"2023-10-18T09:18:22.931248+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:18:22.931248+0000","last_undegraded":"2023-10-18T09:18:22.931248+0000","last_fullsized":"2023-10-18T09:18:22.931248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300078+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:18:23.300078+0000","last_peered":"2023-10-18T09:18:23.300078+0000","last_clean":"2023-10-18T09:18:23.300078+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:18:23.300078+0000","last_undegraded":"2023-10-18T09:18:23.300078+0000","last_fullsized":"2023-10-18T09:18:23.300078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300839+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:18:23.300839+0000","last_peered":"2023-10-18T09:18:23.300839+0000","last_clean":"2023-10-18T09:18:23.300839+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:18:23.300839+0000","last_undegraded":"2023-10-18T09:18:23.300839+0000","last_fullsized":"2023-10-18T09:18:23.300839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.687858+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:18:40.687858+0000","last_peered":"2023-10-18T09:18:40.687858+0000","last_clean":"2023-10-18T09:18:40.687858+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:18:40.687858+0000","last_undegraded":"2023-10-18T09:18:40.687858+0000","last_fullsized":"2023-10-18T09:18:40.687858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301374+0000","last_change":"2023-10-18T09:17:21.920260+0000","last_active":"2023-10-18T09:18:23.301374+0000","last_peered":"2023-10-18T09:18:23.301374+0000","last_clean":"2023-10-18T09:18:23.301374+0000","last_became_active":"2023-10-18T09:17:21.919905+0000","last_became_peered":"2023-10-18T09:17:21.919905+0000","last_unstale":"2023-10-18T09:18:23.301374+0000","last_undegraded":"2023-10-18T09:18:23.301374+0000","last_fullsized":"2023-10-18T09:18:23.301374+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:29.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300727+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:18:23.300727+0000","last_peered":"2023-10-18T09:18:23.300727+0000","last_clean":"2023-10-18T09:18:23.300727+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:18:23.300727+0000","last_undegraded":"2023-10-18T09:18:23.300727+0000","last_fullsized":"2023-10-18T09:18:23.300727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931375+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:18:22.931375+0000","last_peered":"2023-10-18T09:18:22.931375+0000","last_clean":"2023-10-18T09:18:22.931375+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:18:22.931375+0000","last_undegraded":"2023-10-18T09:18:22.931375+0000","last_fullsized":"2023-10-18T09:18:22.931375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.686777+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:18:39.686777+0000","last_peered":"2023-10-18T09:18:39.686777+0000","last_clean":"2023-10-18T09:18:39.686777+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:18:39.686777+0000","last_undegraded":"2023-10-18T09:18:39.686777+0000","last_fullsized":"2023-10-18T09:18:39.686777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301085+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:18:23.301085+0000","last_peered":"2023-10-18T09:18:23.301085+0000","last_clean":"2023-10-18T09:18:23.301085+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:18:23.301085+0000","last_undegraded":"2023-10-18T09:18:23.301085+0000","last_fullsized":"2023-10-18T09:18:23.301085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282146+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:18:23.282146+0000","last_peered":"2023-10-18T09:18:23.282146+0000","last_clean":"2023-10-18T09:18:23.282146+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:18:23.282146+0000","last_undegraded":"2023-10-18T09:18:23.282146+0000","last_fullsized":"2023-10-18T09:18:23.282146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300391+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:18:23.300391+0000","last_peered":"2023-10-18T09:18:23.300391+0000","last_clean":"2023-10-18T09:18:23.300391+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:18:23.300391+0000","last_undegraded":"2023-10-18T09:18:23.300391+0000","last_fullsized":"2023-10-18T09:18:23.300391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573989+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:18:22.573989+0000","last_peered":"2023-10-18T09:18:22.573989+0000","last_clean":"2023-10-18T09:18:22.573989+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:18:22.573989+0000","last_undegraded":"2023-10-18T09:18:22.573989+0000","last_fullsized":"2023-10-18T09:18:22.573989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930232+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:18:21.930232+0000","last_peered":"2023-10-18T09:18:21.930232+0000","last_clean":"2023-10-18T09:18:21.930232+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:18:21.930232+0000","last_undegraded":"2023-10-18T09:18:21.930232+0000","last_fullsized":"2023-10-18T09:18:21.930232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299699+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:18:23.299699+0000","last_peered":"2023-10-18T09:18:23.299699+0000","last_clean":"2023-10-18T09:18:23.299699+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:18:23.299699+0000","last_undegraded":"2023-10-18T09:18:23.299699+0000","last_fullsized":"2023-10-18T09:18:23.299699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300183+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:18:23.300183+0000","last_peered":"2023-10-18T09:18:23.300183+0000","last_clean":"2023-10-18T09:18:23.300183+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:18:23.300183+0000","last_undegraded":"2023-10-18T09:18:23.300183+0000","last_fullsized":"2023-10-18T09:18:23.300183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930784+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:18:22.930784+0000","last_peered":"2023-10-18T09:18:22.930784+0000","last_clean":"2023-10-18T09:18:22.930784+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:18:22.930784+0000","last_undegraded":"2023-10-18T09:18:22.930784+0000","last_fullsized":"2023-10-18T09:18:22.930784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300567+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:18:23.300567+0000","last_peered":"2023-10-18T09:18:23.300567+0000","last_clean":"2023-10-18T09:18:23.300567+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:18:23.300567+0000","last_undegraded":"2023-10-18T09:18:23.300567+0000","last_fullsized":"2023-10-18T09:18:23.300567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300439+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:18:23.300439+0000","last_peered":"2023-10-18T09:18:23.300439+0000","last_clean":"2023-10-18T09:18:23.300439+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:23.300439+0000","last_undegraded":"2023-10-18T09:18:23.300439+0000","last_fullsized":"2023-10-18T09:18:23.300439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281950+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:18:23.281950+0000","last_peered":"2023-10-18T09:18:23.281950+0000","last_clean":"2023-10-18T09:18:23.281950+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:18:23.281950+0000","last_undegraded":"2023-10-18T09:18:23.281950+0000","last_fullsized":"2023-10-18T09:18:23.281950+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:25.214651+0000","last_change":"2023-10-18T09:17:21.962634+0000","last_active":"2023-10-18T09:18:25.214651+0000","last_peered":"2023-10-18T09:18:25.214651+0000","last_clean":"2023-10-18T09:18:25.214651+0000","last_became_active":"2023-10-18T09:17:21.948090+0000","last_became_peered":"2023-10-18T09:17:21.948090+0000","last_unstale":"2023-10-18T09:18:25.214651+0000","last_undegraded":"2023-10-18T09:18:25.214651+0000","last_fullsized":"2023-10-18T09:18:25.214651+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:12:14.390673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282465+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:18:23.282465+0000","last_peered":"2023-10-18T09:18:23.282465+0000","last_clean":"2023-10-18T09:18:23.282465+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:18:23.282465+0000","last_undegraded":"2023-10-18T09:18:23.282465+0000","last_fullsized":"2023-10-18T09:18:23.282465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934023+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:18:22.934023+0000","last_peered":"2023-10-18T09:18:22.934023+0000","last_clean":"2023-10-18T09:18:22.934023+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:18:22.934023+0000","last_undegraded":"2023-10-18T09:18:22.934023+0000","last_fullsized":"2023-10-18T09:18:22.934023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301074+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:18:23.301074+0000","last_peered":"2023-10-18T09:18:23.301074+0000","last_clean":"2023-10-18T09:18:23.301074+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:18:23.301074+0000","last_undegraded":"2023-10-18T09:18:23.301074+0000","last_fullsized":"2023-10-18T09:18:23.301074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300059+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:18:23.300059+0000","last_peered":"2023-10-18T09:18:23.300059+0000","last_clean":"2023-10-18T09:18:23.300059+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:18:23.300059+0000","last_undegraded":"2023-10-18T09:18:23.300059+0000","last_fullsized":"2023-10-18T09:18:23.300059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927146+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:18:21.927146+0000","last_peered":"2023-10-18T09:18:21.927146+0000","last_clean":"2023-10-18T09:18:21.927146+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:18:21.927146+0000","last_undegraded":"2023-10-18T09:18:21.927146+0000","last_fullsized":"2023-10-18T09:18:21.927146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.933383+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:18:22.933383+0000","last_peered":"2023-10-18T09:18:22.933383+0000","last_clean":"2023-10-18T09:18:22.933383+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:18:22.933383+0000","last_undegraded":"2023-10-18T09:18:22.933383+0000","last_fullsized":"2023-10-18T09:18:22.933383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300468+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:18:23.300468+0000","last_peered":"2023-10-18T09:18:23.300468+0000","last_clean":"2023-10-18T09:18:23.300468+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:18:23.300468+0000","last_undegraded":"2023-10-18T09:18:23.300468+0000","last_fullsized":"2023-10-18T09:18:23.300468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936113+0000","last_change":"2023-10-18T09:17:21.949440+0000","last_active":"2023-10-18T09:18:21.936113+0000","last_peered":"2023-10-18T09:18:21.936113+0000","last_clean":"2023-10-18T09:18:21.936113+0000","last_became_active":"2023-10-18T09:17:21.948562+0000","last_became_peered":"2023-10-18T09:17:21.948562+0000","last_unstale":"2023-10-18T09:18:21.936113+0000","last_undegraded":"2023-10-18T09:18:21.936113+0000","last_fullsized":"2023-10-18T09:18:21.936113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:20.976200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281948+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:18:23.281948+0000","last_peered":"2023-10-18T09:18:23.281948+0000","last_clean":"2023-10-18T09:18:23.281948+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:18:23.281948+0000","last_undegraded":"2023-10-18T09:18:23.281948+0000","last_fullsized":"2023-10-18T09:18:23.281948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930407+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:18:22.930407+0000","last_peered":"2023-10-18T09:18:22.930407+0000","last_clean":"2023-10-18T09:18:22.930407+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:18:22.930407+0000","last_undegraded":"2023-10-18T09:18:22.930407+0000","last_fullsized":"2023-10-18T09:18:22.930407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927797+0000","last_change":"2023-10-18T09:17:21.957405+0000","last_active":"2023-10-18T09:18:21.927797+0000","last_peered":"2023-10-18T09:18:21.927797+0000","last_clean":"2023-10-18T09:18:21.927797+0000","last_became_active":"2023-10-18T09:17:21.917875+0000","last_became_peered":"2023-10-18T09:17:21.917875+0000","last_unstale":"2023-10-18T09:18:21.927797+0000","last_undegraded":"2023-10-18T09:18:21.927797+0000","last_fullsized":"2023-10-18T09:18:21.927797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:37.429104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282264+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:18:23.282264+0000","last_peered":"2023-10-18T09:18:23.282264+0000","last_clean":"2023-10-18T09:18:23.282264+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:18:23.282264+0000","last_undegraded":"2023-10-18T09:18:23.282264+0000","last_fullsized":"2023-10-18T09:18:23.282264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927027+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:18:21.927027+0000","last_peered":"2023-10-18T09:18:21.927027+0000","last_clean":"2023-10-18T09:18:21.927027+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:18:21.927027+0000","last_undegraded":"2023-10-18T09:18:21.927027+0000","last_fullsized":"2023-10-18T09:18:21.927027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934530+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:18:21.934530+0000","last_peered":"2023-10-18T09:18:21.934530+0000","last_clean":"2023-10-18T09:18:21.934530+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:18:21.934530+0000","last_undegraded":"2023-10-18T09:18:21.934530+0000","last_fullsized":"2023-10-18T09:18:21.934530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300205+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:18:23.300205+0000","last_peered":"2023-10-18T09:18:23.300205+0000","last_clean":"2023-10-18T09:18:23.300205+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:18:23.300205+0000","last_undegraded":"2023-10-18T09:18:23.300205+0000","last_fullsized":"2023-10-18T09:18:23.300205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933939+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:18:21.933939+0000","last_peered":"2023-10-18T09:18:21.933939+0000","last_clean":"2023-10-18T09:18:21.933939+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:18:21.933939+0000","last_undegraded":"2023-10-18T09:18:21.933939+0000","last_fullsized":"2023-10-18T09:18:21.933939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.688241+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:18:39.688241+0000","last_peered":"2023-10-18T09:18:39.688241+0000","last_clean":"2023-10-18T09:18:39.688241+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:18:39.688241+0000","last_undegraded":"2023-10-18T09:18:39.688241+0000","last_fullsized":"2023-10-18T09:18:39.688241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301515+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:18:23.301515+0000","last_peered":"2023-10-18T09:18:23.301515+0000","last_clean":"2023-10-18T09:18:23.301515+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:18:23.301515+0000","last_undegraded":"2023-10-18T09:18:23.301515+0000","last_fullsized":"2023-10-18T09:18:23.301515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781014+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:18:22.781014+0000","last_peered":"2023-10-18T09:18:22.781014+0000","last_clean":"2023-10-18T09:18:22.781014+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:18:22.781014+0000","last_undegraded":"2023-10-18T09:18:22.781014+0000","last_fullsized":"2023-10-18T09:18:22.781014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930145+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:18:21.930145+0000","last_peered":"2023-10-18T09:18:21.930145+0000","last_clean":"2023-10-18T09:18:21.930145+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:18:21.930145+0000","last_undegraded":"2023-10-18T09:18:21.930145+0000","last_fullsized":"2023-10-18T09:18:21.930145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300529+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:18:23.300529+0000","last_peered":"2023-10-18T09:18:23.300529+0000","last_clean":"2023-10-18T09:18:23.300529+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:18:23.300529+0000","last_undegraded":"2023-10-18T09:18:23.300529+0000","last_fullsized":"2023-10-18T09:18:23.300529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930361+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:18:21.930361+0000","last_peered":"2023-10-18T09:18:21.930361+0000","last_clean":"2023-10-18T09:18:21.930361+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:18:21.930361+0000","last_undegraded":"2023-10-18T09:18:21.930361+0000","last_fullsized":"2023-10-18T09:18:21.930361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300587+0000","last_change":"2023-10-18T09:17:21.966321+0000","last_active":"2023-10-18T09:18:23.300587+0000","last_peered":"2023-10-18T09:18:23.300587+0000","last_clean":"2023-10-18T09:18:23.300587+0000","last_became_active":"2023-10-18T09:17:21.949466+0000","last_became_peered":"2023-10-18T09:17:21.949466+0000","last_unstale":"2023-10-18T09:18:23.300587+0000","last_undegraded":"2023-10-18T09:18:23.300587+0000","last_fullsized":"2023-10-18T09:18:23.300587+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:05.856643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300969+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:18:23.300969+0000","last_peered":"2023-10-18T09:18:23.300969+0000","last_clean":"2023-10-18T09:18:23.300969+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:18:23.300969+0000","last_undegraded":"2023-10-18T09:18:23.300969+0000","last_fullsized":"2023-10-18T09:18:23.300969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930499+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:18:22.930499+0000","last_peered":"2023-10-18T09:18:22.930499+0000","last_clean":"2023-10-18T09:18:22.930499+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:18:22.930499+0000","last_undegraded":"2023-10-18T09:18:22.930499+0000","last_fullsized":"2023-10-18T09:18:22.930499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934230+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:18:22.934230+0000","last_peered":"2023-10-18T09:18:22.934230+0000","last_clean":"2023-10-18T09:18:22.934230+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:22.934230+0000","last_undegraded":"2023-10-18T09:18:22.934230+0000","last_fullsized":"2023-10-18T09:18:22.934230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300500+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:18:23.300500+0000","last_peered":"2023-10-18T09:18:23.300500+0000","last_clean":"2023-10-18T09:18:23.300500+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:18:23.300500+0000","last_undegraded":"2023-10-18T09:18:23.300500+0000","last_fullsized":"2023-10-18T09:18:23.300500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300611+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:18:23.300611+0000","last_peered":"2023-10-18T09:18:23.300611+0000","last_clean":"2023-10-18T09:18:23.300611+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:18:23.300611+0000","last_undegraded":"2023-10-18T09:18:23.300611+0000","last_fullsized":"2023-10-18T09:18:23.300611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931115+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:18:22.931115+0000","last_peered":"2023-10-18T09:18:22.931115+0000","last_clean":"2023-10-18T09:18:22.931115+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:18:22.931115+0000","last_undegraded":"2023-10-18T09:18:22.931115+0000","last_fullsized":"2023-10-18T09:18:22.931115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934563+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:18:22.934563+0000","last_peered":"2023-10-18T09:18:22.934563+0000","last_clean":"2023-10-18T09:18:22.934563+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:18:22.934563+0000","last_undegraded":"2023-10-18T09:18:22.934563+0000","last_fullsized":"2023-10-18T09:18:22.934563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282527+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:18:23.282527+0000","last_peered":"2023-10-18T09:18:23.282527+0000","last_clean":"2023-10-18T09:18:23.282527+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:18:23.282527+0000","last_undegraded":"2023-10-18T09:18:23.282527+0000","last_fullsized":"2023-10-18T09:18:23.282527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931260+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:18:22.931260+0000","last_peered":"2023-10-18T09:18:22.931260+0000","last_clean":"2023-10-18T09:18:22.931260+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:18:22.931260+0000","last_undegraded":"2023-10-18T09:18:22.931260+0000","last_fullsized":"2023-10-18T09:18:22.931260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282117+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:18:23.282117+0000","last_peered":"2023-10-18T09:18:23.282117+0000","last_clean":"2023-10-18T09:18:23.282117+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:18:23.282117+0000","last_undegraded":"2023-10-18T09:18:23.282117+0000","last_fullsized":"2023-10-18T09:18:23.282117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934304+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:18:21.934304+0000","last_peered":"2023-10-18T09:18:21.934304+0000","last_clean":"2023-10-18T09:18:21.934304+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:18:21.934304+0000","last_undegraded":"2023-10-18T09:18:21.934304+0000","last_fullsized":"2023-10-18T09:18:21.934304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300332+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:18:23.300332+0000","last_peered":"2023-10-18T09:18:23.300332+0000","last_clean":"2023-10-18T09:18:23.300332+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:18:23.300332+0000","last_undegraded":"2023-10-18T09:18:23.300332+0000","last_fullsized":"2023-10-18T09:18:23.300332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574002+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:18:22.574002+0000","last_peered":"2023-10-18T09:18:22.574002+0000","last_clean":"2023-10-18T09:18:22.574002+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:18:22.574002+0000","last_undegraded":"2023-10-18T09:18:22.574002+0000","last_fullsized":"2023-10-18T09:18:22.574002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74185,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1072,"num_read_kb":621,"num_write":253,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":510815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.624},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.83199999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.874},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.89500000000000002,"15min":0.81000000000000005},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.055,"5min":13.055,"15min":13.055},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.008,"5min":1.407,"15min":1.407},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.89000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.133,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.052,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.107,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.94499999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1168560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69799999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.747,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42899999999999999,"15min":0.436},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.29599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93700000000000006,"5min":1.196,"15min":1.196},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71999999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77100000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74199999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.242,"15min":1.242},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.141,"15min":1.141},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86199999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.127,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.95099999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91300000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.54700000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93100000000000005,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0469999999999999,"5min":1.532,"15min":1.532},"last":0.86899999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.66400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88700000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":510677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60199999999999998,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1110000000000002,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.502},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.0510000000000002,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.0790000000000002,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81799999999999995,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.8730000000000002,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.85899999999999999,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.82999999999999996,"15min":0.75700000000000001},"min":{"1min":0.53600000000000003,"5min":0.495,"15min":0.495},"max":{"1min":7.1909999999999998,"5min":7.1909999999999998,"15min":7.1909999999999998},"last":0.876}]},{"osd":3,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.754,"15min":0.72599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":4.766,"5min":4.766,"15min":4.766},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.82699999999999996},"min":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.609,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.46899999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88900000000000001,"15min":0.86499999999999999},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.7789999999999999,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88100000000000001,"15min":0.85899999999999999},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.6500000000000004,"5min":4.6500000000000004,"15min":4.6500000000000004},"last":1.0629999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92500000000000004,"15min":0.91200000000000003},"min":{"1min":0.62,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98499999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.63,"15min":0.63},"max":{"1min":3.8140000000000001,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0029999999999999,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":3.6930000000000001,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":0.96999999999999997},"min":{"1min":0.71699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":3.7599999999999998,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1,"15min":0.96499999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9199999999999999,"5min":6.3529999999999998,"15min":6.3529999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0009999999999999,"15min":0.97799999999999998},"min":{"1min":0.69099999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.7360000000000002,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.89600000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.204,"15min":1.2549999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.1060000000000001,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.089,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.123,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":1.1020000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.443}]},{"osd":1,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.38200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.63400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65100000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72699999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71799999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":1.254,"15min":1.254},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78200000000000003,"15min":0.754},"min":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.746,"15min":0.71499999999999997},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.80700000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96599999999999997,"5min":1.05,"15min":1.05},"last":0.82799999999999996}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.502,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.42299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69899999999999995,"5min":0.876,"15min":0.876},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.745,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.46700000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83299999999999996,"5min":1.02,"15min":1.02},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53500000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.52000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.754},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72099999999999997,"15min":0.69499999999999995},"min":{"1min":0.495,"5min":0.433,"15min":0.433},"max":{"1min":0.80000000000000004,"5min":1.173,"15min":1.173},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61899999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71599999999999997,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.58399999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80700000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83499999999999996}]},{"osd":9,"last update":"Wed Oct 18 09:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.65900000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.93799999999999994,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.876},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.95699999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.93600000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.93700000000000006}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.44400000000000001},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.69399999999999995,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.69199999999999995,"15min":0.63600000000000001},"min":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.70099999999999996,"15min":0.66700000000000004},"min":{"1min":0.58599999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64300000000000002,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.75,"15min":0.70299999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4570000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.76600000000000001,"15min":0.72699999999999998},"min":{"1min":0.59299999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91100000000000003,"15min":0.80900000000000005},"min":{"1min":0.63100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7789999999999999,"5min":11.112,"15min":11.112},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83299999999999996,"15min":0.78300000000000003},"min":{"1min":0.64000000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.83499999999999996,"15min":0.80300000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.68899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5269999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2649999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.874,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.91000000000000003,"15min":0.89700000000000002},"min":{"1min":0.78300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.752}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.95099999999999996,"15min":0.93500000000000005},"min":{"1min":0.78800000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3740000000000001,"5min":2.847,"15min":2.847},"last":0.66600000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0209999999999999},"min":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3420000000000001,"5min":2.802,"15min":2.802},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.747,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4670000000000001,"5min":1.581,"15min":1.581},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":568217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.7130000000000001,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60399999999999998,"15min":0.53700000000000003},"min":{"1min":0.47299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.435},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58799999999999997,"15min":0.54900000000000004},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.52700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.42899999999999999,"15min":0.40000000000000002},"min":{"1min":0.34100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.501,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.59199999999999997},"min":{"1min":0.48899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":5.258,"15min":5.258},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.69699999999999995,"15min":0.66100000000000003},"min":{"1min":0.51400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85899999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.502,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89400000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61799999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.65600000000000003},"min":{"1min":0.57799999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.92500000000000004,"5min":1.073,"15min":1.073},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.64700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84399999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73699999999999999,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.484,"5min":1.605,"15min":1.605},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.79200000000000004,"15min":0.76200000000000001},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.83099999999999996}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76300000000000001,"15min":0.74399999999999999},"min":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.91000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":507637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.68899999999999995,"15min":0.55900000000000005},"min":{"1min":0.46899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":4.4989999999999997,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.65100000000000002,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.72999999999999998,"15min":0.624},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.68600000000000005,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.121,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.3679999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78800000000000003,"15min":1.0409999999999999},"min":{"1min":0.442,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.353,"5min":11.539,"15min":11.539},"last":0.505},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.82399999999999995,"15min":1.034},"min":{"1min":0.51800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":2.7370000000000001,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.77900000000000003,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.755,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2130000000000001,"5min":1.448,"15min":1.448},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Oct 18 09:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.079,"5min":0.84399999999999997,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1,"15min":1.02},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.3319999999999999,"5min":2.8540000000000001,"15min":2.8540000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.872,"15min":0.877},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.409},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.873,"15min":0.874},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.155,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":0.94299999999999995,"15min":0.91000000000000003},"min":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.101,"5min":0.95799999999999996,"15min":0.94599999999999995},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94399999999999995,"15min":0.90500000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.2909999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.006}]},{"osd":11,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.95999999999999996,"15min":0.91800000000000004},"min":{"1min":0.68100000000000005,"5min":0.625,"15min":0.625},"max":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.075,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6919999999999999,"5min":1.698,"15min":1.698},"last":0.95899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.60999999999999999},"min":{"1min":0.33200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76300000000000001,"5min":3.016,"15min":3.016},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91000000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60699999999999998,"15min":0.54700000000000004},"min":{"1min":0.439,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.98399999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.66100000000000003,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.315,"15min":0.315},"max":{"1min":7.2140000000000004,"5min":7.2140000000000004,"15min":7.2140000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.41499999999999998},"min":{"1min":0.35999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.76600000000000001,"15min":0.66000000000000003},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1749999999999998,"5min":7.1749999999999998,"15min":7.1749999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.59899999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95899999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.75600000000000001},"min":{"1min":0.58899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1000000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.206,"15min":1.206},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81799999999999995,"15min":0.78600000000000003},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.038,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71499999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0629999999999999,"5min":1.093,"15min":1.093},"last":0.84699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":505889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.47499999999999998,"15min":0.40600000000000003},"min":{"1min":0.38200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.498,"15min":0.433},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.73799999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":7.516,"5min":7.516,"15min":7.516},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.65800000000000003,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.68500000000000005,"15min":0.64200000000000002},"min":{"1min":0.61299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.65300000000000002},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.68999999999999995,"15min":0.61099999999999999},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":0.76600000000000001,"15min":0.64100000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.65900000000000003},"min":{"1min":0.60199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.623},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.68799999999999994},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.27,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.877}]},{"osd":6,"last update":"Wed Oct 18 09:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.67000000000000004,"15min":0.62},"min":{"1min":0.47899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68100000000000005,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.66700000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.83699999999999997}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.83899999999999997,"15min":0.82699999999999996},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89000000000000001,"15min":0.90800000000000003},"min":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.89000000000000001},"min":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.282,"5min":0.95099999999999996,"15min":0.89500000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":7.6070000000000002,"5min":7.6070000000000002,"15min":7.6070000000000002},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.2829999999999999,"15min":1.353},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1220000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.3220000000000001,"15min":1.393},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1830000000000001,"5min":7.6769999999999996,"15min":7.6769999999999996},"last":0.80300000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":511102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.373},"min":{"1min":0.35499999999999998,"5min":0.186,"15min":0.186},"max":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.39600000000000002},"min":{"1min":0.32300000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.60599999999999998,"5min":0.753,"15min":0.753},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.625},{"interface":"front","average":{"1min":1.048,"5min":0.68899999999999995,"15min":0.60099999999999998},"min":{"1min":0.52500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":8.7300000000000004,"5min":8.7300000000000004,"15min":8.7300000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":1.077,"15min":1.077},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.629,"15min":0.56899999999999995},"min":{"1min":0.53000000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.59399999999999997},"min":{"1min":0.623,"5min":0.374,"15min":0.374},"max":{"1min":0.91800000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.64100000000000001},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0329999999999999,"5min":1.131,"15min":1.131},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.624,"5min":0.378,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.65700000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97799999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.52800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.99299999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0129999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.53000000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.99099999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69399999999999995}]},{"osd":11,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.68300000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1,"5min":1,"15min":1},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67200000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1103274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.60099999999999998,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.0570000000000004,"5min":6.0570000000000004,"15min":6.0570000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80300000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.56100000000000005},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.88,"15min":0.88},"last":0.76700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41999999999999998,"15min":0.38},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59699999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.503}]},{"osd":3,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.56899999999999995},"min":{"1min":0.496,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.623},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.82999999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.60799999999999998},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.872,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.89500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:18:43.769 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:18:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2621057285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:43 smithi061 ceph-mon[98573]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-18T09:18:44.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:43 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1431264395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:44.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 -- 172.21.15.61:0/3004622371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c102970 msgr2=0x7fc94c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 --2- 172.21.15.61:0/3004622371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c102970 0x7fc94c102d90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc9480099c0 tx=0x7fc948009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 -- 172.21.15.61:0/3004622371 shutdown_connections 2023-10-18T09:18:44.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 --2- 172.21.15.61:0/3004622371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 0x7fc94c10adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 --2- 172.21.15.61:0/3004622371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c102970 0x7fc94c102d90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 --2- 172.21.15.61:0/3004622371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c103b60 0x7fc94c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 -- 172.21.15.61:0/3004622371 >> 172.21.15.61:0/3004622371 conn(0x7fc94c0fdf00 msgr2=0x7fc94c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:44.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 -- 172.21.15.61:0/3004622371 shutdown_connections 2023-10-18T09:18:44.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.004+0000 7fc9539ff700 1 -- 172.21.15.61:0/3004622371 wait complete. 2023-10-18T09:18:44.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.005+0000 7fc9539ff700 1 Processor -- start 2023-10-18T09:18:44.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.005+0000 7fc9539ff700 1 -- start start 2023-10-18T09:18:44.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c102970 0x7fc94c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 0x7fc94c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a0400 con 0x7fc94c103b60 2023-10-18T09:18:44.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a0540 con 0x7fc94c102970 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc9539ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94c1a06d0 con 0x7fc94c104500 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34580/0 (socket says 172.21.15.61:34580) 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 -- 172.21.15.61:0/2963054618 learned_addr learned my addr 172.21.15.61:0/2963054618 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc95179b700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c102970 0x7fc94c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.012 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc951f9c700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 0x7fc94c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 -- 172.21.15.61:0/2963054618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 msgr2=0x7fc94c19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 0x7fc94c19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 -- 172.21.15.61:0/2963054618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c102970 msgr2=0x7fc94c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c102970 0x7fc94c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.006+0000 7fc950f9a700 1 -- 172.21.15.61:0/2963054618 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480095f0 con 0x7fc94c103b60 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc950f9a700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00e9a0 tx=0x7fc93c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c009790 con 0x7fc94c103b60 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc93c010430 con 0x7fc94c103b60 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94c1a09b0 con 0x7fc94c103b60 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c018680 con 0x7fc94c103b60 2023-10-18T09:18:44.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.007+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94c1a0f10 con 0x7fc94c103b60 2023-10-18T09:18:44.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.008+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94c109d40 con 0x7fc94c103b60 2023-10-18T09:18:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.010+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc93c0105a0 con 0x7fc94c103b60 2023-10-18T09:18:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.011+0000 7fc9427fc700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 0x7fc9380a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.012+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc93c014070 con 0x7fc94c103b60 2023-10-18T09:18:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.012+0000 7fc95179b700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 0x7fc9380a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.012+0000 7fc95179b700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 0x7fc9380a39c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc948009990 tx=0x7fc948000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:44.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.015+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc93c0964b0 con 0x7fc94c103b60 2023-10-18T09:18:44.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2621057285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:43 smithi170 ceph-mon[135150]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-18T09:18:44.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:43 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1431264395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2621057285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:43 smithi110 ceph-mon[134818]: pgmap v363: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2023-10-18T09:18:44.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:43 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1431264395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:44.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.223+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc94c060c30 con 0x7fc9380a1500 2023-10-18T09:18:44.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.237+0000 7fc9427fc700 1 -- 172.21.15.61:0/2963054618 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427392 (secure 0 0 0) 0x7fc94c060c30 con 0x7fc9380a1500 2023-10-18T09:18:44.244 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:44.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 msgr2=0x7fc9380a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 0x7fc9380a39c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc948009990 tx=0x7fc948000bc0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 msgr2=0x7fc94c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00e9a0 tx=0x7fc93c00ecb0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 shutdown_connections 2023-10-18T09:18:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc9380a1500 0x7fc9380a39c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc94c104500 0x7fc94c19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc94c102970 0x7fc94c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 --2- 172.21.15.61:0/2963054618 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc94c103b60 0x7fc94c198e10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 >> 172.21.15.61:0/2963054618 conn(0x7fc94c0fdf00 msgr2=0x7fc94c106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:44.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 shutdown_connections 2023-10-18T09:18:44.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.242+0000 7fc9539ff700 1 -- 172.21.15.61:0/2963054618 wait complete. 2023-10-18T09:18:44.250 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:18:44.264 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2023-10-18T09:18:42.866940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664799,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1198,"num_read_kb":730,"num_write":436,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180927,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338612,"kb_used_data":14376,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603212,"statfs":{"total":1151940427776,"available":1151593689088,"internally_reserved":0,"allocated":14721024,"data_stored":8085621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6011,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931324+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:18:22.931324+0000","last_peered":"2023-10-18T09:18:22.931324+0000","last_clean":"2023-10-18T09:18:22.931324+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:18:22.931324+0000","last_undegraded":"2023-10-18T09:18:22.931324+0000","last_fullsized":"2023-10-18T09:18:22.931324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299959+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:18:23.299959+0000","last_peered":"2023-10-18T09:18:23.299959+0000","last_clean":"2023-10-18T09:18:23.299959+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:18:23.299959+0000","last_undegraded":"2023-10-18T09:18:23.299959+0000","last_fullsized":"2023-10-18T09:18:23.299959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300618+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:18:23.300618+0000","last_peered":"2023-10-18T09:18:23.300618+0000","last_clean":"2023-10-18T09:18:23.300618+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:18:23.300618+0000","last_undegraded":"2023-10-18T09:18:23.300618+0000","last_fullsized":"2023-10-18T09:18:23.300618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300357+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:18:23.300357+0000","last_peered":"2023-10-18T09:18:23.300357+0000","last_clean":"2023-10-18T09:18:23.300357+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:18:23.300357+0000","last_undegraded":"2023-10-18T09:18:23.300357+0000","last_fullsized":"2023-10-18T09:18:23.300357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934459+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:18:21.934459+0000","last_peered":"2023-10-18T09:18:21.934459+0000","last_clean":"2023-10-18T09:18:21.934459+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:18:21.934459+0000","last_undegraded":"2023-10-18T09:18:21.934459+0000","last_fullsized":"2023-10-18T09:18:21.934459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574887+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:18:22.574887+0000","last_peered":"2023-10-18T09:18:22.574887+0000","last_clean":"2023-10-18T09:18:22.574887+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:18:22.574887+0000","last_undegraded":"2023-10-18T09:18:22.574887+0000","last_fullsized":"2023-10-18T09:18:22.574887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574959+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:18:22.574959+0000","last_peered":"2023-10-18T09:18:22.574959+0000","last_clean":"2023-10-18T09:18:22.574959+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:18:22.574959+0000","last_undegraded":"2023-10-18T09:18:22.574959+0000","last_fullsized":"2023-10-18T09:18:22.574959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926842+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:18:21.926842+0000","last_peered":"2023-10-18T09:18:21.926842+0000","last_clean":"2023-10-18T09:18:21.926842+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:18:21.926842+0000","last_undegraded":"2023-10-18T09:18:21.926842+0000","last_fullsized":"2023-10-18T09:18:21.926842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934246+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:18:21.934246+0000","last_peered":"2023-10-18T09:18:21.934246+0000","last_clean":"2023-10-18T09:18:21.934246+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:18:21.934246+0000","last_undegraded":"2023-10-18T09:18:21.934246+0000","last_fullsized":"2023-10-18T09:18:21.934246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.575106+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:18:22.575106+0000","last_peered":"2023-10-18T09:18:22.575106+0000","last_clean":"2023-10-18T09:18:22.575106+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:18:22.575106+0000","last_undegraded":"2023-10-18T09:18:22.575106+0000","last_fullsized":"2023-10-18T09:18:22.575106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574171+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:18:22.574171+0000","last_peered":"2023-10-18T09:18:22.574171+0000","last_clean":"2023-10-18T09:18:22.574171+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:18:22.574171+0000","last_undegraded":"2023-10-18T09:18:22.574171+0000","last_fullsized":"2023-10-18T09:18:22.574171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927896+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:18:21.927896+0000","last_peered":"2023-10-18T09:18:21.927896+0000","last_clean":"2023-10-18T09:18:21.927896+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:18:21.927896+0000","last_undegraded":"2023-10-18T09:18:21.927896+0000","last_fullsized":"2023-10-18T09:18:21.927896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573750+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:18:22.573750+0000","last_peered":"2023-10-18T09:18:22.573750+0000","last_clean":"2023-10-18T09:18:22.573750+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:18:22.573750+0000","last_undegraded":"2023-10-18T09:18:22.573750+0000","last_fullsized":"2023-10-18T09:18:22.573750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781028+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:18:22.781028+0000","last_peered":"2023-10-18T09:18:22.781028+0000","last_clean":"2023-10-18T09:18:22.781028+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:18:22.781028+0000","last_undegraded":"2023-10-18T09:18:22.781028+0000","last_fullsized":"2023-10-18T09:18:22.781028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781064+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:18:22.781064+0000","last_peered":"2023-10-18T09:18:22.781064+0000","last_clean":"2023-10-18T09:18:22.781064+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:18:22.781064+0000","last_undegraded":"2023-10-18T09:18:22.781064+0000","last_fullsized":"2023-10-18T09:18:22.781064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933702+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:18:21.933702+0000","last_peered":"2023-10-18T09:18:21.933702+0000","last_clean":"2023-10-18T09:18:21.933702+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:18:21.933702+0000","last_undegraded":"2023-10-18T09:18:21.933702+0000","last_fullsized":"2023-10-18T09:18:21.933702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574196+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:18:22.574196+0000","last_peered":"2023-10-18T09:18:22.574196+0000","last_clean":"2023-10-18T09:18:22.574196+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:18:22.574196+0000","last_undegraded":"2023-10-18T09:18:22.574196+0000","last_fullsized":"2023-10-18T09:18:22.574196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936722+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:18:21.936722+0000","last_peered":"2023-10-18T09:18:21.936722+0000","last_clean":"2023-10-18T09:18:21.936722+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:18:21.936722+0000","last_undegraded":"2023-10-18T09:18:21.936722+0000","last_fullsized":"2023-10-18T09:18:21.936722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926926+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:18:21.926926+0000","last_peered":"2023-10-18T09:18:21.926926+0000","last_clean":"2023-10-18T09:18:21.926926+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:18:21.926926+0000","last_undegraded":"2023-10-18T09:18:21.926926+0000","last_fullsized":"2023-10-18T09:18:21.926926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926410+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:18:21.926410+0000","last_peered":"2023-10-18T09:18:21.926410+0000","last_clean":"2023-10-18T09:18:21.926410+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:18:21.926410+0000","last_undegraded":"2023-10-18T09:18:21.926410+0000","last_fullsized":"2023-10-18T09:18:21.926410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":136,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:38.684407+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:18:38.684407+0000","last_peered":"2023-10-18T09:18:38.684407+0000","last_clean":"2023-10-18T09:18:38.684407+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:18:38.684407+0000","last_undegraded":"2023-10-18T09:18:38.684407+0000","last_fullsized":"2023-10-18T09:18:38.684407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.936374+0000","last_change":"2023-10-18T09:18:22.935678+0000","last_active":"2023-10-18T09:18:22.936374+0000","last_peered":"2023-10-18T09:18:22.936374+0000","last_clean":"2023-10-18T09:18:22.936374+0000","last_became_active":"2023-10-18T09:18:22.935128+0000","last_became_peered":"2023-10-18T09:18:22.935128+0000","last_unstale":"2023-10-18T09:18:22.936374+0000","last_undegraded":"2023-10-18T09:18:22.936374+0000","last_fullsized":"2023-10-18T09:18:22.936374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:23.547809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926645+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:18:21.926645+0000","last_peered":"2023-10-18T09:18:21.926645+0000","last_clean":"2023-10-18T09:18:21.926645+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:18:21.926645+0000","last_undegraded":"2023-10-18T09:18:21.926645+0000","last_fullsized":"2023-10-18T09:18:21.926645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282769+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:18:23.282769+0000","last_peered":"2023-10-18T09:18:23.282769+0000","last_clean":"2023-10-18T09:18:23.282769+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:18:23.282769+0000","last_undegraded":"2023-10-18T09:18:23.282769+0000","last_fullsized":"2023-10-18T09:18:23.282769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300027+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:18:23.300027+0000","last_peered":"2023-10-18T09:18:23.300027+0000","last_clean":"2023-10-18T09:18:23.300027+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:18:23.300027+0000","last_undegraded":"2023-10-18T09:18:23.300027+0000","last_fullsized":"2023-10-18T09:18:23.300027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926915+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:18:21.926915+0000","last_peered":"2023-10-18T09:18:21.926915+0000","last_clean":"2023-10-18T09:18:21.926915+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:18:21.926915+0000","last_undegraded":"2023-10-18T09:18:21.926915+0000","last_fullsized":"2023-10-18T09:18:21.926915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935048+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:18:21.935048+0000","last_peered":"2023-10-18T09:18:21.935048+0000","last_clean":"2023-10-18T09:18:21.935048+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:18:21.935048+0000","last_undegraded":"2023-10-18T09:18:21.935048+0000","last_fullsized":"2023-10-18T09:18:21.935048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934483+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:18:21.934483+0000","last_peered":"2023-10-18T09:18:21.934483+0000","last_clean":"2023-10-18T09:18:21.934483+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:18:21.934483+0000","last_undegraded":"2023-10-18T09:18:21.934483+0000","last_fullsized":"2023-10-18T09:18:21.934483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926826+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:18:21.926826+0000","last_peered":"2023-10-18T09:18:21.926826+0000","last_clean":"2023-10-18T09:18:21.926826+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:18:21.926826+0000","last_undegraded":"2023-10-18T09:18:21.926826+0000","last_fullsized":"2023-10-18T09:18:21.926826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934579+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:18:21.934579+0000","last_peered":"2023-10-18T09:18:21.934579+0000","last_clean":"2023-10-18T09:18:21.934579+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:18:21.934579+0000","last_undegraded":"2023-10-18T09:18:21.934579+0000","last_fullsized":"2023-10-18T09:18:21.934579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930047+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:18:21.930047+0000","last_peered":"2023-10-18T09:18:21.930047+0000","last_clean":"2023-10-18T09:18:21.930047+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:18:21.930047+0000","last_undegraded":"2023-10-18T09:18:21.930047+0000","last_fullsized":"2023-10-18T09:18:21.930047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927100+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:18:21.927100+0000","last_peered":"2023-10-18T09:18:21.927100+0000","last_clean":"2023-10-18T09:18:21.927100+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:18:21.927100+0000","last_undegraded":"2023-10-18T09:18:21.927100+0000","last_fullsized":"2023-10-18T09:18:21.927100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684206+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:18:37.684206+0000","last_peered":"2023-10-18T09:18:37.684206+0000","last_clean":"2023-10-18T09:18:37.684206+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:18:37.684206+0000","last_undegraded":"2023-10-18T09:18:37.684206+0000","last_fullsized":"2023-10-18T09:18:37.684206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934348+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:18:21.934348+0000","last_peered":"2023-10-18T09:18:21.934348+0000","last_clean":"2023-10-18T09:18:21.934348+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:18:21.934348+0000","last_undegraded":"2023-10-18T09:18:21.934348+0000","last_fullsized":"2023-10-18T09:18:21.934348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926656+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:18:21.926656+0000","last_peered":"2023-10-18T09:18:21.926656+0000","last_clean":"2023-10-18T09:18:21.926656+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:18:21.926656+0000","last_undegraded":"2023-10-18T09:18:21.926656+0000","last_fullsized":"2023-10-18T09:18:21.926656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574771+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:18:22.574771+0000","last_peered":"2023-10-18T09:18:22.574771+0000","last_clean":"2023-10-18T09:18:22.574771+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:18:22.574771+0000","last_undegraded":"2023-10-18T09:18:22.574771+0000","last_fullsized":"2023-10-18T09:18:22.574771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930860+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:18:21.930860+0000","last_peered":"2023-10-18T09:18:21.930860+0000","last_clean":"2023-10-18T09:18:21.930860+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:18:21.930860+0000","last_undegraded":"2023-10-18T09:18:21.930860+0000","last_fullsized":"2023-10-18T09:18:21.930860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931037+0000","last_change":"2023-10-18T09:17:21.918712+0000","last_active":"2023-10-18T09:18:22.931037+0000","last_peered":"2023-10-18T09:18:22.931037+0000","last_clean":"2023-10-18T09:18:22.931037+0000","last_became_active":"2023-10-18T09:17:21.918229+0000","last_became_peered":"2023-10-18T09:17:21.918229+0000","last_unstale":"2023-10-18T09:18:22.931037+0000","last_undegraded":"2023-10-18T09:18:22.931037+0000","last_fullsized":"2023-10-18T09:18:22.931037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:31.948581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573599+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:18:22.573599+0000","last_peered":"2023-10-18T09:18:22.573599+0000","last_clean":"2023-10-18T09:18:22.573599+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:18:22.573599+0000","last_undegraded":"2023-10-18T09:18:22.573599+0000","last_fullsized":"2023-10-18T09:18:22.573599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926937+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:18:21.926937+0000","last_peered":"2023-10-18T09:18:21.926937+0000","last_clean":"2023-10-18T09:18:21.926937+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:18:21.926937+0000","last_undegraded":"2023-10-18T09:18:21.926937+0000","last_fullsized":"2023-10-18T09:18:21.926937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299862+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:18:23.299862+0000","last_peered":"2023-10-18T09:18:23.299862+0000","last_clean":"2023-10-18T09:18:23.299862+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:18:23.299862+0000","last_undegraded":"2023-10-18T09:18:23.299862+0000","last_fullsized":"2023-10-18T09:18:23.299862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930648+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:18:21.930648+0000","last_peered":"2023-10-18T09:18:21.930648+0000","last_clean":"2023-10-18T09:18:21.930648+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:18:21.930648+0000","last_undegraded":"2023-10-18T09:18:21.930648+0000","last_fullsized":"2023-10-18T09:18:21.930648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933262+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:18:21.933262+0000","last_peered":"2023-10-18T09:18:21.933262+0000","last_clean":"2023-10-18T09:18:21.933262+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:18:21.933262+0000","last_undegraded":"2023-10-18T09:18:21.933262+0000","last_fullsized":"2023-10-18T09:18:21.933262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926979+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:18:21.926979+0000","last_peered":"2023-10-18T09:18:21.926979+0000","last_clean":"2023-10-18T09:18:21.926979+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:18:21.926979+0000","last_undegraded":"2023-10-18T09:18:21.926979+0000","last_fullsized":"2023-10-18T09:18:21.926979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":232,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688656+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:18:40.688656+0000","last_peered":"2023-10-18T09:18:40.688656+0000","last_clean":"2023-10-18T09:18:40.688656+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:18:40.688656+0000","last_undegraded":"2023-10-18T09:18:40.688656+0000","last_fullsized":"2023-10-18T09:18:40.688656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934250+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:18:21.934250+0000","last_peered":"2023-10-18T09:18:21.934250+0000","last_clean":"2023-10-18T09:18:21.934250+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:18:21.934250+0000","last_undegraded":"2023-10-18T09:18:21.934250+0000","last_fullsized":"2023-10-18T09:18:21.934250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932772+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:18:21.932772+0000","last_peered":"2023-10-18T09:18:21.932772+0000","last_clean":"2023-10-18T09:18:21.932772+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:18:21.932772+0000","last_undegraded":"2023-10-18T09:18:21.932772+0000","last_fullsized":"2023-10-18T09:18:21.932772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574023+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:18:22.574023+0000","last_peered":"2023-10-18T09:18:22.574023+0000","last_clean":"2023-10-18T09:18:22.574023+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:18:22.574023+0000","last_undegraded":"2023-10-18T09:18:22.574023+0000","last_fullsized":"2023-10-18T09:18:22.574023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781543+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:18:22.781543+0000","last_peered":"2023-10-18T09:18:22.781543+0000","last_clean":"2023-10-18T09:18:22.781543+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:18:22.781543+0000","last_undegraded":"2023-10-18T09:18:22.781543+0000","last_fullsized":"2023-10-18T09:18:22.781543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927375+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:18:21.927375+0000","last_peered":"2023-10-18T09:18:21.927375+0000","last_clean":"2023-10-18T09:18:21.927375+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:18:21.927375+0000","last_undegraded":"2023-10-18T09:18:21.927375+0000","last_fullsized":"2023-10-18T09:18:21.927375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.929940+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:18:21.929940+0000","last_peered":"2023-10-18T09:18:21.929940+0000","last_clean":"2023-10-18T09:18:21.929940+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:18:21.929940+0000","last_undegraded":"2023-10-18T09:18:21.929940+0000","last_fullsized":"2023-10-18T09:18:21.929940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934572+0000","last_change":"2023-10-18T09:17:21.921019+0000","last_active":"2023-10-18T09:18:21.934572+0000","last_peered":"2023-10-18T09:18:21.934572+0000","last_clean":"2023-10-18T09:18:21.934572+0000","last_became_active":"2023-10-18T09:17:21.920611+0000","last_became_peered":"2023-10-18T09:17:21.920611+0000","last_unstale":"2023-10-18T09:18:21.934572+0000","last_undegraded":"2023-10-18T09:18:21.934572+0000","last_fullsized":"2023-10-18T09:18:21.934572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:44.317766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932701+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:18:21.932701+0000","last_peered":"2023-10-18T09:18:21.932701+0000","last_clean":"2023-10-18T09:18:21.932701+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:18:21.932701+0000","last_undegraded":"2023-10-18T09:18:21.932701+0000","last_fullsized":"2023-10-18T09:18:21.932701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":282,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688670+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:18:40.688670+0000","last_peered":"2023-10-18T09:18:40.688670+0000","last_clean":"2023-10-18T09:18:40.688670+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:18:40.688670+0000","last_undegraded":"2023-10-18T09:18:40.688670+0000","last_fullsized":"2023-10-18T09:18:40.688670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":221,"num_read_kb":177,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934847+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:18:21.934847+0000","last_peered":"2023-10-18T09:18:21.934847+0000","last_clean":"2023-10-18T09:18:21.934847+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:18:21.934847+0000","last_undegraded":"2023-10-18T09:18:21.934847+0000","last_fullsized":"2023-10-18T09:18:21.934847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930547+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:18:21.930547+0000","last_peered":"2023-10-18T09:18:21.930547+0000","last_clean":"2023-10-18T09:18:21.930547+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:18:21.930547+0000","last_undegraded":"2023-10-18T09:18:21.930547+0000","last_fullsized":"2023-10-18T09:18:21.930547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573816+0000","last_change":"2023-10-18T09:17:21.956449+0000","last_active":"2023-10-18T09:18:22.573816+0000","last_peered":"2023-10-18T09:18:22.573816+0000","last_clean":"2023-10-18T09:18:22.573816+0000","last_became_active":"2023-10-18T09:17:21.946784+0000","last_became_peered":"2023-10-18T09:17:21.946784+0000","last_unstale":"2023-10-18T09:18:22.573816+0000","last_undegraded":"2023-10-18T09:18:22.573816+0000","last_fullsized":"2023-10-18T09:18:22.573816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:49:21.415006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573678+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:18:22.573678+0000","last_peered":"2023-10-18T09:18:22.573678+0000","last_clean":"2023-10-18T09:18:22.573678+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:18:22.573678+0000","last_undegraded":"2023-10-18T09:18:22.573678+0000","last_fullsized":"2023-10-18T09:18:22.573678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573914+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:18:22.573914+0000","last_peered":"2023-10-18T09:18:22.573914+0000","last_clean":"2023-10-18T09:18:22.573914+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:18:22.573914+0000","last_undegraded":"2023-10-18T09:18:22.573914+0000","last_fullsized":"2023-10-18T09:18:22.573914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930527+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:18:21.930527+0000","last_peered":"2023-10-18T09:18:21.930527+0000","last_clean":"2023-10-18T09:18:21.930527+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:18:21.930527+0000","last_undegraded":"2023-10-18T09:18:21.930527+0000","last_fullsized":"2023-10-18T09:18:21.930527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574246+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:18:22.574246+0000","last_peered":"2023-10-18T09:18:22.574246+0000","last_clean":"2023-10-18T09:18:22.574246+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:18:22.574246+0000","last_undegraded":"2023-10-18T09:18:22.574246+0000","last_fullsized":"2023-10-18T09:18:22.574246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936027+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:18:21.936027+0000","last_peered":"2023-10-18T09:18:21.936027+0000","last_clean":"2023-10-18T09:18:21.936027+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:18:21.936027+0000","last_undegraded":"2023-10-18T09:18:21.936027+0000","last_fullsized":"2023-10-18T09:18:21.936027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684689+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:18:37.684689+0000","last_peered":"2023-10-18T09:18:37.684689+0000","last_clean":"2023-10-18T09:18:37.684689+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:18:37.684689+0000","last_undegraded":"2023-10-18T09:18:37.684689+0000","last_fullsized":"2023-10-18T09:18:37.684689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574313+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:18:22.574313+0000","last_peered":"2023-10-18T09:18:22.574313+0000","last_clean":"2023-10-18T09:18:22.574313+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:18:22.574313+0000","last_undegraded":"2023-10-18T09:18:22.574313+0000","last_fullsized":"2023-10-18T09:18:22.574313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574038+0000","last_change":"2023-10-18T09:17:21.948883+0000","last_active":"2023-10-18T09:18:22.574038+0000","last_peered":"2023-10-18T09:18:22.574038+0000","last_clean":"2023-10-18T09:18:22.574038+0000","last_became_active":"2023-10-18T09:17:21.948451+0000","last_became_peered":"2023-10-18T09:17:21.948451+0000","last_unstale":"2023-10-18T09:18:22.574038+0000","last_undegraded":"2023-10-18T09:18:22.574038+0000","last_fullsized":"2023-10-18T09:18:22.574038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:09:25.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933791+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:18:21.933791+0000","last_peered":"2023-10-18T09:18:21.933791+0000","last_clean":"2023-10-18T09:18:21.933791+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:18:21.933791+0000","last_undegraded":"2023-10-18T09:18:21.933791+0000","last_fullsized":"2023-10-18T09:18:21.933791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301638+0000","last_change":"2023-10-18T09:18:23.301638+0000","last_active":"2023-10-18T09:18:23.301638+0000","last_peered":"2023-10-18T09:18:23.301638+0000","last_clean":"2023-10-18T09:18:23.301638+0000","last_became_active":"2023-10-18T09:18:23.301186+0000","last_became_peered":"2023-10-18T09:18:23.301186+0000","last_unstale":"2023-10-18T09:18:23.301638+0000","last_undegraded":"2023-10-18T09:18:23.301638+0000","last_fullsized":"2023-10-18T09:18:23.301638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:34.532983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932148+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:18:21.932148+0000","last_peered":"2023-10-18T09:18:21.932148+0000","last_clean":"2023-10-18T09:18:21.932148+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:18:21.932148+0000","last_undegraded":"2023-10-18T09:18:21.932148+0000","last_fullsized":"2023-10-18T09:18:21.932148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935119+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:18:21.935119+0000","last_peered":"2023-10-18T09:18:21.935119+0000","last_clean":"2023-10-18T09:18:21.935119+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:18:21.935119+0000","last_undegraded":"2023-10-18T09:18:21.935119+0000","last_fullsized":"2023-10-18T09:18:21.935119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574093+0000","last_change":"2023-10-18T09:17:21.919795+0000","last_active":"2023-10-18T09:18:22.574093+0000","last_peered":"2023-10-18T09:18:22.574093+0000","last_clean":"2023-10-18T09:18:22.574093+0000","last_became_active":"2023-10-18T09:17:21.919419+0000","last_became_peered":"2023-10-18T09:17:21.919419+0000","last_unstale":"2023-10-18T09:18:22.574093+0000","last_undegraded":"2023-10-18T09:18:22.574093+0000","last_fullsized":"2023-10-18T09:18:22.574093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:45.138506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300194+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:18:23.300194+0000","last_peered":"2023-10-18T09:18:23.300194+0000","last_clean":"2023-10-18T09:18:23.300194+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:18:23.300194+0000","last_undegraded":"2023-10-18T09:18:23.300194+0000","last_fullsized":"2023-10-18T09:18:23.300194+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931248+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:18:22.931248+0000","last_peered":"2023-10-18T09:18:22.931248+0000","last_clean":"2023-10-18T09:18:22.931248+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:18:22.931248+0000","last_undegraded":"2023-10-18T09:18:22.931248+0000","last_fullsized":"2023-10-18T09:18:22.931248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300078+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:18:23.300078+0000","last_peered":"2023-10-18T09:18:23.300078+0000","last_clean":"2023-10-18T09:18:23.300078+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:18:23.300078+0000","last_undegraded":"2023-10-18T09:18:23.300078+0000","last_fullsized":"2023-10-18T09:18:23.300078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300839+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:18:23.300839+0000","last_peered":"2023-10-18T09:18:23.300839+0000","last_clean":"2023-10-18T09:18:23.300839+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:18:23.300839+0000","last_undegraded":"2023-10-18T09:18:23.300839+0000","last_fullsized":"2023-10-18T09:18:23.300839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.687858+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:18:40.687858+0000","last_peered":"2023-10-18T09:18:40.687858+0000","last_clean":"2023-10-18T09:18:40.687858+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:18:40.687858+0000","last_undegraded":"2023-10-18T09:18:40.687858+0000","last_fullsized":"2023-10-18T09:18:40.687858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301374+0000","last_change":"2023-10-18T09:17:21.920260+0000","last_active":"2023-10-18T09:18:23.301374+0000","last_peered":"2023-10-18T09:18:23.301374+0000","last_clean":"2023-10-18T09:18:23.301374+0000","last_became_active":"2023-10-18T09:17:21.919905+0000","last_became_peered":"2023-10-18T09:17:21.919905+0000","last_unstale":"2023-10-18T09:18:23.301374+0000","last_undegraded":"2023-10-18T09:18:23.301374+0000","last_fullsized":"2023-10-18T09:18:23.301374+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:29.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300727+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:18:23.300727+0000","last_peered":"2023-10-18T09:18:23.300727+0000","last_clean":"2023-10-18T09:18:23.300727+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:18:23.300727+0000","last_undegraded":"2023-10-18T09:18:23.300727+0000","last_fullsized":"2023-10-18T09:18:23.300727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931375+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:18:22.931375+0000","last_peered":"2023-10-18T09:18:22.931375+0000","last_clean":"2023-10-18T09:18:22.931375+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:18:22.931375+0000","last_undegraded":"2023-10-18T09:18:22.931375+0000","last_fullsized":"2023-10-18T09:18:22.931375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.686777+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:18:39.686777+0000","last_peered":"2023-10-18T09:18:39.686777+0000","last_clean":"2023-10-18T09:18:39.686777+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:18:39.686777+0000","last_undegraded":"2023-10-18T09:18:39.686777+0000","last_fullsized":"2023-10-18T09:18:39.686777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301085+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:18:23.301085+0000","last_peered":"2023-10-18T09:18:23.301085+0000","last_clean":"2023-10-18T09:18:23.301085+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:18:23.301085+0000","last_undegraded":"2023-10-18T09:18:23.301085+0000","last_fullsized":"2023-10-18T09:18:23.301085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282146+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:18:23.282146+0000","last_peered":"2023-10-18T09:18:23.282146+0000","last_clean":"2023-10-18T09:18:23.282146+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:18:23.282146+0000","last_undegraded":"2023-10-18T09:18:23.282146+0000","last_fullsized":"2023-10-18T09:18:23.282146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300391+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:18:23.300391+0000","last_peered":"2023-10-18T09:18:23.300391+0000","last_clean":"2023-10-18T09:18:23.300391+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:18:23.300391+0000","last_undegraded":"2023-10-18T09:18:23.300391+0000","last_fullsized":"2023-10-18T09:18:23.300391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573989+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:18:22.573989+0000","last_peered":"2023-10-18T09:18:22.573989+0000","last_clean":"2023-10-18T09:18:22.573989+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:18:22.573989+0000","last_undegraded":"2023-10-18T09:18:22.573989+0000","last_fullsized":"2023-10-18T09:18:22.573989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930232+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:18:21.930232+0000","last_peered":"2023-10-18T09:18:21.930232+0000","last_clean":"2023-10-18T09:18:21.930232+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:18:21.930232+0000","last_undegraded":"2023-10-18T09:18:21.930232+0000","last_fullsized":"2023-10-18T09:18:21.930232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299699+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:18:23.299699+0000","last_peered":"2023-10-18T09:18:23.299699+0000","last_clean":"2023-10-18T09:18:23.299699+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:18:23.299699+0000","last_undegraded":"2023-10-18T09:18:23.299699+0000","last_fullsized":"2023-10-18T09:18:23.299699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300183+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:18:23.300183+0000","last_peered":"2023-10-18T09:18:23.300183+0000","last_clean":"2023-10-18T09:18:23.300183+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:18:23.300183+0000","last_undegraded":"2023-10-18T09:18:23.300183+0000","last_fullsized":"2023-10-18T09:18:23.300183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930784+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:18:22.930784+0000","last_peered":"2023-10-18T09:18:22.930784+0000","last_clean":"2023-10-18T09:18:22.930784+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:18:22.930784+0000","last_undegraded":"2023-10-18T09:18:22.930784+0000","last_fullsized":"2023-10-18T09:18:22.930784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300567+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:18:23.300567+0000","last_peered":"2023-10-18T09:18:23.300567+0000","last_clean":"2023-10-18T09:18:23.300567+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:18:23.300567+0000","last_undegraded":"2023-10-18T09:18:23.300567+0000","last_fullsized":"2023-10-18T09:18:23.300567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300439+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:18:23.300439+0000","last_peered":"2023-10-18T09:18:23.300439+0000","last_clean":"2023-10-18T09:18:23.300439+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:23.300439+0000","last_undegraded":"2023-10-18T09:18:23.300439+0000","last_fullsized":"2023-10-18T09:18:23.300439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281950+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:18:23.281950+0000","last_peered":"2023-10-18T09:18:23.281950+0000","last_clean":"2023-10-18T09:18:23.281950+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:18:23.281950+0000","last_undegraded":"2023-10-18T09:18:23.281950+0000","last_fullsized":"2023-10-18T09:18:23.281950+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:25.214651+0000","last_change":"2023-10-18T09:17:21.962634+0000","last_active":"2023-10-18T09:18:25.214651+0000","last_peered":"2023-10-18T09:18:25.214651+0000","last_clean":"2023-10-18T09:18:25.214651+0000","last_became_active":"2023-10-18T09:17:21.948090+0000","last_became_peered":"2023-10-18T09:17:21.948090+0000","last_unstale":"2023-10-18T09:18:25.214651+0000","last_undegraded":"2023-10-18T09:18:25.214651+0000","last_fullsized":"2023-10-18T09:18:25.214651+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:12:14.390673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282465+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:18:23.282465+0000","last_peered":"2023-10-18T09:18:23.282465+0000","last_clean":"2023-10-18T09:18:23.282465+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:18:23.282465+0000","last_undegraded":"2023-10-18T09:18:23.282465+0000","last_fullsized":"2023-10-18T09:18:23.282465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934023+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:18:22.934023+0000","last_peered":"2023-10-18T09:18:22.934023+0000","last_clean":"2023-10-18T09:18:22.934023+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:18:22.934023+0000","last_undegraded":"2023-10-18T09:18:22.934023+0000","last_fullsized":"2023-10-18T09:18:22.934023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301074+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:18:23.301074+0000","last_peered":"2023-10-18T09:18:23.301074+0000","last_clean":"2023-10-18T09:18:23.301074+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:18:23.301074+0000","last_undegraded":"2023-10-18T09:18:23.301074+0000","last_fullsized":"2023-10-18T09:18:23.301074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300059+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:18:23.300059+0000","last_peered":"2023-10-18T09:18:23.300059+0000","last_clean":"2023-10-18T09:18:23.300059+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:18:23.300059+0000","last_undegraded":"2023-10-18T09:18:23.300059+0000","last_fullsized":"2023-10-18T09:18:23.300059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927146+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:18:21.927146+0000","last_peered":"2023-10-18T09:18:21.927146+0000","last_clean":"2023-10-18T09:18:21.927146+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:18:21.927146+0000","last_undegraded":"2023-10-18T09:18:21.927146+0000","last_fullsized":"2023-10-18T09:18:21.927146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.933383+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:18:22.933383+0000","last_peered":"2023-10-18T09:18:22.933383+0000","last_clean":"2023-10-18T09:18:22.933383+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:18:22.933383+0000","last_undegraded":"2023-10-18T09:18:22.933383+0000","last_fullsized":"2023-10-18T09:18:22.933383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300468+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:18:23.300468+0000","last_peered":"2023-10-18T09:18:23.300468+0000","last_clean":"2023-10-18T09:18:23.300468+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:18:23.300468+0000","last_undegraded":"2023-10-18T09:18:23.300468+0000","last_fullsized":"2023-10-18T09:18:23.300468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936113+0000","last_change":"2023-10-18T09:17:21.949440+0000","last_active":"2023-10-18T09:18:21.936113+0000","last_peered":"2023-10-18T09:18:21.936113+0000","last_clean":"2023-10-18T09:18:21.936113+0000","last_became_active":"2023-10-18T09:17:21.948562+0000","last_became_peered":"2023-10-18T09:17:21.948562+0000","last_unstale":"2023-10-18T09:18:21.936113+0000","last_undegraded":"2023-10-18T09:18:21.936113+0000","last_fullsized":"2023-10-18T09:18:21.936113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:20.976200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281948+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:18:23.281948+0000","last_peered":"2023-10-18T09:18:23.281948+0000","last_clean":"2023-10-18T09:18:23.281948+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:18:23.281948+0000","last_undegraded":"2023-10-18T09:18:23.281948+0000","last_fullsized":"2023-10-18T09:18:23.281948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930407+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:18:22.930407+0000","last_peered":"2023-10-18T09:18:22.930407+0000","last_clean":"2023-10-18T09:18:22.930407+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:18:22.930407+0000","last_undegraded":"2023-10-18T09:18:22.930407+0000","last_fullsized":"2023-10-18T09:18:22.930407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927797+0000","last_change":"2023-10-18T09:17:21.957405+0000","last_active":"2023-10-18T09:18:21.927797+0000","last_peered":"2023-10-18T09:18:21.927797+0000","last_clean":"2023-10-18T09:18:21.927797+0000","last_became_active":"2023-10-18T09:17:21.917875+0000","last_became_peered":"2023-10-18T09:17:21.917875+0000","last_unstale":"2023-10-18T09:18:21.927797+0000","last_undegraded":"2023-10-18T09:18:21.927797+0000","last_fullsized":"2023-10-18T09:18:21.927797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:37.429104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282264+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:18:23.282264+0000","last_peered":"2023-10-18T09:18:23.282264+0000","last_clean":"2023-10-18T09:18:23.282264+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:18:23.282264+0000","last_undegraded":"2023-10-18T09:18:23.282264+0000","last_fullsized":"2023-10-18T09:18:23.282264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927027+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:18:21.927027+0000","last_peered":"2023-10-18T09:18:21.927027+0000","last_clean":"2023-10-18T09:18:21.927027+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:18:21.927027+0000","last_undegraded":"2023-10-18T09:18:21.927027+0000","last_fullsized":"2023-10-18T09:18:21.927027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934530+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:18:21.934530+0000","last_peered":"2023-10-18T09:18:21.934530+0000","last_clean":"2023-10-18T09:18:21.934530+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:18:21.934530+0000","last_undegraded":"2023-10-18T09:18:21.934530+0000","last_fullsized":"2023-10-18T09:18:21.934530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300205+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:18:23.300205+0000","last_peered":"2023-10-18T09:18:23.300205+0000","last_clean":"2023-10-18T09:18:23.300205+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:18:23.300205+0000","last_undegraded":"2023-10-18T09:18:23.300205+0000","last_fullsized":"2023-10-18T09:18:23.300205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933939+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:18:21.933939+0000","last_peered":"2023-10-18T09:18:21.933939+0000","last_clean":"2023-10-18T09:18:21.933939+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:18:21.933939+0000","last_undegraded":"2023-10-18T09:18:21.933939+0000","last_fullsized":"2023-10-18T09:18:21.933939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.688241+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:18:39.688241+0000","last_peered":"2023-10-18T09:18:39.688241+0000","last_clean":"2023-10-18T09:18:39.688241+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:18:39.688241+0000","last_undegraded":"2023-10-18T09:18:39.688241+0000","last_fullsized":"2023-10-18T09:18:39.688241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301515+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:18:23.301515+0000","last_peered":"2023-10-18T09:18:23.301515+0000","last_clean":"2023-10-18T09:18:23.301515+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:18:23.301515+0000","last_undegraded":"2023-10-18T09:18:23.301515+0000","last_fullsized":"2023-10-18T09:18:23.301515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781014+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:18:22.781014+0000","last_peered":"2023-10-18T09:18:22.781014+0000","last_clean":"2023-10-18T09:18:22.781014+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:18:22.781014+0000","last_undegraded":"2023-10-18T09:18:22.781014+0000","last_fullsized":"2023-10-18T09:18:22.781014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930145+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:18:21.930145+0000","last_peered":"2023-10-18T09:18:21.930145+0000","last_clean":"2023-10-18T09:18:21.930145+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:18:21.930145+0000","last_undegraded":"2023-10-18T09:18:21.930145+0000","last_fullsized":"2023-10-18T09:18:21.930145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300529+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:18:23.300529+0000","last_peered":"2023-10-18T09:18:23.300529+0000","last_clean":"2023-10-18T09:18:23.300529+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:18:23.300529+0000","last_undegraded":"2023-10-18T09:18:23.300529+0000","last_fullsized":"2023-10-18T09:18:23.300529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930361+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:18:21.930361+0000","last_peered":"2023-10-18T09:18:21.930361+0000","last_clean":"2023-10-18T09:18:21.930361+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:18:21.930361+0000","last_undegraded":"2023-10-18T09:18:21.930361+0000","last_fullsized":"2023-10-18T09:18:21.930361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300587+0000","last_change":"2023-10-18T09:17:21.966321+0000","last_active":"2023-10-18T09:18:23.300587+0000","last_peered":"2023-10-18T09:18:23.300587+0000","last_clean":"2023-10-18T09:18:23.300587+0000","last_became_active":"2023-10-18T09:17:21.949466+0000","last_became_peered":"2023-10-18T09:17:21.949466+0000","last_unstale":"2023-10-18T09:18:23.300587+0000","last_undegraded":"2023-10-18T09:18:23.300587+0000","last_fullsized":"2023-10-18T09:18:23.300587+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:05.856643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300969+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:18:23.300969+0000","last_peered":"2023-10-18T09:18:23.300969+0000","last_clean":"2023-10-18T09:18:23.300969+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:18:23.300969+0000","last_undegraded":"2023-10-18T09:18:23.300969+0000","last_fullsized":"2023-10-18T09:18:23.300969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930499+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:18:22.930499+0000","last_peered":"2023-10-18T09:18:22.930499+0000","last_clean":"2023-10-18T09:18:22.930499+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:18:22.930499+0000","last_undegraded":"2023-10-18T09:18:22.930499+0000","last_fullsized":"2023-10-18T09:18:22.930499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934230+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:18:22.934230+0000","last_peered":"2023-10-18T09:18:22.934230+0000","last_clean":"2023-10-18T09:18:22.934230+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:22.934230+0000","last_undegraded":"2023-10-18T09:18:22.934230+0000","last_fullsized":"2023-10-18T09:18:22.934230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300500+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:18:23.300500+0000","last_peered":"2023-10-18T09:18:23.300500+0000","last_clean":"2023-10-18T09:18:23.300500+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:18:23.300500+0000","last_undegraded":"2023-10-18T09:18:23.300500+0000","last_fullsized":"2023-10-18T09:18:23.300500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300611+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:18:23.300611+0000","last_peered":"2023-10-18T09:18:23.300611+0000","last_clean":"2023-10-18T09:18:23.300611+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:18:23.300611+0000","last_undegraded":"2023-10-18T09:18:23.300611+0000","last_fullsized":"2023-10-18T09:18:23.300611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931115+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:18:22.931115+0000","last_peered":"2023-10-18T09:18:22.931115+0000","last_clean":"2023-10-18T09:18:22.931115+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:18:22.931115+0000","last_undegraded":"2023-10-18T09:18:22.931115+0000","last_fullsized":"2023-10-18T09:18:22.931115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934563+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:18:22.934563+0000","last_peered":"2023-10-18T09:18:22.934563+0000","last_clean":"2023-10-18T09:18:22.934563+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:18:22.934563+0000","last_undegraded":"2023-10-18T09:18:22.934563+0000","last_fullsized":"2023-10-18T09:18:22.934563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282527+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:18:23.282527+0000","last_peered":"2023-10-18T09:18:23.282527+0000","last_clean":"2023-10-18T09:18:23.282527+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:18:23.282527+0000","last_undegraded":"2023-10-18T09:18:23.282527+0000","last_fullsized":"2023-10-18T09:18:23.282527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931260+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:18:22.931260+0000","last_peered":"2023-10-18T09:18:22.931260+0000","last_clean":"2023-10-18T09:18:22.931260+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:18:22.931260+0000","last_undegraded":"2023-10-18T09:18:22.931260+0000","last_fullsized":"2023-10-18T09:18:22.931260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282117+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:18:23.282117+0000","last_peered":"2023-10-18T09:18:23.282117+0000","last_clean":"2023-10-18T09:18:23.282117+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:18:23.282117+0000","last_undegraded":"2023-10-18T09:18:23.282117+0000","last_fullsized":"2023-10-18T09:18:23.282117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934304+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:18:21.934304+0000","last_peered":"2023-10-18T09:18:21.934304+0000","last_clean":"2023-10-18T09:18:21.934304+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:18:21.934304+0000","last_undegraded":"2023-10-18T09:18:21.934304+0000","last_fullsized":"2023-10-18T09:18:21.934304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300332+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:18:23.300332+0000","last_peered":"2023-10-18T09:18:23.300332+0000","last_clean":"2023-10-18T09:18:23.300332+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:18:23.300332+0000","last_undegraded":"2023-10-18T09:18:23.300332+0000","last_fullsized":"2023-10-18T09:18:23.300332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574002+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:18:22.574002+0000","last_peered":"2023-10-18T09:18:22.574002+0000","last_clean":"2023-10-18T09:18:22.574002+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:18:22.574002+0000","last_undegraded":"2023-10-18T09:18:22.574002+0000","last_fullsized":"2023-10-18T09:18:22.574002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74185,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1072,"num_read_kb":621,"num_write":253,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":510815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.624},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.83199999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.874},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.89500000000000002,"15min":0.81000000000000005},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.055,"5min":13.055,"15min":13.055},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.008,"5min":1.407,"15min":1.407},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.89000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.133,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.052,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.107,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.94499999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1168560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69799999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.747,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42899999999999999,"15min":0.436},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.29599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93700000000000006,"5min":1.196,"15min":1.196},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71999999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77100000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74199999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.242,"15min":1.242},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.141,"15min":1.141},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86199999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.127,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.95099999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91300000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.54700000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93100000000000005,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0469999999999999,"5min":1.532,"15min":1.532},"last":0.86899999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.66400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88700000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":510677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60199999999999998,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1110000000000002,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.502},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.0510000000000002,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.0790000000000002,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81799999999999995,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.8730000000000002,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.85899999999999999,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.82999999999999996,"15min":0.75700000000000001},"min":{"1min":0.53600000000000003,"5min":0.495,"15min":0.495},"max":{"1min":7.1909999999999998,"5min":7.1909999999999998,"15min":7.1909999999999998},"last":0.876}]},{"osd":3,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.754,"15min":0.72599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":4.766,"5min":4.766,"15min":4.766},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.82699999999999996},"min":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.609,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.46899999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88900000000000001,"15min":0.86499999999999999},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.7789999999999999,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88100000000000001,"15min":0.85899999999999999},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.6500000000000004,"5min":4.6500000000000004,"15min":4.6500000000000004},"last":1.0629999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92500000000000004,"15min":0.91200000000000003},"min":{"1min":0.62,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98499999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.63,"15min":0.63},"max":{"1min":3.8140000000000001,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0029999999999999,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":3.6930000000000001,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":0.96999999999999997},"min":{"1min":0.71699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":3.7599999999999998,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1,"15min":0.96499999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9199999999999999,"5min":6.3529999999999998,"15min":6.3529999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0009999999999999,"15min":0.97799999999999998},"min":{"1min":0.69099999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.7360000000000002,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.89600000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.204,"15min":1.2549999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.1060000000000001,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.089,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.123,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":1.1020000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.443}]},{"osd":1,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.38200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.63400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65100000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72699999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71799999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":1.254,"15min":1.254},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78200000000000003,"15min":0.754},"min":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.746,"15min":0.71499999999999997},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.80700000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96599999999999997,"5min":1.05,"15min":1.05},"last":0.82799999999999996}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.502,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.42299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69899999999999995,"5min":0.876,"15min":0.876},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.745,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.46700000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83299999999999996,"5min":1.02,"15min":1.02},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53500000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.52000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.754},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72099999999999997,"15min":0.69499999999999995},"min":{"1min":0.495,"5min":0.433,"15min":0.433},"max":{"1min":0.80000000000000004,"5min":1.173,"15min":1.173},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61899999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71599999999999997,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.58399999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80700000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83499999999999996}]},{"osd":9,"last update":"Wed Oct 18 09:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.65900000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.93799999999999994,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.876},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.95699999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.93600000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.93700000000000006}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.44400000000000001},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.69399999999999995,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.69199999999999995,"15min":0.63600000000000001},"min":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.70099999999999996,"15min":0.66700000000000004},"min":{"1min":0.58599999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64300000000000002,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.75,"15min":0.70299999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4570000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.76600000000000001,"15min":0.72699999999999998},"min":{"1min":0.59299999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91100000000000003,"15min":0.80900000000000005},"min":{"1min":0.63100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7789999999999999,"5min":11.112,"15min":11.112},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83299999999999996,"15min":0.78300000000000003},"min":{"1min":0.64000000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.83499999999999996,"15min":0.80300000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.68899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5269999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2649999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.874,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.91000000000000003,"15min":0.89700000000000002},"min":{"1min":0.78300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.752}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.95099999999999996,"15min":0.93500000000000005},"min":{"1min":0.78800000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3740000000000001,"5min":2.847,"15min":2.847},"last":0.66600000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0209999999999999},"min":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3420000000000001,"5min":2.802,"15min":2.802},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.747,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4670000000000001,"5min":1.581,"15min":1.581},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":568217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.7130000000000001,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60399999999999998,"15min":0.53700000000000003},"min":{"1min":0.47299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.435},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58799999999999997,"15min":0.54900000000000004},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.52700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.42899999999999999,"15min":0.40000000000000002},"min":{"1min":0.34100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.501,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.59199999999999997},"min":{"1min":0.48899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":5.258,"15min":5.258},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.69699999999999995,"15min":0.66100000000000003},"min":{"1min":0.51400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85899999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.502,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89400000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61799999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.65600000000000003},"min":{"1min":0.57799999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.92500000000000004,"5min":1.073,"15min":1.073},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.64700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84399999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73699999999999999,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.484,"5min":1.605,"15min":1.605},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.79200000000000004,"15min":0.76200000000000001},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.83099999999999996}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76300000000000001,"15min":0.74399999999999999},"min":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.91000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":507637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.68899999999999995,"15min":0.55900000000000005},"min":{"1min":0.46899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":4.4989999999999997,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.65100000000000002,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.72999999999999998,"15min":0.624},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.68600000000000005,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.121,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.3679999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78800000000000003,"15min":1.0409999999999999},"min":{"1min":0.442,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.353,"5min":11.539,"15min":11.539},"last":0.505},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.82399999999999995,"15min":1.034},"min":{"1min":0.51800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":2.7370000000000001,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.77900000000000003,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.755,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2130000000000001,"5min":1.448,"15min":1.448},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Oct 18 09:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.079,"5min":0.84399999999999997,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1,"15min":1.02},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.3319999999999999,"5min":2.8540000000000001,"15min":2.8540000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.872,"15min":0.877},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.409},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.873,"15min":0.874},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.155,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":0.94299999999999995,"15min":0.91000000000000003},"min":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.101,"5min":0.95799999999999996,"15min":0.94599999999999995},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94399999999999995,"15min":0.90500000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.2909999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.006}]},{"osd":11,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.95999999999999996,"15min":0.91800000000000004},"min":{"1min":0.68100000000000005,"5min":0.625,"15min":0.625},"max":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.075,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6919999999999999,"5min":1.698,"15min":1.698},"last":0.95899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.60999999999999999},"min":{"1min":0.33200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76300000000000001,"5min":3.016,"15min":3.016},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91000000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60699999999999998,"15min":0.54700000000000004},"min":{"1min":0.439,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.98399999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.66100000000000003,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.315,"15min":0.315},"max":{"1min":7.2140000000000004,"5min":7.2140000000000004,"15min":7.2140000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.41499999999999998},"min":{"1min":0.35999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.76600000000000001,"15min":0.66000000000000003},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1749999999999998,"5min":7.1749999999999998,"15min":7.1749999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.59899999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95899999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.75600000000000001},"min":{"1min":0.58899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1000000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.206,"15min":1.206},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81799999999999995,"15min":0.78600000000000003},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.038,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71499999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0629999999999999,"5min":1.093,"15min":1.093},"last":0.84699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":505889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.47499999999999998,"15min":0.40600000000000003},"min":{"1min":0.38200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.498,"15min":0.433},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.73799999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":7.516,"5min":7.516,"15min":7.516},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.65800000000000003,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.68500000000000005,"15min":0.64200000000000002},"min":{"1min":0.61299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.65300000000000002},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.68999999999999995,"15min":0.61099999999999999},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":0.76600000000000001,"15min":0.64100000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.65900000000000003},"min":{"1min":0.60199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.623},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.68799999999999994},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.27,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.877}]},{"osd":6,"last update":"Wed Oct 18 09:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.67000000000000004,"15min":0.62},"min":{"1min":0.47899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68100000000000005,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.66700000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.83699999999999997}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.83899999999999997,"15min":0.82699999999999996},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89000000000000001,"15min":0.90800000000000003},"min":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.89000000000000001},"min":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.282,"5min":0.95099999999999996,"15min":0.89500000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":7.6070000000000002,"5min":7.6070000000000002,"15min":7.6070000000000002},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.2829999999999999,"15min":1.353},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1220000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.3220000000000001,"15min":1.393},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1830000000000001,"5min":7.6769999999999996,"15min":7.6769999999999996},"last":0.80300000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":511102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.373},"min":{"1min":0.35499999999999998,"5min":0.186,"15min":0.186},"max":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.39600000000000002},"min":{"1min":0.32300000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.60599999999999998,"5min":0.753,"15min":0.753},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.625},{"interface":"front","average":{"1min":1.048,"5min":0.68899999999999995,"15min":0.60099999999999998},"min":{"1min":0.52500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":8.7300000000000004,"5min":8.7300000000000004,"15min":8.7300000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":1.077,"15min":1.077},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.629,"15min":0.56899999999999995},"min":{"1min":0.53000000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.59399999999999997},"min":{"1min":0.623,"5min":0.374,"15min":0.374},"max":{"1min":0.91800000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.64100000000000001},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0329999999999999,"5min":1.131,"15min":1.131},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.624,"5min":0.378,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.65700000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97799999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.52800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.99299999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0129999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.53000000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.99099999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69399999999999995}]},{"osd":11,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.68300000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1,"5min":1,"15min":1},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67200000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1103274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.60099999999999998,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.0570000000000004,"5min":6.0570000000000004,"15min":6.0570000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80300000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.56100000000000005},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.88,"15min":0.88},"last":0.76700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41999999999999998,"15min":0.38},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59699999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.503}]},{"osd":3,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.56899999999999995},"min":{"1min":0.496,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.623},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.82999999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.60799999999999998},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.872,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.89500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:18:44.273 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-18T09:18:44.273 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-18T09:18:44.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 -- 172.21.15.61:0/3709261122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 msgr2=0x7f936c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 --2- 172.21.15.61:0/3709261122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c073670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f935c0098f0 tx=0x7f935c009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:44.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 -- 172.21.15.61:0/3709261122 shutdown_connections 2023-10-18T09:18:44.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 --2- 172.21.15.61:0/3709261122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 0x7f936c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 --2- 172.21.15.61:0/3709261122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 --2- 172.21.15.61:0/3709261122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c073670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 -- 172.21.15.61:0/3709261122 >> 172.21.15.61:0/3709261122 conn(0x7f936c0fbc90 msgr2=0x7f936c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:44.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 -- 172.21.15.61:0/3709261122 shutdown_connections 2023-10-18T09:18:44.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.508+0000 7f93743ff700 1 -- 172.21.15.61:0/3709261122 wait complete. 2023-10-18T09:18:44.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 Processor -- start 2023-10-18T09:18:44.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 -- start start 2023-10-18T09:18:44.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 0x7f936c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1a0480 con 0x7f936c0747d0 2023-10-18T09:18:44.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1a05c0 con 0x7f936c108610 2023-10-18T09:18:44.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.509+0000 7f93743ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1a0750 con 0x7f936c0731f0 2023-10-18T09:18:44.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937299c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 0x7f936c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34610/0 (socket says 172.21.15.61:34610) 2023-10-18T09:18:44.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 -- 172.21.15.61:0/1895502018 learned_addr learned my addr 172.21.15.61:0/1895502018 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:44.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937219b700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 -- 172.21.15.61:0/1895502018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 msgr2=0x7f936c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 -- 172.21.15.61:0/1895502018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 msgr2=0x7f936c19feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 0x7f936c19feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 -- 172.21.15.61:0/1895502018 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93680095f0 con 0x7f936c0747d0 2023-10-18T09:18:44.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.510+0000 7f937199a700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c00bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:44.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.511+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c01c070 con 0x7f936c0747d0 2023-10-18T09:18:44.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.511+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f935c00bb20 con 0x7f936c0747d0 2023-10-18T09:18:44.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.511+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c009520 con 0x7f936c0747d0 2023-10-18T09:18:44.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.511+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c004930 con 0x7f936c0747d0 2023-10-18T09:18:44.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.511+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c1a0c60 con 0x7f936c0747d0 2023-10-18T09:18:44.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.512+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c107420 con 0x7f936c0747d0 2023-10-18T09:18:44.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.514+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f935c020070 con 0x7f936c0747d0 2023-10-18T09:18:44.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.515+0000 7f93637fe700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 0x7f93580a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:44.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.515+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f935c0cddd0 con 0x7f936c0747d0 2023-10-18T09:18:44.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.515+0000 7f937219b700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 0x7f93580a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:44.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.516+0000 7f937219b700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 0x7f93580a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9368005fd0 tx=0x7f9368009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:44.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.519+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f935c097110 con 0x7f936c0747d0 2023-10-18T09:18:44.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.729+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 --> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f936c02cfd0 con 0x7f93580a1480 2023-10-18T09:18:44.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.743+0000 7f93637fe700 1 -- 172.21.15.61:0/1895502018 <== mgr.14150 v2:172.21.15.61:6800/3155127227 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427392 (secure 0 0 0) 0x7f936c02cfd0 con 0x7f93580a1480 2023-10-18T09:18:44.751 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:44.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 msgr2=0x7f93580a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 0x7f93580a3940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9368005fd0 tx=0x7f9368009310 comp rx=0 tx=0).stop 2023-10-18T09:18:44.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 msgr2=0x7f936c198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:44.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f935c009230 tx=0x7f935c00bef0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 shutdown_connections 2023-10-18T09:18:44.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f93580a1480 0x7f93580a3940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f936c108610 0x7f936c19feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f936c0731f0 0x7f936c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 --2- 172.21.15.61:0/1895502018 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f936c0747d0 0x7f936c198e90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:44.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.748+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 >> 172.21.15.61:0/1895502018 conn(0x7f936c0fbc90 msgr2=0x7f936c104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:44.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.749+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 shutdown_connections 2023-10-18T09:18:44.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:44.749+0000 7f93743ff700 1 -- 172.21.15.61:0/1895502018 wait complete. 2023-10-18T09:18:44.757 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-10-18T09:18:44.770 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2023-10-18T09:18:42.866940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664799,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1198,"num_read_kb":730,"num_write":436,"num_write_kb":3095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180927,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":861,"ondisk_log_size":861,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338612,"kb_used_data":14376,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603212,"statfs":{"total":1151940427776,"available":1151593689088,"internally_reserved":0,"allocated":14721024,"data_stored":8085621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":331855174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6011,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":78,"num_write":5,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931324+0000","last_change":"2023-10-18T09:16:26.515530+0000","last_active":"2023-10-18T09:18:22.931324+0000","last_peered":"2023-10-18T09:18:22.931324+0000","last_clean":"2023-10-18T09:18:22.931324+0000","last_became_active":"2023-10-18T09:16:26.514909+0000","last_became_peered":"2023-10-18T09:16:26.514909+0000","last_unstale":"2023-10-18T09:18:22.931324+0000","last_undegraded":"2023-10-18T09:18:22.931324+0000","last_fullsized":"2023-10-18T09:18:22.931324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:10:00.685777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299959+0000","last_change":"2023-10-18T09:16:26.524627+0000","last_active":"2023-10-18T09:18:23.299959+0000","last_peered":"2023-10-18T09:18:23.299959+0000","last_clean":"2023-10-18T09:18:23.299959+0000","last_became_active":"2023-10-18T09:16:26.524281+0000","last_became_peered":"2023-10-18T09:16:26.524281+0000","last_unstale":"2023-10-18T09:18:23.299959+0000","last_undegraded":"2023-10-18T09:18:23.299959+0000","last_fullsized":"2023-10-18T09:18:23.299959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:15:27.162654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300618+0000","last_change":"2023-10-18T09:16:26.513514+0000","last_active":"2023-10-18T09:18:23.300618+0000","last_peered":"2023-10-18T09:18:23.300618+0000","last_clean":"2023-10-18T09:18:23.300618+0000","last_became_active":"2023-10-18T09:16:26.513163+0000","last_became_peered":"2023-10-18T09:16:26.513163+0000","last_unstale":"2023-10-18T09:18:23.300618+0000","last_undegraded":"2023-10-18T09:18:23.300618+0000","last_fullsized":"2023-10-18T09:18:23.300618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:21:52.219653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300357+0000","last_change":"2023-10-18T09:16:26.512965+0000","last_active":"2023-10-18T09:18:23.300357+0000","last_peered":"2023-10-18T09:18:23.300357+0000","last_clean":"2023-10-18T09:18:23.300357+0000","last_became_active":"2023-10-18T09:16:26.512697+0000","last_became_peered":"2023-10-18T09:16:26.512697+0000","last_unstale":"2023-10-18T09:18:23.300357+0000","last_undegraded":"2023-10-18T09:18:23.300357+0000","last_fullsized":"2023-10-18T09:18:23.300357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:20:54.185689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934459+0000","last_change":"2023-10-18T09:16:26.511088+0000","last_active":"2023-10-18T09:18:21.934459+0000","last_peered":"2023-10-18T09:18:21.934459+0000","last_clean":"2023-10-18T09:18:21.934459+0000","last_became_active":"2023-10-18T09:16:26.510743+0000","last_became_peered":"2023-10-18T09:16:26.510743+0000","last_unstale":"2023-10-18T09:18:21.934459+0000","last_undegraded":"2023-10-18T09:18:21.934459+0000","last_fullsized":"2023-10-18T09:18:21.934459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:40:14.344148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574887+0000","last_change":"2023-10-18T09:16:26.521664+0000","last_active":"2023-10-18T09:18:22.574887+0000","last_peered":"2023-10-18T09:18:22.574887+0000","last_clean":"2023-10-18T09:18:22.574887+0000","last_became_active":"2023-10-18T09:16:26.521288+0000","last_became_peered":"2023-10-18T09:16:26.521288+0000","last_unstale":"2023-10-18T09:18:22.574887+0000","last_undegraded":"2023-10-18T09:18:22.574887+0000","last_fullsized":"2023-10-18T09:18:22.574887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:04:31.714930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574959+0000","last_change":"2023-10-18T09:16:26.521544+0000","last_active":"2023-10-18T09:18:22.574959+0000","last_peered":"2023-10-18T09:18:22.574959+0000","last_clean":"2023-10-18T09:18:22.574959+0000","last_became_active":"2023-10-18T09:16:26.521141+0000","last_became_peered":"2023-10-18T09:16:26.521141+0000","last_unstale":"2023-10-18T09:18:22.574959+0000","last_undegraded":"2023-10-18T09:18:22.574959+0000","last_fullsized":"2023-10-18T09:18:22.574959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:56:49.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926842+0000","last_change":"2023-10-18T09:16:26.513838+0000","last_active":"2023-10-18T09:18:21.926842+0000","last_peered":"2023-10-18T09:18:21.926842+0000","last_clean":"2023-10-18T09:18:21.926842+0000","last_became_active":"2023-10-18T09:16:26.512396+0000","last_became_peered":"2023-10-18T09:16:26.512396+0000","last_unstale":"2023-10-18T09:18:21.926842+0000","last_undegraded":"2023-10-18T09:18:21.926842+0000","last_fullsized":"2023-10-18T09:18:21.926842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:39:33.019915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934246+0000","last_change":"2023-10-18T09:16:26.510999+0000","last_active":"2023-10-18T09:18:21.934246+0000","last_peered":"2023-10-18T09:18:21.934246+0000","last_clean":"2023-10-18T09:18:21.934246+0000","last_became_active":"2023-10-18T09:16:26.510652+0000","last_became_peered":"2023-10-18T09:16:26.510652+0000","last_unstale":"2023-10-18T09:18:21.934246+0000","last_undegraded":"2023-10-18T09:18:21.934246+0000","last_fullsized":"2023-10-18T09:18:21.934246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:52:49.648845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.575106+0000","last_change":"2023-10-18T09:16:26.518793+0000","last_active":"2023-10-18T09:18:22.575106+0000","last_peered":"2023-10-18T09:18:22.575106+0000","last_clean":"2023-10-18T09:18:22.575106+0000","last_became_active":"2023-10-18T09:16:26.518481+0000","last_became_peered":"2023-10-18T09:16:26.518481+0000","last_unstale":"2023-10-18T09:18:22.575106+0000","last_undegraded":"2023-10-18T09:18:22.575106+0000","last_fullsized":"2023-10-18T09:18:22.575106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:42:20.841916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574171+0000","last_change":"2023-10-18T09:16:26.511389+0000","last_active":"2023-10-18T09:18:22.574171+0000","last_peered":"2023-10-18T09:18:22.574171+0000","last_clean":"2023-10-18T09:18:22.574171+0000","last_became_active":"2023-10-18T09:16:26.511050+0000","last_became_peered":"2023-10-18T09:16:26.511050+0000","last_unstale":"2023-10-18T09:18:22.574171+0000","last_undegraded":"2023-10-18T09:18:22.574171+0000","last_fullsized":"2023-10-18T09:18:22.574171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:24:40.216915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927896+0000","last_change":"2023-10-18T09:16:26.516596+0000","last_active":"2023-10-18T09:18:21.927896+0000","last_peered":"2023-10-18T09:18:21.927896+0000","last_clean":"2023-10-18T09:18:21.927896+0000","last_became_active":"2023-10-18T09:16:26.516278+0000","last_became_peered":"2023-10-18T09:16:26.516278+0000","last_unstale":"2023-10-18T09:18:21.927896+0000","last_undegraded":"2023-10-18T09:18:21.927896+0000","last_fullsized":"2023-10-18T09:18:21.927896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:52:54.659084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573750+0000","last_change":"2023-10-18T09:16:26.517443+0000","last_active":"2023-10-18T09:18:22.573750+0000","last_peered":"2023-10-18T09:18:22.573750+0000","last_clean":"2023-10-18T09:18:22.573750+0000","last_became_active":"2023-10-18T09:16:26.516920+0000","last_became_peered":"2023-10-18T09:16:26.516920+0000","last_unstale":"2023-10-18T09:18:22.573750+0000","last_undegraded":"2023-10-18T09:18:22.573750+0000","last_fullsized":"2023-10-18T09:18:22.573750+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:38:48.549037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781028+0000","last_change":"2023-10-18T09:16:26.518458+0000","last_active":"2023-10-18T09:18:22.781028+0000","last_peered":"2023-10-18T09:18:22.781028+0000","last_clean":"2023-10-18T09:18:22.781028+0000","last_became_active":"2023-10-18T09:16:26.514142+0000","last_became_peered":"2023-10-18T09:16:26.514142+0000","last_unstale":"2023-10-18T09:18:22.781028+0000","last_undegraded":"2023-10-18T09:18:22.781028+0000","last_fullsized":"2023-10-18T09:18:22.781028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:59:59.134996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781064+0000","last_change":"2023-10-18T09:16:26.523274+0000","last_active":"2023-10-18T09:18:22.781064+0000","last_peered":"2023-10-18T09:18:22.781064+0000","last_clean":"2023-10-18T09:18:22.781064+0000","last_became_active":"2023-10-18T09:16:26.521499+0000","last_became_peered":"2023-10-18T09:16:26.521499+0000","last_unstale":"2023-10-18T09:18:22.781064+0000","last_undegraded":"2023-10-18T09:18:22.781064+0000","last_fullsized":"2023-10-18T09:18:22.781064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:08:45.089970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933702+0000","last_change":"2023-10-18T09:16:26.510141+0000","last_active":"2023-10-18T09:18:21.933702+0000","last_peered":"2023-10-18T09:18:21.933702+0000","last_clean":"2023-10-18T09:18:21.933702+0000","last_became_active":"2023-10-18T09:16:26.509860+0000","last_became_peered":"2023-10-18T09:16:26.509860+0000","last_unstale":"2023-10-18T09:18:21.933702+0000","last_undegraded":"2023-10-18T09:18:21.933702+0000","last_fullsized":"2023-10-18T09:18:21.933702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:00:12.990312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574196+0000","last_change":"2023-10-18T09:16:26.515705+0000","last_active":"2023-10-18T09:18:22.574196+0000","last_peered":"2023-10-18T09:18:22.574196+0000","last_clean":"2023-10-18T09:18:22.574196+0000","last_became_active":"2023-10-18T09:16:26.515327+0000","last_became_peered":"2023-10-18T09:16:26.515327+0000","last_unstale":"2023-10-18T09:18:22.574196+0000","last_undegraded":"2023-10-18T09:18:22.574196+0000","last_fullsized":"2023-10-18T09:18:22.574196+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:53:54.074098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936722+0000","last_change":"2023-10-18T09:16:26.522339+0000","last_active":"2023-10-18T09:18:21.936722+0000","last_peered":"2023-10-18T09:18:21.936722+0000","last_clean":"2023-10-18T09:18:21.936722+0000","last_became_active":"2023-10-18T09:16:26.522040+0000","last_became_peered":"2023-10-18T09:16:26.522040+0000","last_unstale":"2023-10-18T09:18:21.936722+0000","last_undegraded":"2023-10-18T09:18:21.936722+0000","last_fullsized":"2023-10-18T09:18:21.936722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:41:31.916315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926926+0000","last_change":"2023-10-18T09:16:26.514249+0000","last_active":"2023-10-18T09:18:21.926926+0000","last_peered":"2023-10-18T09:18:21.926926+0000","last_clean":"2023-10-18T09:18:21.926926+0000","last_became_active":"2023-10-18T09:16:26.513965+0000","last_became_peered":"2023-10-18T09:16:26.513965+0000","last_unstale":"2023-10-18T09:18:21.926926+0000","last_undegraded":"2023-10-18T09:18:21.926926+0000","last_fullsized":"2023-10-18T09:18:21.926926+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:08.215127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926410+0000","last_change":"2023-10-18T09:16:30.910085+0000","last_active":"2023-10-18T09:18:21.926410+0000","last_peered":"2023-10-18T09:18:21.926410+0000","last_clean":"2023-10-18T09:18:21.926410+0000","last_became_active":"2023-10-18T09:16:30.909666+0000","last_became_peered":"2023-10-18T09:16:30.909666+0000","last_unstale":"2023-10-18T09:18:21.926410+0000","last_undegraded":"2023-10-18T09:18:21.926410+0000","last_fullsized":"2023-10-18T09:18:21.926410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:30:46.535031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":136,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:38.684407+0000","last_change":"2023-10-18T09:16:25.870001+0000","last_active":"2023-10-18T09:18:38.684407+0000","last_peered":"2023-10-18T09:18:38.684407+0000","last_clean":"2023-10-18T09:18:38.684407+0000","last_became_active":"2023-10-18T09:16:25.869674+0000","last_became_peered":"2023-10-18T09:16:25.869674+0000","last_unstale":"2023-10-18T09:18:38.684407+0000","last_undegraded":"2023-10-18T09:18:38.684407+0000","last_fullsized":"2023-10-18T09:18:38.684407+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:37:42.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.936374+0000","last_change":"2023-10-18T09:18:22.935678+0000","last_active":"2023-10-18T09:18:22.936374+0000","last_peered":"2023-10-18T09:18:22.936374+0000","last_clean":"2023-10-18T09:18:22.936374+0000","last_became_active":"2023-10-18T09:18:22.935128+0000","last_became_peered":"2023-10-18T09:18:22.935128+0000","last_unstale":"2023-10-18T09:18:22.936374+0000","last_undegraded":"2023-10-18T09:18:22.936374+0000","last_fullsized":"2023-10-18T09:18:22.936374+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:33:23.547809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926645+0000","last_change":"2023-10-18T09:16:30.911174+0000","last_active":"2023-10-18T09:18:21.926645+0000","last_peered":"2023-10-18T09:18:21.926645+0000","last_clean":"2023-10-18T09:18:21.926645+0000","last_became_active":"2023-10-18T09:16:30.910678+0000","last_became_peered":"2023-10-18T09:16:30.910678+0000","last_unstale":"2023-10-18T09:18:21.926645+0000","last_undegraded":"2023-10-18T09:18:21.926645+0000","last_fullsized":"2023-10-18T09:18:21.926645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:05:53.866299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282769+0000","last_change":"2023-10-18T09:16:25.875200+0000","last_active":"2023-10-18T09:18:23.282769+0000","last_peered":"2023-10-18T09:18:23.282769+0000","last_clean":"2023-10-18T09:18:23.282769+0000","last_became_active":"2023-10-18T09:16:25.874601+0000","last_became_peered":"2023-10-18T09:16:25.874601+0000","last_unstale":"2023-10-18T09:18:23.282769+0000","last_undegraded":"2023-10-18T09:18:23.282769+0000","last_fullsized":"2023-10-18T09:18:23.282769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:00:40.850037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300027+0000","last_change":"2023-10-18T09:16:26.520553+0000","last_active":"2023-10-18T09:18:23.300027+0000","last_peered":"2023-10-18T09:18:23.300027+0000","last_clean":"2023-10-18T09:18:23.300027+0000","last_became_active":"2023-10-18T09:16:26.519955+0000","last_became_peered":"2023-10-18T09:16:26.519955+0000","last_unstale":"2023-10-18T09:18:23.300027+0000","last_undegraded":"2023-10-18T09:18:23.300027+0000","last_fullsized":"2023-10-18T09:18:23.300027+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:16:55.721317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926915+0000","last_change":"2023-10-18T09:16:30.909522+0000","last_active":"2023-10-18T09:18:21.926915+0000","last_peered":"2023-10-18T09:18:21.926915+0000","last_clean":"2023-10-18T09:18:21.926915+0000","last_became_active":"2023-10-18T09:16:30.909123+0000","last_became_peered":"2023-10-18T09:16:30.909123+0000","last_unstale":"2023-10-18T09:18:21.926915+0000","last_undegraded":"2023-10-18T09:18:21.926915+0000","last_fullsized":"2023-10-18T09:18:21.926915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:25:49.524235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935048+0000","last_change":"2023-10-18T09:16:25.866848+0000","last_active":"2023-10-18T09:18:21.935048+0000","last_peered":"2023-10-18T09:18:21.935048+0000","last_clean":"2023-10-18T09:18:21.935048+0000","last_became_active":"2023-10-18T09:16:25.866175+0000","last_became_peered":"2023-10-18T09:16:25.866175+0000","last_unstale":"2023-10-18T09:18:21.935048+0000","last_undegraded":"2023-10-18T09:18:21.935048+0000","last_fullsized":"2023-10-18T09:18:21.935048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:23:02.700379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934483+0000","last_change":"2023-10-18T09:16:26.514222+0000","last_active":"2023-10-18T09:18:21.934483+0000","last_peered":"2023-10-18T09:18:21.934483+0000","last_clean":"2023-10-18T09:18:21.934483+0000","last_became_active":"2023-10-18T09:16:26.513883+0000","last_became_peered":"2023-10-18T09:16:26.513883+0000","last_unstale":"2023-10-18T09:18:21.934483+0000","last_undegraded":"2023-10-18T09:18:21.934483+0000","last_fullsized":"2023-10-18T09:18:21.934483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:27:30.185339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926826+0000","last_change":"2023-10-18T09:16:30.907179+0000","last_active":"2023-10-18T09:18:21.926826+0000","last_peered":"2023-10-18T09:18:21.926826+0000","last_clean":"2023-10-18T09:18:21.926826+0000","last_became_active":"2023-10-18T09:16:30.906711+0000","last_became_peered":"2023-10-18T09:16:30.906711+0000","last_unstale":"2023-10-18T09:18:21.926826+0000","last_undegraded":"2023-10-18T09:18:21.926826+0000","last_fullsized":"2023-10-18T09:18:21.926826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:54:26.937324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934579+0000","last_change":"2023-10-18T09:16:25.859923+0000","last_active":"2023-10-18T09:18:21.934579+0000","last_peered":"2023-10-18T09:18:21.934579+0000","last_clean":"2023-10-18T09:18:21.934579+0000","last_became_active":"2023-10-18T09:16:25.859540+0000","last_became_peered":"2023-10-18T09:16:25.859540+0000","last_unstale":"2023-10-18T09:18:21.934579+0000","last_undegraded":"2023-10-18T09:18:21.934579+0000","last_fullsized":"2023-10-18T09:18:21.934579+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:00:24.604672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930047+0000","last_change":"2023-10-18T09:16:26.517387+0000","last_active":"2023-10-18T09:18:21.930047+0000","last_peered":"2023-10-18T09:18:21.930047+0000","last_clean":"2023-10-18T09:18:21.930047+0000","last_became_active":"2023-10-18T09:16:26.517034+0000","last_became_peered":"2023-10-18T09:16:26.517034+0000","last_unstale":"2023-10-18T09:18:21.930047+0000","last_undegraded":"2023-10-18T09:18:21.930047+0000","last_fullsized":"2023-10-18T09:18:21.930047+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:41:58.105883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927100+0000","last_change":"2023-10-18T09:16:30.898292+0000","last_active":"2023-10-18T09:18:21.927100+0000","last_peered":"2023-10-18T09:18:21.927100+0000","last_clean":"2023-10-18T09:18:21.927100+0000","last_became_active":"2023-10-18T09:16:30.897894+0000","last_became_peered":"2023-10-18T09:16:30.897894+0000","last_unstale":"2023-10-18T09:18:21.927100+0000","last_undegraded":"2023-10-18T09:18:21.927100+0000","last_fullsized":"2023-10-18T09:18:21.927100+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:17:03.775769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684206+0000","last_change":"2023-10-18T09:16:25.865803+0000","last_active":"2023-10-18T09:18:37.684206+0000","last_peered":"2023-10-18T09:18:37.684206+0000","last_clean":"2023-10-18T09:18:37.684206+0000","last_became_active":"2023-10-18T09:16:25.865471+0000","last_became_peered":"2023-10-18T09:16:25.865471+0000","last_unstale":"2023-10-18T09:18:37.684206+0000","last_undegraded":"2023-10-18T09:18:37.684206+0000","last_fullsized":"2023-10-18T09:18:37.684206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:55:54.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1153,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934348+0000","last_change":"2023-10-18T09:16:26.517154+0000","last_active":"2023-10-18T09:18:21.934348+0000","last_peered":"2023-10-18T09:18:21.934348+0000","last_clean":"2023-10-18T09:18:21.934348+0000","last_became_active":"2023-10-18T09:16:26.516850+0000","last_became_peered":"2023-10-18T09:16:26.516850+0000","last_unstale":"2023-10-18T09:18:21.934348+0000","last_undegraded":"2023-10-18T09:18:21.934348+0000","last_fullsized":"2023-10-18T09:18:21.934348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:37:37.737493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926656+0000","last_change":"2023-10-18T09:16:30.909598+0000","last_active":"2023-10-18T09:18:21.926656+0000","last_peered":"2023-10-18T09:18:21.926656+0000","last_clean":"2023-10-18T09:18:21.926656+0000","last_became_active":"2023-10-18T09:16:30.908701+0000","last_became_peered":"2023-10-18T09:16:30.908701+0000","last_unstale":"2023-10-18T09:18:21.926656+0000","last_undegraded":"2023-10-18T09:18:21.926656+0000","last_fullsized":"2023-10-18T09:18:21.926656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:47:02.233935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574771+0000","last_change":"2023-10-18T09:16:25.859589+0000","last_active":"2023-10-18T09:18:22.574771+0000","last_peered":"2023-10-18T09:18:22.574771+0000","last_clean":"2023-10-18T09:18:22.574771+0000","last_became_active":"2023-10-18T09:16:25.859237+0000","last_became_peered":"2023-10-18T09:16:25.859237+0000","last_unstale":"2023-10-18T09:18:22.574771+0000","last_undegraded":"2023-10-18T09:18:22.574771+0000","last_fullsized":"2023-10-18T09:18:22.574771+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:25:30.844868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930860+0000","last_change":"2023-10-18T09:16:26.516882+0000","last_active":"2023-10-18T09:18:21.930860+0000","last_peered":"2023-10-18T09:18:21.930860+0000","last_clean":"2023-10-18T09:18:21.930860+0000","last_became_active":"2023-10-18T09:16:26.516533+0000","last_became_peered":"2023-10-18T09:16:26.516533+0000","last_unstale":"2023-10-18T09:18:21.930860+0000","last_undegraded":"2023-10-18T09:18:21.930860+0000","last_fullsized":"2023-10-18T09:18:21.930860+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:24:27.118000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931037+0000","last_change":"2023-10-18T09:17:21.918712+0000","last_active":"2023-10-18T09:18:22.931037+0000","last_peered":"2023-10-18T09:18:22.931037+0000","last_clean":"2023-10-18T09:18:22.931037+0000","last_became_active":"2023-10-18T09:17:21.918229+0000","last_became_peered":"2023-10-18T09:17:21.918229+0000","last_unstale":"2023-10-18T09:18:22.931037+0000","last_undegraded":"2023-10-18T09:18:22.931037+0000","last_fullsized":"2023-10-18T09:18:22.931037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:07:31.948581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573599+0000","last_change":"2023-10-18T09:16:25.878477+0000","last_active":"2023-10-18T09:18:22.573599+0000","last_peered":"2023-10-18T09:18:22.573599+0000","last_clean":"2023-10-18T09:18:22.573599+0000","last_became_active":"2023-10-18T09:16:25.874427+0000","last_became_peered":"2023-10-18T09:16:25.874427+0000","last_unstale":"2023-10-18T09:18:22.573599+0000","last_undegraded":"2023-10-18T09:18:22.573599+0000","last_fullsized":"2023-10-18T09:18:22.573599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:15:47.245041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926937+0000","last_change":"2023-10-18T09:16:26.512665+0000","last_active":"2023-10-18T09:18:21.926937+0000","last_peered":"2023-10-18T09:18:21.926937+0000","last_clean":"2023-10-18T09:18:21.926937+0000","last_became_active":"2023-10-18T09:16:26.511639+0000","last_became_peered":"2023-10-18T09:16:26.511639+0000","last_unstale":"2023-10-18T09:18:21.926937+0000","last_undegraded":"2023-10-18T09:18:21.926937+0000","last_fullsized":"2023-10-18T09:18:21.926937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:10:23.188858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299862+0000","last_change":"2023-10-18T09:16:30.908013+0000","last_active":"2023-10-18T09:18:23.299862+0000","last_peered":"2023-10-18T09:18:23.299862+0000","last_clean":"2023-10-18T09:18:23.299862+0000","last_became_active":"2023-10-18T09:16:30.907542+0000","last_became_peered":"2023-10-18T09:16:30.907542+0000","last_unstale":"2023-10-18T09:18:23.299862+0000","last_undegraded":"2023-10-18T09:18:23.299862+0000","last_fullsized":"2023-10-18T09:18:23.299862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:57:38.790583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930648+0000","last_change":"2023-10-18T09:16:25.862108+0000","last_active":"2023-10-18T09:18:21.930648+0000","last_peered":"2023-10-18T09:18:21.930648+0000","last_clean":"2023-10-18T09:18:21.930648+0000","last_became_active":"2023-10-18T09:16:25.861706+0000","last_became_peered":"2023-10-18T09:16:25.861706+0000","last_unstale":"2023-10-18T09:18:21.930648+0000","last_undegraded":"2023-10-18T09:18:21.930648+0000","last_fullsized":"2023-10-18T09:18:21.930648+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:33:02.000153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933262+0000","last_change":"2023-10-18T09:16:26.519190+0000","last_active":"2023-10-18T09:18:21.933262+0000","last_peered":"2023-10-18T09:18:21.933262+0000","last_clean":"2023-10-18T09:18:21.933262+0000","last_became_active":"2023-10-18T09:16:26.518901+0000","last_became_peered":"2023-10-18T09:16:26.518901+0000","last_unstale":"2023-10-18T09:18:21.933262+0000","last_undegraded":"2023-10-18T09:18:21.933262+0000","last_fullsized":"2023-10-18T09:18:21.933262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:21.355742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.926979+0000","last_change":"2023-10-18T09:16:30.900194+0000","last_active":"2023-10-18T09:18:21.926979+0000","last_peered":"2023-10-18T09:18:21.926979+0000","last_clean":"2023-10-18T09:18:21.926979+0000","last_became_active":"2023-10-18T09:16:30.899782+0000","last_became_peered":"2023-10-18T09:16:30.899782+0000","last_unstale":"2023-10-18T09:18:21.926979+0000","last_undegraded":"2023-10-18T09:18:21.926979+0000","last_fullsized":"2023-10-18T09:18:21.926979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:13.695816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'5","reported_seq":232,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688656+0000","last_change":"2023-10-18T09:16:25.874624+0000","last_active":"2023-10-18T09:18:40.688656+0000","last_peered":"2023-10-18T09:18:40.688656+0000","last_clean":"2023-10-18T09:18:40.688656+0000","last_became_active":"2023-10-18T09:16:25.874305+0000","last_became_peered":"2023-10-18T09:16:25.874305+0000","last_unstale":"2023-10-18T09:18:40.688656+0000","last_undegraded":"2023-10-18T09:18:40.688656+0000","last_fullsized":"2023-10-18T09:18:40.688656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:21:39.971920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934250+0000","last_change":"2023-10-18T09:16:26.518000+0000","last_active":"2023-10-18T09:18:21.934250+0000","last_peered":"2023-10-18T09:18:21.934250+0000","last_clean":"2023-10-18T09:18:21.934250+0000","last_became_active":"2023-10-18T09:16:26.517488+0000","last_became_peered":"2023-10-18T09:16:26.517488+0000","last_unstale":"2023-10-18T09:18:21.934250+0000","last_undegraded":"2023-10-18T09:18:21.934250+0000","last_fullsized":"2023-10-18T09:18:21.934250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:10:18.960837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932772+0000","last_change":"2023-10-18T09:16:30.911919+0000","last_active":"2023-10-18T09:18:21.932772+0000","last_peered":"2023-10-18T09:18:21.932772+0000","last_clean":"2023-10-18T09:18:21.932772+0000","last_became_active":"2023-10-18T09:16:30.911512+0000","last_became_peered":"2023-10-18T09:16:30.911512+0000","last_unstale":"2023-10-18T09:18:21.932772+0000","last_undegraded":"2023-10-18T09:18:21.932772+0000","last_fullsized":"2023-10-18T09:18:21.932772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:47:59.816973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574023+0000","last_change":"2023-10-18T09:16:25.864826+0000","last_active":"2023-10-18T09:18:22.574023+0000","last_peered":"2023-10-18T09:18:22.574023+0000","last_clean":"2023-10-18T09:18:22.574023+0000","last_became_active":"2023-10-18T09:16:25.864466+0000","last_became_peered":"2023-10-18T09:16:25.864466+0000","last_unstale":"2023-10-18T09:18:22.574023+0000","last_undegraded":"2023-10-18T09:18:22.574023+0000","last_fullsized":"2023-10-18T09:18:22.574023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:21:47.957892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781543+0000","last_change":"2023-10-18T09:16:26.520731+0000","last_active":"2023-10-18T09:18:22.781543+0000","last_peered":"2023-10-18T09:18:22.781543+0000","last_clean":"2023-10-18T09:18:22.781543+0000","last_became_active":"2023-10-18T09:16:26.520269+0000","last_became_peered":"2023-10-18T09:16:26.520269+0000","last_unstale":"2023-10-18T09:18:22.781543+0000","last_undegraded":"2023-10-18T09:18:22.781543+0000","last_fullsized":"2023-10-18T09:18:22.781543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:35.146974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927375+0000","last_change":"2023-10-18T09:16:30.909575+0000","last_active":"2023-10-18T09:18:21.927375+0000","last_peered":"2023-10-18T09:18:21.927375+0000","last_clean":"2023-10-18T09:18:21.927375+0000","last_became_active":"2023-10-18T09:16:30.909146+0000","last_became_peered":"2023-10-18T09:16:30.909146+0000","last_unstale":"2023-10-18T09:18:21.927375+0000","last_undegraded":"2023-10-18T09:18:21.927375+0000","last_fullsized":"2023-10-18T09:18:21.927375+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:10:41.314335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.929940+0000","last_change":"2023-10-18T09:16:25.873649+0000","last_active":"2023-10-18T09:18:21.929940+0000","last_peered":"2023-10-18T09:18:21.929940+0000","last_clean":"2023-10-18T09:18:21.929940+0000","last_became_active":"2023-10-18T09:16:25.873277+0000","last_became_peered":"2023-10-18T09:16:25.873277+0000","last_unstale":"2023-10-18T09:18:21.929940+0000","last_undegraded":"2023-10-18T09:18:21.929940+0000","last_fullsized":"2023-10-18T09:18:21.929940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:32.490408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934572+0000","last_change":"2023-10-18T09:17:21.921019+0000","last_active":"2023-10-18T09:18:21.934572+0000","last_peered":"2023-10-18T09:18:21.934572+0000","last_clean":"2023-10-18T09:18:21.934572+0000","last_became_active":"2023-10-18T09:17:21.920611+0000","last_became_peered":"2023-10-18T09:17:21.920611+0000","last_unstale":"2023-10-18T09:18:21.934572+0000","last_undegraded":"2023-10-18T09:18:21.934572+0000","last_fullsized":"2023-10-18T09:18:21.934572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:00:44.317766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932701+0000","last_change":"2023-10-18T09:16:30.914136+0000","last_active":"2023-10-18T09:18:21.932701+0000","last_peered":"2023-10-18T09:18:21.932701+0000","last_clean":"2023-10-18T09:18:21.932701+0000","last_became_active":"2023-10-18T09:16:30.912147+0000","last_became_peered":"2023-10-18T09:16:30.912147+0000","last_unstale":"2023-10-18T09:18:21.932701+0000","last_undegraded":"2023-10-18T09:18:21.932701+0000","last_fullsized":"2023-10-18T09:18:21.932701+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:44:10.463798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":282,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.688670+0000","last_change":"2023-10-18T09:16:25.864368+0000","last_active":"2023-10-18T09:18:40.688670+0000","last_peered":"2023-10-18T09:18:40.688670+0000","last_clean":"2023-10-18T09:18:40.688670+0000","last_became_active":"2023-10-18T09:16:25.863992+0000","last_became_peered":"2023-10-18T09:16:25.863992+0000","last_unstale":"2023-10-18T09:18:40.688670+0000","last_undegraded":"2023-10-18T09:18:40.688670+0000","last_fullsized":"2023-10-18T09:18:40.688670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:09:29.769383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66385,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":221,"num_read_kb":177,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934847+0000","last_change":"2023-10-18T09:16:26.514112+0000","last_active":"2023-10-18T09:18:21.934847+0000","last_peered":"2023-10-18T09:18:21.934847+0000","last_clean":"2023-10-18T09:18:21.934847+0000","last_became_active":"2023-10-18T09:16:26.513824+0000","last_became_peered":"2023-10-18T09:16:26.513824+0000","last_unstale":"2023-10-18T09:18:21.934847+0000","last_undegraded":"2023-10-18T09:18:21.934847+0000","last_fullsized":"2023-10-18T09:18:21.934847+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:43:05.141719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930547+0000","last_change":"2023-10-18T09:16:30.909338+0000","last_active":"2023-10-18T09:18:21.930547+0000","last_peered":"2023-10-18T09:18:21.930547+0000","last_clean":"2023-10-18T09:18:21.930547+0000","last_became_active":"2023-10-18T09:16:30.908799+0000","last_became_peered":"2023-10-18T09:16:30.908799+0000","last_unstale":"2023-10-18T09:18:21.930547+0000","last_undegraded":"2023-10-18T09:18:21.930547+0000","last_fullsized":"2023-10-18T09:18:21.930547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:41:20.431095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573816+0000","last_change":"2023-10-18T09:17:21.956449+0000","last_active":"2023-10-18T09:18:22.573816+0000","last_peered":"2023-10-18T09:18:22.573816+0000","last_clean":"2023-10-18T09:18:22.573816+0000","last_became_active":"2023-10-18T09:17:21.946784+0000","last_became_peered":"2023-10-18T09:17:21.946784+0000","last_unstale":"2023-10-18T09:18:22.573816+0000","last_undegraded":"2023-10-18T09:18:22.573816+0000","last_fullsized":"2023-10-18T09:18:22.573816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:49:21.415006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573678+0000","last_change":"2023-10-18T09:16:26.515508+0000","last_active":"2023-10-18T09:18:22.573678+0000","last_peered":"2023-10-18T09:18:22.573678+0000","last_clean":"2023-10-18T09:18:22.573678+0000","last_became_active":"2023-10-18T09:16:26.515143+0000","last_became_peered":"2023-10-18T09:16:26.515143+0000","last_unstale":"2023-10-18T09:18:22.573678+0000","last_undegraded":"2023-10-18T09:18:22.573678+0000","last_fullsized":"2023-10-18T09:18:22.573678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:02:37.367689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573914+0000","last_change":"2023-10-18T09:16:30.910409+0000","last_active":"2023-10-18T09:18:22.573914+0000","last_peered":"2023-10-18T09:18:22.573914+0000","last_clean":"2023-10-18T09:18:22.573914+0000","last_became_active":"2023-10-18T09:16:30.909933+0000","last_became_peered":"2023-10-18T09:16:30.909933+0000","last_unstale":"2023-10-18T09:18:22.573914+0000","last_undegraded":"2023-10-18T09:18:22.573914+0000","last_fullsized":"2023-10-18T09:18:22.573914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:59:04.754753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930527+0000","last_change":"2023-10-18T09:16:25.866477+0000","last_active":"2023-10-18T09:18:21.930527+0000","last_peered":"2023-10-18T09:18:21.930527+0000","last_clean":"2023-10-18T09:18:21.930527+0000","last_became_active":"2023-10-18T09:16:25.866148+0000","last_became_peered":"2023-10-18T09:16:25.866148+0000","last_unstale":"2023-10-18T09:18:21.930527+0000","last_undegraded":"2023-10-18T09:18:21.930527+0000","last_fullsized":"2023-10-18T09:18:21.930527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:57:42.959211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574246+0000","last_change":"2023-10-18T09:16:26.516497+0000","last_active":"2023-10-18T09:18:22.574246+0000","last_peered":"2023-10-18T09:18:22.574246+0000","last_clean":"2023-10-18T09:18:22.574246+0000","last_became_active":"2023-10-18T09:16:26.516198+0000","last_became_peered":"2023-10-18T09:16:26.516198+0000","last_unstale":"2023-10-18T09:18:22.574246+0000","last_undegraded":"2023-10-18T09:18:22.574246+0000","last_fullsized":"2023-10-18T09:18:22.574246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:48:44.246195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936027+0000","last_change":"2023-10-18T09:16:30.911358+0000","last_active":"2023-10-18T09:18:21.936027+0000","last_peered":"2023-10-18T09:18:21.936027+0000","last_clean":"2023-10-18T09:18:21.936027+0000","last_became_active":"2023-10-18T09:16:30.908822+0000","last_became_peered":"2023-10-18T09:16:30.908822+0000","last_unstale":"2023-10-18T09:18:21.936027+0000","last_undegraded":"2023-10-18T09:18:21.936027+0000","last_fullsized":"2023-10-18T09:18:21.936027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:40:20.265449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'8","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:37.684689+0000","last_change":"2023-10-18T09:16:25.865787+0000","last_active":"2023-10-18T09:18:37.684689+0000","last_peered":"2023-10-18T09:18:37.684689+0000","last_clean":"2023-10-18T09:18:37.684689+0000","last_became_active":"2023-10-18T09:16:25.865389+0000","last_became_peered":"2023-10-18T09:16:25.865389+0000","last_unstale":"2023-10-18T09:18:37.684689+0000","last_undegraded":"2023-10-18T09:18:37.684689+0000","last_fullsized":"2023-10-18T09:18:37.684689+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:12:39.423301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1435,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574313+0000","last_change":"2023-10-18T09:16:26.515351+0000","last_active":"2023-10-18T09:18:22.574313+0000","last_peered":"2023-10-18T09:18:22.574313+0000","last_clean":"2023-10-18T09:18:22.574313+0000","last_became_active":"2023-10-18T09:16:26.515060+0000","last_became_peered":"2023-10-18T09:16:26.515060+0000","last_unstale":"2023-10-18T09:18:22.574313+0000","last_undegraded":"2023-10-18T09:18:22.574313+0000","last_fullsized":"2023-10-18T09:18:22.574313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:43:38.390551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574038+0000","last_change":"2023-10-18T09:17:21.948883+0000","last_active":"2023-10-18T09:18:22.574038+0000","last_peered":"2023-10-18T09:18:22.574038+0000","last_clean":"2023-10-18T09:18:22.574038+0000","last_became_active":"2023-10-18T09:17:21.948451+0000","last_became_peered":"2023-10-18T09:17:21.948451+0000","last_unstale":"2023-10-18T09:18:22.574038+0000","last_undegraded":"2023-10-18T09:18:22.574038+0000","last_fullsized":"2023-10-18T09:18:22.574038+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:09:25.830789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933791+0000","last_change":"2023-10-18T09:16:25.863404+0000","last_active":"2023-10-18T09:18:21.933791+0000","last_peered":"2023-10-18T09:18:21.933791+0000","last_clean":"2023-10-18T09:18:21.933791+0000","last_became_active":"2023-10-18T09:16:25.863017+0000","last_became_peered":"2023-10-18T09:16:25.863017+0000","last_unstale":"2023-10-18T09:18:21.933791+0000","last_undegraded":"2023-10-18T09:18:21.933791+0000","last_fullsized":"2023-10-18T09:18:21.933791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:15:21.806029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301638+0000","last_change":"2023-10-18T09:18:23.301638+0000","last_active":"2023-10-18T09:18:23.301638+0000","last_peered":"2023-10-18T09:18:23.301638+0000","last_clean":"2023-10-18T09:18:23.301638+0000","last_became_active":"2023-10-18T09:18:23.301186+0000","last_became_peered":"2023-10-18T09:18:23.301186+0000","last_unstale":"2023-10-18T09:18:23.301638+0000","last_undegraded":"2023-10-18T09:18:23.301638+0000","last_fullsized":"2023-10-18T09:18:23.301638+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:29:34.532983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.932148+0000","last_change":"2023-10-18T09:16:30.902110+0000","last_active":"2023-10-18T09:18:21.932148+0000","last_peered":"2023-10-18T09:18:21.932148+0000","last_clean":"2023-10-18T09:18:21.932148+0000","last_became_active":"2023-10-18T09:16:30.901699+0000","last_became_peered":"2023-10-18T09:16:30.901699+0000","last_unstale":"2023-10-18T09:18:21.932148+0000","last_undegraded":"2023-10-18T09:18:21.932148+0000","last_fullsized":"2023-10-18T09:18:21.932148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:22:03.849161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.935119+0000","last_change":"2023-10-18T09:16:25.865452+0000","last_active":"2023-10-18T09:18:21.935119+0000","last_peered":"2023-10-18T09:18:21.935119+0000","last_clean":"2023-10-18T09:18:21.935119+0000","last_became_active":"2023-10-18T09:16:25.865054+0000","last_became_peered":"2023-10-18T09:16:25.865054+0000","last_unstale":"2023-10-18T09:18:21.935119+0000","last_undegraded":"2023-10-18T09:18:21.935119+0000","last_fullsized":"2023-10-18T09:18:21.935119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:52:02.326316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574093+0000","last_change":"2023-10-18T09:17:21.919795+0000","last_active":"2023-10-18T09:18:22.574093+0000","last_peered":"2023-10-18T09:18:22.574093+0000","last_clean":"2023-10-18T09:18:22.574093+0000","last_became_active":"2023-10-18T09:17:21.919419+0000","last_became_peered":"2023-10-18T09:17:21.919419+0000","last_unstale":"2023-10-18T09:18:22.574093+0000","last_undegraded":"2023-10-18T09:18:22.574093+0000","last_fullsized":"2023-10-18T09:18:22.574093+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:37:45.138506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300194+0000","last_change":"2023-10-18T09:16:30.915049+0000","last_active":"2023-10-18T09:18:23.300194+0000","last_peered":"2023-10-18T09:18:23.300194+0000","last_clean":"2023-10-18T09:18:23.300194+0000","last_became_active":"2023-10-18T09:16:30.914561+0000","last_became_peered":"2023-10-18T09:16:30.914561+0000","last_unstale":"2023-10-18T09:18:23.300194+0000","last_undegraded":"2023-10-18T09:18:23.300194+0000","last_fullsized":"2023-10-18T09:18:23.300194+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:29:25.345696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931248+0000","last_change":"2023-10-18T09:16:25.873183+0000","last_active":"2023-10-18T09:18:22.931248+0000","last_peered":"2023-10-18T09:18:22.931248+0000","last_clean":"2023-10-18T09:18:22.931248+0000","last_became_active":"2023-10-18T09:16:25.872794+0000","last_became_peered":"2023-10-18T09:16:25.872794+0000","last_unstale":"2023-10-18T09:18:22.931248+0000","last_undegraded":"2023-10-18T09:18:22.931248+0000","last_fullsized":"2023-10-18T09:18:22.931248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:07.090748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300078+0000","last_change":"2023-10-18T09:16:26.514964+0000","last_active":"2023-10-18T09:18:23.300078+0000","last_peered":"2023-10-18T09:18:23.300078+0000","last_clean":"2023-10-18T09:18:23.300078+0000","last_became_active":"2023-10-18T09:16:26.514672+0000","last_became_peered":"2023-10-18T09:16:26.514672+0000","last_unstale":"2023-10-18T09:18:23.300078+0000","last_undegraded":"2023-10-18T09:18:23.300078+0000","last_fullsized":"2023-10-18T09:18:23.300078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:39:35.011292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300839+0000","last_change":"2023-10-18T09:16:30.908060+0000","last_active":"2023-10-18T09:18:23.300839+0000","last_peered":"2023-10-18T09:18:23.300839+0000","last_clean":"2023-10-18T09:18:23.300839+0000","last_became_active":"2023-10-18T09:16:30.906357+0000","last_became_peered":"2023-10-18T09:16:30.906357+0000","last_unstale":"2023-10-18T09:18:23.300839+0000","last_undegraded":"2023-10-18T09:18:23.300839+0000","last_fullsized":"2023-10-18T09:18:23.300839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:17:12.757700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:40.687858+0000","last_change":"2023-10-18T09:16:25.870969+0000","last_active":"2023-10-18T09:18:40.687858+0000","last_peered":"2023-10-18T09:18:40.687858+0000","last_clean":"2023-10-18T09:18:40.687858+0000","last_became_active":"2023-10-18T09:16:25.870680+0000","last_became_peered":"2023-10-18T09:16:25.870680+0000","last_unstale":"2023-10-18T09:18:40.687858+0000","last_undegraded":"2023-10-18T09:18:40.687858+0000","last_fullsized":"2023-10-18T09:18:40.687858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:01:09.270357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301374+0000","last_change":"2023-10-18T09:17:21.920260+0000","last_active":"2023-10-18T09:18:23.301374+0000","last_peered":"2023-10-18T09:18:23.301374+0000","last_clean":"2023-10-18T09:18:23.301374+0000","last_became_active":"2023-10-18T09:17:21.919905+0000","last_became_peered":"2023-10-18T09:17:21.919905+0000","last_unstale":"2023-10-18T09:18:23.301374+0000","last_undegraded":"2023-10-18T09:18:23.301374+0000","last_fullsized":"2023-10-18T09:18:23.301374+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:55:29.827092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300727+0000","last_change":"2023-10-18T09:16:30.918314+0000","last_active":"2023-10-18T09:18:23.300727+0000","last_peered":"2023-10-18T09:18:23.300727+0000","last_clean":"2023-10-18T09:18:23.300727+0000","last_became_active":"2023-10-18T09:16:30.917933+0000","last_became_peered":"2023-10-18T09:16:30.917933+0000","last_unstale":"2023-10-18T09:18:23.300727+0000","last_undegraded":"2023-10-18T09:18:23.300727+0000","last_fullsized":"2023-10-18T09:18:23.300727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:01:50.184655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931375+0000","last_change":"2023-10-18T09:16:26.517597+0000","last_active":"2023-10-18T09:18:22.931375+0000","last_peered":"2023-10-18T09:18:22.931375+0000","last_clean":"2023-10-18T09:18:22.931375+0000","last_became_active":"2023-10-18T09:16:26.517251+0000","last_became_peered":"2023-10-18T09:16:26.517251+0000","last_unstale":"2023-10-18T09:18:22.931375+0000","last_undegraded":"2023-10-18T09:18:22.931375+0000","last_fullsized":"2023-10-18T09:18:22.931375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:59:48.883405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'7","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.686777+0000","last_change":"2023-10-18T09:16:25.867328+0000","last_active":"2023-10-18T09:18:39.686777+0000","last_peered":"2023-10-18T09:18:39.686777+0000","last_clean":"2023-10-18T09:18:39.686777+0000","last_became_active":"2023-10-18T09:16:25.865210+0000","last_became_peered":"2023-10-18T09:16:25.865210+0000","last_unstale":"2023-10-18T09:18:39.686777+0000","last_undegraded":"2023-10-18T09:18:39.686777+0000","last_fullsized":"2023-10-18T09:18:39.686777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:13:18.567144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301085+0000","last_change":"2023-10-18T09:16:30.917301+0000","last_active":"2023-10-18T09:18:23.301085+0000","last_peered":"2023-10-18T09:18:23.301085+0000","last_clean":"2023-10-18T09:18:23.301085+0000","last_became_active":"2023-10-18T09:16:30.916799+0000","last_became_peered":"2023-10-18T09:16:30.916799+0000","last_unstale":"2023-10-18T09:18:23.301085+0000","last_undegraded":"2023-10-18T09:18:23.301085+0000","last_fullsized":"2023-10-18T09:18:23.301085+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:26:32.414863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282146+0000","last_change":"2023-10-18T09:16:26.501181+0000","last_active":"2023-10-18T09:18:23.282146+0000","last_peered":"2023-10-18T09:18:23.282146+0000","last_clean":"2023-10-18T09:18:23.282146+0000","last_became_active":"2023-10-18T09:16:26.500699+0000","last_became_peered":"2023-10-18T09:16:26.500699+0000","last_unstale":"2023-10-18T09:18:23.282146+0000","last_undegraded":"2023-10-18T09:18:23.282146+0000","last_fullsized":"2023-10-18T09:18:23.282146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:23:34.477848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300391+0000","last_change":"2023-10-18T09:16:25.866824+0000","last_active":"2023-10-18T09:18:23.300391+0000","last_peered":"2023-10-18T09:18:23.300391+0000","last_clean":"2023-10-18T09:18:23.300391+0000","last_became_active":"2023-10-18T09:16:25.866430+0000","last_became_peered":"2023-10-18T09:16:25.866430+0000","last_unstale":"2023-10-18T09:18:23.300391+0000","last_undegraded":"2023-10-18T09:18:23.300391+0000","last_fullsized":"2023-10-18T09:18:23.300391+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:32.721139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.573989+0000","last_change":"2023-10-18T09:15:19.259856+0000","last_active":"2023-10-18T09:18:22.573989+0000","last_peered":"2023-10-18T09:18:22.573989+0000","last_clean":"2023-10-18T09:18:22.573989+0000","last_became_active":"2023-10-18T09:15:19.259442+0000","last_became_peered":"2023-10-18T09:15:19.259442+0000","last_unstale":"2023-10-18T09:18:22.573989+0000","last_undegraded":"2023-10-18T09:18:22.573989+0000","last_fullsized":"2023-10-18T09:18:22.573989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:12:11.450295+0000","last_clean_scrub_stamp":"2023-10-18T09:12:11.450295+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:58:37.049676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930232+0000","last_change":"2023-10-18T09:16:30.915148+0000","last_active":"2023-10-18T09:18:21.930232+0000","last_peered":"2023-10-18T09:18:21.930232+0000","last_clean":"2023-10-18T09:18:21.930232+0000","last_became_active":"2023-10-18T09:16:30.914664+0000","last_became_peered":"2023-10-18T09:16:30.914664+0000","last_unstale":"2023-10-18T09:18:21.930232+0000","last_undegraded":"2023-10-18T09:18:21.930232+0000","last_fullsized":"2023-10-18T09:18:21.930232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:27:16.818132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.299699+0000","last_change":"2023-10-18T09:16:26.522212+0000","last_active":"2023-10-18T09:18:23.299699+0000","last_peered":"2023-10-18T09:18:23.299699+0000","last_clean":"2023-10-18T09:18:23.299699+0000","last_became_active":"2023-10-18T09:16:26.520531+0000","last_became_peered":"2023-10-18T09:16:26.520531+0000","last_unstale":"2023-10-18T09:18:23.299699+0000","last_undegraded":"2023-10-18T09:18:23.299699+0000","last_fullsized":"2023-10-18T09:18:23.299699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:36:52.095806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300183+0000","last_change":"2023-10-18T09:16:25.864725+0000","last_active":"2023-10-18T09:18:23.300183+0000","last_peered":"2023-10-18T09:18:23.300183+0000","last_clean":"2023-10-18T09:18:23.300183+0000","last_became_active":"2023-10-18T09:16:25.864391+0000","last_became_peered":"2023-10-18T09:16:25.864391+0000","last_unstale":"2023-10-18T09:18:23.300183+0000","last_undegraded":"2023-10-18T09:18:23.300183+0000","last_fullsized":"2023-10-18T09:18:23.300183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:47:39.276515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930784+0000","last_change":"2023-10-18T09:16:30.907139+0000","last_active":"2023-10-18T09:18:22.930784+0000","last_peered":"2023-10-18T09:18:22.930784+0000","last_clean":"2023-10-18T09:18:22.930784+0000","last_became_active":"2023-10-18T09:16:30.906701+0000","last_became_peered":"2023-10-18T09:16:30.906701+0000","last_unstale":"2023-10-18T09:18:22.930784+0000","last_undegraded":"2023-10-18T09:18:22.930784+0000","last_fullsized":"2023-10-18T09:18:22.930784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:47:14.713540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300567+0000","last_change":"2023-10-18T09:16:25.873745+0000","last_active":"2023-10-18T09:18:23.300567+0000","last_peered":"2023-10-18T09:18:23.300567+0000","last_clean":"2023-10-18T09:18:23.300567+0000","last_became_active":"2023-10-18T09:16:25.873347+0000","last_became_peered":"2023-10-18T09:16:25.873347+0000","last_unstale":"2023-10-18T09:18:23.300567+0000","last_undegraded":"2023-10-18T09:18:23.300567+0000","last_fullsized":"2023-10-18T09:18:23.300567+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:30:40.081399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300439+0000","last_change":"2023-10-18T09:16:26.516984+0000","last_active":"2023-10-18T09:18:23.300439+0000","last_peered":"2023-10-18T09:18:23.300439+0000","last_clean":"2023-10-18T09:18:23.300439+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:23.300439+0000","last_undegraded":"2023-10-18T09:18:23.300439+0000","last_fullsized":"2023-10-18T09:18:23.300439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:19:35.588970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281950+0000","last_change":"2023-10-18T09:16:30.916420+0000","last_active":"2023-10-18T09:18:23.281950+0000","last_peered":"2023-10-18T09:18:23.281950+0000","last_clean":"2023-10-18T09:18:23.281950+0000","last_became_active":"2023-10-18T09:16:30.915660+0000","last_became_peered":"2023-10-18T09:16:30.915660+0000","last_unstale":"2023-10-18T09:18:23.281950+0000","last_undegraded":"2023-10-18T09:18:23.281950+0000","last_fullsized":"2023-10-18T09:18:23.281950+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:52:39.817535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'11","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:25.214651+0000","last_change":"2023-10-18T09:17:21.962634+0000","last_active":"2023-10-18T09:18:25.214651+0000","last_peered":"2023-10-18T09:18:25.214651+0000","last_clean":"2023-10-18T09:18:25.214651+0000","last_became_active":"2023-10-18T09:17:21.948090+0000","last_became_peered":"2023-10-18T09:17:21.948090+0000","last_unstale":"2023-10-18T09:18:25.214651+0000","last_undegraded":"2023-10-18T09:18:25.214651+0000","last_fullsized":"2023-10-18T09:18:25.214651+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:12:14.390673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282465+0000","last_change":"2023-10-18T09:16:26.508836+0000","last_active":"2023-10-18T09:18:23.282465+0000","last_peered":"2023-10-18T09:18:23.282465+0000","last_clean":"2023-10-18T09:18:23.282465+0000","last_became_active":"2023-10-18T09:16:26.508502+0000","last_became_peered":"2023-10-18T09:16:26.508502+0000","last_unstale":"2023-10-18T09:18:23.282465+0000","last_undegraded":"2023-10-18T09:18:23.282465+0000","last_fullsized":"2023-10-18T09:18:23.282465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:28:24.531580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934023+0000","last_change":"2023-10-18T09:16:30.896218+0000","last_active":"2023-10-18T09:18:22.934023+0000","last_peered":"2023-10-18T09:18:22.934023+0000","last_clean":"2023-10-18T09:18:22.934023+0000","last_became_active":"2023-10-18T09:16:30.895740+0000","last_became_peered":"2023-10-18T09:16:30.895740+0000","last_unstale":"2023-10-18T09:18:22.934023+0000","last_undegraded":"2023-10-18T09:18:22.934023+0000","last_fullsized":"2023-10-18T09:18:22.934023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:30.262532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301074+0000","last_change":"2023-10-18T09:16:25.861250+0000","last_active":"2023-10-18T09:18:23.301074+0000","last_peered":"2023-10-18T09:18:23.301074+0000","last_clean":"2023-10-18T09:18:23.301074+0000","last_became_active":"2023-10-18T09:16:25.860868+0000","last_became_peered":"2023-10-18T09:16:25.860868+0000","last_unstale":"2023-10-18T09:18:23.301074+0000","last_undegraded":"2023-10-18T09:18:23.301074+0000","last_fullsized":"2023-10-18T09:18:23.301074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:21:52.305453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300059+0000","last_change":"2023-10-18T09:16:26.524833+0000","last_active":"2023-10-18T09:18:23.300059+0000","last_peered":"2023-10-18T09:18:23.300059+0000","last_clean":"2023-10-18T09:18:23.300059+0000","last_became_active":"2023-10-18T09:16:26.524259+0000","last_became_peered":"2023-10-18T09:16:26.524259+0000","last_unstale":"2023-10-18T09:18:23.300059+0000","last_undegraded":"2023-10-18T09:18:23.300059+0000","last_fullsized":"2023-10-18T09:18:23.300059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:17:58.867484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927146+0000","last_change":"2023-10-18T09:16:30.915063+0000","last_active":"2023-10-18T09:18:21.927146+0000","last_peered":"2023-10-18T09:18:21.927146+0000","last_clean":"2023-10-18T09:18:21.927146+0000","last_became_active":"2023-10-18T09:16:30.914592+0000","last_became_peered":"2023-10-18T09:16:30.914592+0000","last_unstale":"2023-10-18T09:18:21.927146+0000","last_undegraded":"2023-10-18T09:18:21.927146+0000","last_fullsized":"2023-10-18T09:18:21.927146+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:39:02.665937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.933383+0000","last_change":"2023-10-18T09:16:25.866724+0000","last_active":"2023-10-18T09:18:22.933383+0000","last_peered":"2023-10-18T09:18:22.933383+0000","last_clean":"2023-10-18T09:18:22.933383+0000","last_became_active":"2023-10-18T09:16:25.866360+0000","last_became_peered":"2023-10-18T09:16:25.866360+0000","last_unstale":"2023-10-18T09:18:22.933383+0000","last_undegraded":"2023-10-18T09:18:22.933383+0000","last_fullsized":"2023-10-18T09:18:22.933383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:36:18.392709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300468+0000","last_change":"2023-10-18T09:16:26.510160+0000","last_active":"2023-10-18T09:18:23.300468+0000","last_peered":"2023-10-18T09:18:23.300468+0000","last_clean":"2023-10-18T09:18:23.300468+0000","last_became_active":"2023-10-18T09:16:26.509850+0000","last_became_peered":"2023-10-18T09:16:26.509850+0000","last_unstale":"2023-10-18T09:18:23.300468+0000","last_undegraded":"2023-10-18T09:18:23.300468+0000","last_fullsized":"2023-10-18T09:18:23.300468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:36:15.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.936113+0000","last_change":"2023-10-18T09:17:21.949440+0000","last_active":"2023-10-18T09:18:21.936113+0000","last_peered":"2023-10-18T09:18:21.936113+0000","last_clean":"2023-10-18T09:18:21.936113+0000","last_became_active":"2023-10-18T09:17:21.948562+0000","last_became_peered":"2023-10-18T09:17:21.948562+0000","last_unstale":"2023-10-18T09:18:21.936113+0000","last_undegraded":"2023-10-18T09:18:21.936113+0000","last_fullsized":"2023-10-18T09:18:21.936113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:42:20.976200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.281948+0000","last_change":"2023-10-18T09:16:25.868572+0000","last_active":"2023-10-18T09:18:23.281948+0000","last_peered":"2023-10-18T09:18:23.281948+0000","last_clean":"2023-10-18T09:18:23.281948+0000","last_became_active":"2023-10-18T09:16:25.865494+0000","last_became_peered":"2023-10-18T09:16:25.865494+0000","last_unstale":"2023-10-18T09:18:23.281948+0000","last_undegraded":"2023-10-18T09:18:23.281948+0000","last_fullsized":"2023-10-18T09:18:23.281948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:26:15.968155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930407+0000","last_change":"2023-10-18T09:16:26.512010+0000","last_active":"2023-10-18T09:18:22.930407+0000","last_peered":"2023-10-18T09:18:22.930407+0000","last_clean":"2023-10-18T09:18:22.930407+0000","last_became_active":"2023-10-18T09:16:26.511718+0000","last_became_peered":"2023-10-18T09:16:26.511718+0000","last_unstale":"2023-10-18T09:18:22.930407+0000","last_undegraded":"2023-10-18T09:18:22.930407+0000","last_fullsized":"2023-10-18T09:18:22.930407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:25:02.541784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927797+0000","last_change":"2023-10-18T09:17:21.957405+0000","last_active":"2023-10-18T09:18:21.927797+0000","last_peered":"2023-10-18T09:18:21.927797+0000","last_clean":"2023-10-18T09:18:21.927797+0000","last_became_active":"2023-10-18T09:17:21.917875+0000","last_became_peered":"2023-10-18T09:17:21.917875+0000","last_unstale":"2023-10-18T09:18:21.927797+0000","last_undegraded":"2023-10-18T09:18:21.927797+0000","last_fullsized":"2023-10-18T09:18:21.927797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:55:37.429104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:25.866620+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:25.865845+0000","last_became_peered":"2023-10-18T09:16:25.865845+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:48:02.302006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282264+0000","last_change":"2023-10-18T09:16:26.517200+0000","last_active":"2023-10-18T09:18:23.282264+0000","last_peered":"2023-10-18T09:18:23.282264+0000","last_clean":"2023-10-18T09:18:23.282264+0000","last_became_active":"2023-10-18T09:16:26.516884+0000","last_became_peered":"2023-10-18T09:16:26.516884+0000","last_unstale":"2023-10-18T09:18:23.282264+0000","last_undegraded":"2023-10-18T09:18:23.282264+0000","last_fullsized":"2023-10-18T09:18:23.282264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:46:07.605177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.927027+0000","last_change":"2023-10-18T09:16:30.915233+0000","last_active":"2023-10-18T09:18:21.927027+0000","last_peered":"2023-10-18T09:18:21.927027+0000","last_clean":"2023-10-18T09:18:21.927027+0000","last_became_active":"2023-10-18T09:16:30.914707+0000","last_became_peered":"2023-10-18T09:16:30.914707+0000","last_unstale":"2023-10-18T09:18:21.927027+0000","last_undegraded":"2023-10-18T09:18:21.927027+0000","last_fullsized":"2023-10-18T09:18:21.927027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T12:35:14.389214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934530+0000","last_change":"2023-10-18T09:16:25.864252+0000","last_active":"2023-10-18T09:18:21.934530+0000","last_peered":"2023-10-18T09:18:21.934530+0000","last_clean":"2023-10-18T09:18:21.934530+0000","last_became_active":"2023-10-18T09:16:25.863934+0000","last_became_peered":"2023-10-18T09:16:25.863934+0000","last_unstale":"2023-10-18T09:18:21.934530+0000","last_undegraded":"2023-10-18T09:18:21.934530+0000","last_fullsized":"2023-10-18T09:18:21.934530+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:39:36.357581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300205+0000","last_change":"2023-10-18T09:16:26.511869+0000","last_active":"2023-10-18T09:18:23.300205+0000","last_peered":"2023-10-18T09:18:23.300205+0000","last_clean":"2023-10-18T09:18:23.300205+0000","last_became_active":"2023-10-18T09:16:26.511538+0000","last_became_peered":"2023-10-18T09:16:26.511538+0000","last_unstale":"2023-10-18T09:18:23.300205+0000","last_undegraded":"2023-10-18T09:18:23.300205+0000","last_fullsized":"2023-10-18T09:18:23.300205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:16:16.571154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.933939+0000","last_change":"2023-10-18T09:16:30.906423+0000","last_active":"2023-10-18T09:18:21.933939+0000","last_peered":"2023-10-18T09:18:21.933939+0000","last_clean":"2023-10-18T09:18:21.933939+0000","last_became_active":"2023-10-18T09:16:30.906024+0000","last_became_peered":"2023-10-18T09:16:30.906024+0000","last_unstale":"2023-10-18T09:18:21.933939+0000","last_undegraded":"2023-10-18T09:18:21.933939+0000","last_fullsized":"2023-10-18T09:18:21.933939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T19:11:03.504525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'5","reported_seq":130,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:39.688241+0000","last_change":"2023-10-18T09:16:25.870660+0000","last_active":"2023-10-18T09:18:39.688241+0000","last_peered":"2023-10-18T09:18:39.688241+0000","last_clean":"2023-10-18T09:18:39.688241+0000","last_became_active":"2023-10-18T09:16:25.870303+0000","last_became_peered":"2023-10-18T09:16:25.870303+0000","last_unstale":"2023-10-18T09:18:39.688241+0000","last_undegraded":"2023-10-18T09:18:39.688241+0000","last_fullsized":"2023-10-18T09:18:39.688241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:49:08.050742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.301515+0000","last_change":"2023-10-18T09:16:26.508451+0000","last_active":"2023-10-18T09:18:23.301515+0000","last_peered":"2023-10-18T09:18:23.301515+0000","last_clean":"2023-10-18T09:18:23.301515+0000","last_became_active":"2023-10-18T09:16:26.508131+0000","last_became_peered":"2023-10-18T09:16:26.508131+0000","last_unstale":"2023-10-18T09:18:23.301515+0000","last_undegraded":"2023-10-18T09:18:23.301515+0000","last_fullsized":"2023-10-18T09:18:23.301515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:21:08.680819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.781014+0000","last_change":"2023-10-18T09:16:30.911897+0000","last_active":"2023-10-18T09:18:22.781014+0000","last_peered":"2023-10-18T09:18:22.781014+0000","last_clean":"2023-10-18T09:18:22.781014+0000","last_became_active":"2023-10-18T09:16:30.910030+0000","last_became_peered":"2023-10-18T09:16:30.910030+0000","last_unstale":"2023-10-18T09:18:22.781014+0000","last_undegraded":"2023-10-18T09:18:22.781014+0000","last_fullsized":"2023-10-18T09:18:22.781014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T18:44:14.346626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930145+0000","last_change":"2023-10-18T09:16:25.864076+0000","last_active":"2023-10-18T09:18:21.930145+0000","last_peered":"2023-10-18T09:18:21.930145+0000","last_clean":"2023-10-18T09:18:21.930145+0000","last_became_active":"2023-10-18T09:16:25.863759+0000","last_became_peered":"2023-10-18T09:16:25.863759+0000","last_unstale":"2023-10-18T09:18:21.930145+0000","last_undegraded":"2023-10-18T09:18:21.930145+0000","last_fullsized":"2023-10-18T09:18:21.930145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:28:21.596190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300529+0000","last_change":"2023-10-18T09:16:26.517692+0000","last_active":"2023-10-18T09:18:23.300529+0000","last_peered":"2023-10-18T09:18:23.300529+0000","last_clean":"2023-10-18T09:18:23.300529+0000","last_became_active":"2023-10-18T09:16:26.517356+0000","last_became_peered":"2023-10-18T09:16:26.517356+0000","last_unstale":"2023-10-18T09:18:23.300529+0000","last_undegraded":"2023-10-18T09:18:23.300529+0000","last_fullsized":"2023-10-18T09:18:23.300529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T10:15:56.505239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.930361+0000","last_change":"2023-10-18T09:16:30.901947+0000","last_active":"2023-10-18T09:18:21.930361+0000","last_peered":"2023-10-18T09:18:21.930361+0000","last_clean":"2023-10-18T09:18:21.930361+0000","last_became_active":"2023-10-18T09:16:30.901420+0000","last_became_peered":"2023-10-18T09:16:30.901420+0000","last_unstale":"2023-10-18T09:18:21.930361+0000","last_undegraded":"2023-10-18T09:18:21.930361+0000","last_fullsized":"2023-10-18T09:18:21.930361+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:29.864087+0000","last_clean_scrub_stamp":"2023-10-18T09:16:29.864087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T21:15:20.292185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300587+0000","last_change":"2023-10-18T09:17:21.966321+0000","last_active":"2023-10-18T09:18:23.300587+0000","last_peered":"2023-10-18T09:18:23.300587+0000","last_clean":"2023-10-18T09:18:23.300587+0000","last_became_active":"2023-10-18T09:17:21.949466+0000","last_became_peered":"2023-10-18T09:17:21.949466+0000","last_unstale":"2023-10-18T09:18:23.300587+0000","last_undegraded":"2023-10-18T09:18:23.300587+0000","last_fullsized":"2023-10-18T09:18:23.300587+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:24.831322+0000","last_clean_scrub_stamp":"2023-10-18T09:16:24.831322+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:06:05.856643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300969+0000","last_change":"2023-10-18T09:16:26.510334+0000","last_active":"2023-10-18T09:18:23.300969+0000","last_peered":"2023-10-18T09:18:23.300969+0000","last_clean":"2023-10-18T09:18:23.300969+0000","last_became_active":"2023-10-18T09:16:26.510013+0000","last_became_peered":"2023-10-18T09:16:26.510013+0000","last_unstale":"2023-10-18T09:18:23.300969+0000","last_undegraded":"2023-10-18T09:18:23.300969+0000","last_fullsized":"2023-10-18T09:18:23.300969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T14:33:28.140306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.930499+0000","last_change":"2023-10-18T09:16:26.512732+0000","last_active":"2023-10-18T09:18:22.930499+0000","last_peered":"2023-10-18T09:18:22.930499+0000","last_clean":"2023-10-18T09:18:22.930499+0000","last_became_active":"2023-10-18T09:16:26.512439+0000","last_became_peered":"2023-10-18T09:16:26.512439+0000","last_unstale":"2023-10-18T09:18:22.930499+0000","last_undegraded":"2023-10-18T09:18:22.930499+0000","last_fullsized":"2023-10-18T09:18:22.930499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:11:09.634625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934230+0000","last_change":"2023-10-18T09:16:26.516279+0000","last_active":"2023-10-18T09:18:22.934230+0000","last_peered":"2023-10-18T09:18:22.934230+0000","last_clean":"2023-10-18T09:18:22.934230+0000","last_became_active":"2023-10-18T09:16:26.515990+0000","last_became_peered":"2023-10-18T09:16:26.515990+0000","last_unstale":"2023-10-18T09:18:22.934230+0000","last_undegraded":"2023-10-18T09:18:22.934230+0000","last_fullsized":"2023-10-18T09:18:22.934230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:57:03.010562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934162+0000","last_change":"2023-10-18T09:16:26.524732+0000","last_active":"2023-10-18T09:18:22.934162+0000","last_peered":"2023-10-18T09:18:22.934162+0000","last_clean":"2023-10-18T09:18:22.934162+0000","last_became_active":"2023-10-18T09:16:26.524416+0000","last_became_peered":"2023-10-18T09:16:26.524416+0000","last_unstale":"2023-10-18T09:18:22.934162+0000","last_undegraded":"2023-10-18T09:18:22.934162+0000","last_fullsized":"2023-10-18T09:18:22.934162+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:41:00.251503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300500+0000","last_change":"2023-10-18T09:16:26.526403+0000","last_active":"2023-10-18T09:18:23.300500+0000","last_peered":"2023-10-18T09:18:23.300500+0000","last_clean":"2023-10-18T09:18:23.300500+0000","last_became_active":"2023-10-18T09:16:26.525510+0000","last_became_peered":"2023-10-18T09:16:26.525510+0000","last_unstale":"2023-10-18T09:18:23.300500+0000","last_undegraded":"2023-10-18T09:18:23.300500+0000","last_fullsized":"2023-10-18T09:18:23.300500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T17:58:41.712428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300611+0000","last_change":"2023-10-18T09:16:26.521301+0000","last_active":"2023-10-18T09:18:23.300611+0000","last_peered":"2023-10-18T09:18:23.300611+0000","last_clean":"2023-10-18T09:18:23.300611+0000","last_became_active":"2023-10-18T09:16:26.520930+0000","last_became_peered":"2023-10-18T09:16:26.520930+0000","last_unstale":"2023-10-18T09:18:23.300611+0000","last_undegraded":"2023-10-18T09:18:23.300611+0000","last_fullsized":"2023-10-18T09:18:23.300611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T15:23:13.368794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931115+0000","last_change":"2023-10-18T09:16:26.510769+0000","last_active":"2023-10-18T09:18:22.931115+0000","last_peered":"2023-10-18T09:18:22.931115+0000","last_clean":"2023-10-18T09:18:22.931115+0000","last_became_active":"2023-10-18T09:16:26.510541+0000","last_became_peered":"2023-10-18T09:16:26.510541+0000","last_unstale":"2023-10-18T09:18:22.931115+0000","last_undegraded":"2023-10-18T09:18:22.931115+0000","last_fullsized":"2023-10-18T09:18:22.931115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T11:02:08.823678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.934563+0000","last_change":"2023-10-18T09:16:26.510892+0000","last_active":"2023-10-18T09:18:22.934563+0000","last_peered":"2023-10-18T09:18:22.934563+0000","last_clean":"2023-10-18T09:18:22.934563+0000","last_became_active":"2023-10-18T09:16:26.510563+0000","last_became_peered":"2023-10-18T09:16:26.510563+0000","last_unstale":"2023-10-18T09:18:22.934563+0000","last_undegraded":"2023-10-18T09:18:22.934563+0000","last_fullsized":"2023-10-18T09:18:22.934563+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T16:50:46.798560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282527+0000","last_change":"2023-10-18T09:16:26.518676+0000","last_active":"2023-10-18T09:18:23.282527+0000","last_peered":"2023-10-18T09:18:23.282527+0000","last_clean":"2023-10-18T09:18:23.282527+0000","last_became_active":"2023-10-18T09:16:26.518395+0000","last_became_peered":"2023-10-18T09:16:26.518395+0000","last_unstale":"2023-10-18T09:18:23.282527+0000","last_undegraded":"2023-10-18T09:18:23.282527+0000","last_fullsized":"2023-10-18T09:18:23.282527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:25:49.121282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.931260+0000","last_change":"2023-10-18T09:16:26.514473+0000","last_active":"2023-10-18T09:18:22.931260+0000","last_peered":"2023-10-18T09:18:22.931260+0000","last_clean":"2023-10-18T09:18:22.931260+0000","last_became_active":"2023-10-18T09:16:26.514132+0000","last_became_peered":"2023-10-18T09:16:26.514132+0000","last_unstale":"2023-10-18T09:18:22.931260+0000","last_undegraded":"2023-10-18T09:18:22.931260+0000","last_fullsized":"2023-10-18T09:18:22.931260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:05:32.439587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.282117+0000","last_change":"2023-10-18T09:16:26.510747+0000","last_active":"2023-10-18T09:18:23.282117+0000","last_peered":"2023-10-18T09:18:23.282117+0000","last_clean":"2023-10-18T09:18:23.282117+0000","last_became_active":"2023-10-18T09:16:26.509674+0000","last_became_peered":"2023-10-18T09:16:26.509674+0000","last_unstale":"2023-10-18T09:18:23.282117+0000","last_undegraded":"2023-10-18T09:18:23.282117+0000","last_fullsized":"2023-10-18T09:18:23.282117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T20:34:09.359022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:21.934304+0000","last_change":"2023-10-18T09:16:26.505114+0000","last_active":"2023-10-18T09:18:21.934304+0000","last_peered":"2023-10-18T09:18:21.934304+0000","last_clean":"2023-10-18T09:18:21.934304+0000","last_became_active":"2023-10-18T09:16:26.504432+0000","last_became_peered":"2023-10-18T09:16:26.504432+0000","last_unstale":"2023-10-18T09:18:21.934304+0000","last_undegraded":"2023-10-18T09:18:21.934304+0000","last_fullsized":"2023-10-18T09:18:21.934304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:57:54.711241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-18T09:18:23.300332+0000","last_change":"2023-10-18T09:16:26.517223+0000","last_active":"2023-10-18T09:18:23.300332+0000","last_peered":"2023-10-18T09:18:23.300332+0000","last_clean":"2023-10-18T09:18:23.300332+0000","last_became_active":"2023-10-18T09:16:26.516414+0000","last_became_peered":"2023-10-18T09:16:26.516414+0000","last_unstale":"2023-10-18T09:18:23.300332+0000","last_undegraded":"2023-10-18T09:18:23.300332+0000","last_fullsized":"2023-10-18T09:18:23.300332+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T09:49:40.595259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-18T09:18:22.574002+0000","last_change":"2023-10-18T09:16:26.518225+0000","last_active":"2023-10-18T09:18:22.574002+0000","last_peered":"2023-10-18T09:18:22.574002+0000","last_clean":"2023-10-18T09:18:22.574002+0000","last_became_active":"2023-10-18T09:16:26.517947+0000","last_became_peered":"2023-10-18T09:16:26.517947+0000","last_unstale":"2023-10-18T09:18:22.574002+0000","last_undegraded":"2023-10-18T09:18:22.574002+0000","last_fullsized":"2023-10-18T09:18:22.574002+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-18T09:16:25.837299+0000","last_clean_scrub_stamp":"2023-10-18T09:16:25.837299+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-19T13:22:04.942132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74185,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1072,"num_read_kb":621,"num_write":253,"num_write_kb":155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":68,"num_keys_recovered":17,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":510815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.46500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84199999999999997,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.624},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53300000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.80100000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.60699999999999998},"min":{"1min":0.44,"5min":0.436,"15min":0.436},"max":{"1min":0.83199999999999996,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73899999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.48499999999999999,"15min":0.47899999999999998},"min":{"1min":0.34499999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52600000000000002,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.874},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.79000000000000004}]},{"osd":5,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64400000000000002,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82899999999999996}]},{"osd":6,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46200000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.94999999999999996,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.4059999999999999,"5min":0.89500000000000002,"15min":0.81000000000000005},"min":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":13.055,"5min":13.055,"15min":13.055},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.59299999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.008,"5min":1.407,"15min":1.407},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75700000000000001,"15min":0.745},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.89000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.68899999999999995,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.133,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.052,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81699999999999995,"15min":0.81499999999999995},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.107,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.94499999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645356,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1744896,"data_stored":1168560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58899999999999997,"15min":0.59899999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69799999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.747,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67100000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42899999999999999,"15min":0.436},"min":{"1min":0.29799999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.56100000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.29599999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64500000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93700000000000006,"5min":1.196,"15min":1.196},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77900000000000003,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71999999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41599999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.77100000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.74199999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76700000000000002,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.242,"15min":1.242},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85799999999999998,"5min":1.141,"15min":1.141},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.68100000000000005,"15min":0.68300000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86199999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.50800000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.127,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:16:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.72299999999999998},"min":{"1min":0.48699999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.95099999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91300000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.90000000000000002}]},{"osd":9,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.54700000000000004,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.93100000000000005,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0469999999999999,"5min":1.532,"15min":1.532},"last":0.86899999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.66400000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.88700000000000001}]}]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":510677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1597,"internal_metadata":27654595},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60199999999999998,"15min":0.55800000000000005},"min":{"1min":0.38100000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.1110000000000002,"5min":6.1280000000000001,"15min":6.1280000000000001},"last":0.502},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.0510000000000002,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69199999999999995,"15min":0.67300000000000004},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.0790000000000002,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.81799999999999995,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":3.8730000000000002,"5min":6.2610000000000001,"15min":6.2610000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.85899999999999999,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1600000000000001,"5min":7.1600000000000001,"15min":7.1600000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.82999999999999996,"15min":0.75700000000000001},"min":{"1min":0.53600000000000003,"5min":0.495,"15min":0.495},"max":{"1min":7.1909999999999998,"5min":7.1909999999999998,"15min":7.1909999999999998},"last":0.876}]},{"osd":3,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.754,"15min":0.72599999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":4.766,"5min":4.766,"15min":4.766},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85599999999999998,"15min":0.82699999999999996},"min":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.609,"5min":6.5679999999999996,"15min":6.5679999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.80700000000000005},"min":{"1min":0.46899999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.82399999999999995,"15min":0.81000000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.5469999999999997,"5min":4.5469999999999997,"15min":4.5469999999999997},"last":0.97499999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88900000000000001,"15min":0.86499999999999999},"min":{"1min":0.53900000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.7789999999999999,"5min":4.5190000000000001,"15min":4.5190000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.88100000000000001,"15min":0.85899999999999999},"min":{"1min":0.56399999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":4.6500000000000004,"5min":4.6500000000000004,"15min":4.6500000000000004},"last":1.0629999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.77200000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.92500000000000004,"15min":0.91200000000000003},"min":{"1min":0.62,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":4.8049999999999997,"5min":4.8049999999999997,"15min":4.8049999999999997},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.98499999999999999,"15min":0.96199999999999997},"min":{"1min":0.66400000000000003,"5min":0.63,"15min":0.63},"max":{"1min":3.8140000000000001,"5min":6.3090000000000002,"15min":6.3090000000000002},"last":0.82799999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0029999999999999,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":3.6930000000000001,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.0069999999999999,"15min":0.96999999999999997},"min":{"1min":0.71699999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":3.7599999999999998,"5min":6.4130000000000003,"15min":6.4130000000000003},"last":0.80700000000000005}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1,"15min":0.96499999999999997},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":3.9199999999999999,"5min":6.3529999999999998,"15min":6.3529999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0009999999999999,"15min":0.97799999999999998},"min":{"1min":0.69099999999999995,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":3.7360000000000002,"5min":4.8819999999999997,"15min":4.8819999999999997},"last":0.89600000000000002}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":1.204,"15min":1.2549999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.1060000000000001,"5min":4.8339999999999996,"15min":4.8339999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.089,"15min":1.123},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.123,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":1.1020000000000001}]}]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.371,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.443}]},{"osd":1,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.432},"min":{"1min":0.38200000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.63400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65100000000000002,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58899999999999997,"15min":0.58699999999999997},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.51300000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.626,"15min":0.624},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.871,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64100000000000001,"15min":0.628},"min":{"1min":0.48599999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.72699999999999998}]},{"osd":5,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40899999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57599999999999996,"15min":0.56899999999999995},"min":{"1min":0.35799999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90900000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.71799999999999997,"15min":0.69599999999999995},"min":{"1min":0.60499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.51400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.64800000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:17:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.72099999999999997,"15min":0.71999999999999997},"min":{"1min":0.46300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93100000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70299999999999996,"15min":0.69199999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95299999999999996,"5min":1.254,"15min":1.254},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:06 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.78200000000000003,"15min":0.754},"min":{"1min":0.56000000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.58799999999999997}]},{"osd":10,"last update":"Wed Oct 18 09:18:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.746,"15min":0.71499999999999997},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.58699999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86799999999999999}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80800000000000005,"15min":0.80700000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.94399999999999995,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96599999999999997,"5min":1.05,"15min":1.05},"last":0.82799999999999996}]}]},{"osd":7,"up_from":52,"seq":223338299449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1658880,"data_stored":1108208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:07 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54800000000000004,"15min":0.53700000000000003},"min":{"1min":0.44800000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.753},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.377,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65900000000000003}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.41699999999999998,"15min":0.39900000000000002},"min":{"1min":0.309,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.502,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.44900000000000001,"15min":0.42299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.69899999999999995,"5min":0.876,"15min":0.876},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.57099999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59199999999999997,"15min":0.56399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.85599999999999998}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58999999999999997,"15min":0.55900000000000005},"min":{"1min":0.46200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.745,"5min":0.995,"15min":0.995},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.63600000000000001,"15min":0.621},"min":{"1min":0.46700000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83299999999999996,"5min":1.02,"15min":1.02},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.76700000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.48899999999999999,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53500000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.52000000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.80400000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.754},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.85699999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72099999999999997,"15min":0.69499999999999995},"min":{"1min":0.495,"5min":0.433,"15min":0.433},"max":{"1min":0.80000000000000004,"5min":1.173,"15min":1.173},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.443,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.61899999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71599999999999997,"15min":0.69199999999999995},"min":{"1min":0.51200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.878,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.58399999999999996,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.80700000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83499999999999996}]},{"osd":9,"last update":"Wed Oct 18 09:17:56 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":0.998,"15min":0.998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.65900000000000003,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.018,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.90500000000000003}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.93799999999999994,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.876},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73999999999999999,"15min":0.76000000000000001},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88900000000000001,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.95699999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.78200000000000003},"min":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":0.93600000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63200000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.92900000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.93700000000000006}]}]},{"osd":6,"up_from":47,"seq":201863462975,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.48799999999999999,"15min":0.44400000000000001},"min":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84499999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.36199999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.70099999999999996}]},{"osd":1,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.69399999999999995,"15min":0.629},"min":{"1min":0.44600000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.69199999999999995,"15min":0.63600000000000001},"min":{"1min":0.496,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.70099999999999996,"15min":0.66700000000000004},"min":{"1min":0.58599999999999997,"5min":0.442,"15min":0.442},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.64300000000000002,"15min":0.60199999999999998},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.65300000000000002,"15min":0.61899999999999999},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.75,"15min":0.70299999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4570000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.76600000000000001,"15min":0.72699999999999998},"min":{"1min":0.59299999999999997,"5min":0.498,"15min":0.498},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91100000000000003,"15min":0.80900000000000005},"min":{"1min":0.63100000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7789999999999999,"5min":11.112,"15min":11.112},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.83299999999999996,"15min":0.78300000000000003},"min":{"1min":0.64000000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.83499999999999996,"15min":0.80300000000000005},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.73099999999999998}]},{"osd":8,"last update":"Wed Oct 18 09:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.68899999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5269999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.61899999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2649999999999999,"5min":2.7549999999999999,"15min":2.7549999999999999},"last":0.74299999999999999}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.874,"15min":0.86599999999999999},"min":{"1min":0.53700000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.91000000000000003,"15min":0.89700000000000002},"min":{"1min":0.78300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.752}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.95099999999999996,"15min":0.93500000000000005},"min":{"1min":0.78800000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.85499999999999998,"15min":0.82399999999999995},"min":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.3740000000000001,"5min":2.847,"15min":2.847},"last":0.66600000000000004}]},{"osd":11,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0089999999999999,"15min":1.0209999999999999},"min":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3420000000000001,"5min":2.802,"15min":2.802},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0349999999999999,"15min":1.0369999999999999},"min":{"1min":0.747,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.4670000000000001,"5min":1.581,"15min":1.581},"last":0.85799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":568217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.58799999999999997,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.7130000000000001,"5min":5.2779999999999996,"15min":5.2779999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.51300000000000001},"min":{"1min":0.39200000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.75700000000000001,"5min":0.875,"15min":0.875},"last":0.505}]},{"osd":2,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60399999999999998,"15min":0.53700000000000003},"min":{"1min":0.47299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.435},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58599999999999997,"15min":0.52700000000000002},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46600000000000003}]},{"osd":3,"last update":"Wed Oct 18 09:17:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.58799999999999997,"15min":0.54900000000000004},"min":{"1min":0.45800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.63300000000000001,"15min":0.58499999999999996},"min":{"1min":0.52700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64200000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:17:01 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.42899999999999999,"15min":0.40000000000000002},"min":{"1min":0.34100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.32200000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.628,"15min":0.55800000000000005},"min":{"1min":0.501,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.87,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66800000000000004,"15min":0.59199999999999997},"min":{"1min":0.48899999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.80000000000000004,"5min":5.258,"15min":5.258},"last":0.75}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.69699999999999995,"15min":0.66100000000000003},"min":{"1min":0.51400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":2.6760000000000002,"5min":2.6760000000000002,"15min":2.6760000000000002},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Oct 18 09:18:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85899999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.61699999999999999},"min":{"1min":0.502,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.89400000000000002,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61799999999999999}]},{"osd":8,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70799999999999996,"15min":0.65600000000000003},"min":{"1min":0.57799999999999996,"5min":0.433,"15min":0.433},"max":{"1min":0.92500000000000004,"5min":1.073,"15min":1.073},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.64700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84399999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.78600000000000003}]},{"osd":9,"last update":"Wed Oct 18 09:18:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73699999999999999,"15min":0.71099999999999997},"min":{"1min":0.55400000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.484,"5min":1.605,"15min":1.605},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.79200000000000004,"15min":0.76200000000000001},"min":{"1min":0.66800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.83099999999999996}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76300000000000001,"15min":0.74399999999999999},"min":{"1min":0.51400000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95999999999999996,"5min":1.099,"15min":1.099},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.48499999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93999999999999995,"5min":1,"15min":1},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.85899999999999999,"15min":0.871},"min":{"1min":0.624,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.91000000000000003,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1069056,"data_stored":507637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.68899999999999995,"15min":0.55900000000000005},"min":{"1min":0.46899999999999997,"5min":0.314,"15min":0.314},"max":{"1min":4.4989999999999997,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.65100000000000002,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.72999999999999998,"15min":0.624},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.68600000000000005,"15min":0.61599999999999999},"min":{"1min":0.46800000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.121,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.71699999999999997,"15min":0.72799999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":2.3679999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.60899999999999999,"15min":0.58099999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.6159999999999997,"5min":4.6159999999999997,"15min":4.6159999999999997},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Oct 18 09:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.78800000000000003,"15min":1.0409999999999999},"min":{"1min":0.442,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.353,"5min":11.539,"15min":11.539},"last":0.505},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.82399999999999995,"15min":1.034},"min":{"1min":0.51800000000000002,"5min":0.37,"15min":0.37},"max":{"1min":2.7370000000000001,"5min":11.516999999999999,"15min":11.516999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.77900000000000003,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":4.54,"5min":4.54,"15min":4.54},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.755,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.2130000000000001,"5min":1.448,"15min":1.448},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Oct 18 09:17:55 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.55600000000000005,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.927,"5min":2.927,"15min":2.927},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.079,"5min":0.84399999999999997,"15min":0.77500000000000002},"min":{"1min":0.60199999999999998,"5min":0.437,"15min":0.437},"max":{"1min":4.6479999999999997,"5min":4.6479999999999997,"15min":4.6479999999999997},"last":0.89900000000000002}]},{"osd":7,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":1,"15min":1.02},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":2.3319999999999999,"5min":2.8540000000000001,"15min":2.8540000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.872,"15min":0.877},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2729999999999999,"5min":1.409,"15min":1.409},"last":0.81299999999999994}]},{"osd":8,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.873,"15min":0.874},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.155,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.93400000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":0.94299999999999995,"15min":0.91000000000000003},"min":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":4.6790000000000003,"5min":4.6790000000000003,"15min":4.6790000000000003},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.101,"5min":0.95799999999999996,"15min":0.94599999999999995},"min":{"1min":0.68300000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Wed Oct 18 09:18:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96499999999999997,"15min":0.96099999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6419999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94399999999999995,"15min":0.90500000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.2909999999999999,"5min":2.8889999999999998,"15min":2.8889999999999998},"last":1.006}]},{"osd":11,"last update":"Wed Oct 18 09:17:38 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":0.95999999999999996,"15min":0.91800000000000004},"min":{"1min":0.68100000000000005,"5min":0.625,"15min":0.625},"max":{"1min":4.5830000000000002,"5min":4.5830000000000002,"15min":4.5830000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.075,"5min":0.96599999999999997,"15min":0.94799999999999995},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6919999999999999,"5min":1.698,"15min":1.698},"last":0.95899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.879,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.58299999999999996,"15min":0.60999999999999999},"min":{"1min":0.33200000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.76300000000000001,"5min":3.016,"15min":3.016},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.61699999999999999,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91000000000000003,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60699999999999998,"15min":0.54700000000000004},"min":{"1min":0.439,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88700000000000001,"5min":0.91100000000000003,"15min":0.98399999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.66100000000000003,"15min":0.59499999999999997},"min":{"1min":0.46600000000000003,"5min":0.315,"15min":0.315},"max":{"1min":7.2140000000000004,"5min":7.2140000000000004,"15min":7.2140000000000004},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55700000000000005,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76800000000000002,"5min":1.056,"15min":1.056},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.76200000000000001,"15min":0.77100000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86699999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.41499999999999998},"min":{"1min":0.35999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.46400000000000002},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Oct 18 09:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":0.76600000000000001,"15min":0.66000000000000003},"min":{"1min":0.59099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":7.1749999999999998,"5min":7.1749999999999998,"15min":7.1749999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.59899999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Wed Oct 18 09:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.95899999999999996,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75700000000000001,"15min":0.75600000000000001},"min":{"1min":0.58899999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1000000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0509999999999999,"5min":1.206,"15min":1.206},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.81799999999999995,"15min":0.78600000000000003},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.038,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71499999999999997}]},{"osd":11,"last update":"Wed Oct 18 09:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.90600000000000003,"15min":0.91400000000000003},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.0629999999999999,"5min":1.093,"15min":1.093},"last":0.84699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":505889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.47499999999999998,"15min":0.40600000000000003},"min":{"1min":0.38200000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.466,"5min":1.466,"15min":1.466},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.498,"15min":0.433},"min":{"1min":0.373,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.55300000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":0.73799999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":7.516,"5min":7.516,"15min":7.516},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.65800000000000003,"15min":0.58799999999999997},"min":{"1min":0.46700000000000003,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.77800000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:17:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.68500000000000005,"15min":0.64200000000000002},"min":{"1min":0.61299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.70599999999999996,"15min":0.65300000000000002},"min":{"1min":0.44900000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.68999999999999995,"15min":0.61099999999999999},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":0.76600000000000001,"15min":0.64100000000000001},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":7.5339999999999998,"5min":7.5339999999999998,"15min":7.5339999999999998},"last":0.73699999999999999}]},{"osd":5,"last update":"Wed Oct 18 09:18:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.65900000000000003},"min":{"1min":0.60199999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98699999999999999,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.623},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.68799999999999994},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.27,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.877}]},{"osd":6,"last update":"Wed Oct 18 09:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.67000000000000004,"15min":0.62},"min":{"1min":0.47899999999999998,"5min":0.373,"15min":0.373},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.68100000000000005,"15min":0.629},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":0.66700000000000004}]},{"osd":7,"last update":"Wed Oct 18 09:18:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78000000000000003,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.83699999999999997}]},{"osd":8,"last update":"Wed Oct 18 09:15:30 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.755}]},{"osd":9,"last update":"Wed Oct 18 09:18:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.83899999999999997,"15min":0.82699999999999996},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.89000000000000001,"15min":0.90800000000000003},"min":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.64000000000000001}]},{"osd":10,"last update":"Wed Oct 18 09:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88100000000000001,"15min":0.89000000000000001},"min":{"1min":0.63400000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.282,"5min":0.95099999999999996,"15min":0.89500000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":7.6070000000000002,"5min":7.6070000000000002,"15min":7.6070000000000002},"last":0.70199999999999996}]},{"osd":11,"last update":"Wed Oct 18 09:18:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.2829999999999999,"15min":1.353},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.1220000000000001,"5min":7.6449999999999996,"15min":7.6449999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":1.3220000000000001,"15min":1.393},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.1830000000000001,"5min":7.6769999999999996,"15min":7.6769999999999996},"last":0.80300000000000005}]}]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":511102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.46000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86799999999999999,"5min":1.04,"15min":1.04},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.437,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85099999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.41699999999999998,"15min":0.373},"min":{"1min":0.35499999999999998,"5min":0.186,"15min":0.186},"max":{"1min":0.70399999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.435},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.39600000000000002},"min":{"1min":0.32300000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.60599999999999998,"5min":0.753,"15min":0.753},"last":0.46100000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54600000000000004},"min":{"1min":0.42699999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.625},{"interface":"front","average":{"1min":1.048,"5min":0.68899999999999995,"15min":0.60099999999999998},"min":{"1min":0.52500000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":8.7300000000000004,"5min":8.7300000000000004,"15min":8.7300000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Wed Oct 18 09:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.63300000000000001,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89700000000000002,"5min":1.077,"15min":1.077},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59999999999999998,"15min":0.59799999999999998},"min":{"1min":0.443,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.872,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.629,"15min":0.56899999999999995},"min":{"1min":0.53000000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84299999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.59399999999999997},"min":{"1min":0.623,"5min":0.374,"15min":0.374},"max":{"1min":0.91800000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67100000000000004,"15min":0.64100000000000001},"min":{"1min":0.47899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.85999999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64400000000000002,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.60299999999999998},"min":{"1min":0.44,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0329999999999999,"5min":1.131,"15min":1.131},"last":0.79200000000000004}]},{"osd":8,"last update":"Wed Oct 18 09:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.624,"5min":0.378,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.753},"min":{"1min":0.65700000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97799999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88800000000000001}]},{"osd":9,"last update":"Wed Oct 18 09:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.52800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":0.99299999999999999,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.80300000000000005,"15min":0.81799999999999995},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.0129999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed Oct 18 09:18:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.53000000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.99099999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93799999999999994,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69399999999999995}]},{"osd":11,"last update":"Wed Oct 18 09:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.80400000000000005,"15min":0.79600000000000004},"min":{"1min":0.68300000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1,"5min":1,"15min":1},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.81699999999999995,"15min":0.81100000000000005},"min":{"1min":0.67200000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84599999999999997}]}]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1103274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.60099999999999998,"15min":0.53500000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":6.0570000000000004,"5min":6.0570000000000004,"15min":6.0570000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.80300000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.53500000000000003},"min":{"1min":0.38900000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.746,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58899999999999997,"15min":0.56100000000000005},"min":{"1min":0.38600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78400000000000003,"5min":0.88,"15min":0.88},"last":0.76700000000000002}]},{"osd":2,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.41999999999999998,"15min":0.38},"min":{"1min":0.34100000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59699999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44400000000000001,"15min":0.40100000000000002},"min":{"1min":0.29499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.503}]},{"osd":3,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57499999999999996,"15min":0.54400000000000004},"min":{"1min":0.436,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58599999999999997,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.57199999999999995},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61799999999999999,"15min":0.56899999999999995},"min":{"1min":0.496,"5min":0.375,"15min":0.375},"max":{"1min":0.77900000000000003,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Oct 18 09:17:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.64500000000000002,"15min":0.623},"min":{"1min":0.51900000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63900000000000001,"15min":0.60899999999999999},"min":{"1min":0.46100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Oct 18 09:18:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.85799999999999998,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.48999999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86699999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.82999999999999996}]},{"osd":8,"last update":"Wed Oct 18 09:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.626,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.60799999999999998},"min":{"1min":0.51900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.80700000000000005}]},{"osd":9,"last update":"Wed Oct 18 09:18:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.872,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.89500000000000002}]},{"osd":10,"last update":"Wed Oct 18 09:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.746,"15min":0.72999999999999998},"min":{"1min":0.55900000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.88}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-18T09:18:44.780 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:45.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:44 smithi061 ceph-mon[98573]: from='client.25622 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:45.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.013+0000 7fb2cbdff700 1 -- 172.21.15.61:0/2876839047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c40731f0 msgr2=0x7fb2c4073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.013+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/2876839047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c40731f0 0x7fb2c4073670 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009990 tx=0x7fb2b4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 -- 172.21.15.61:0/2876839047 shutdown_connections 2023-10-18T09:18:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/2876839047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c4108610 0x7fb2c410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/2876839047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 0x7fb2c4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/2876839047 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c40731f0 0x7fb2c4073670 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 -- 172.21.15.61:0/2876839047 >> 172.21.15.61:0/2876839047 conn(0x7fb2c40fbc90 msgr2=0x7fb2c40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:45.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 -- 172.21.15.61:0/2876839047 shutdown_connections 2023-10-18T09:18:45.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.014+0000 7fb2cbdff700 1 -- 172.21.15.61:0/2876839047 wait complete. 2023-10-18T09:18:45.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 Processor -- start 2023-10-18T09:18:45.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 -- start start 2023-10-18T09:18:45.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c40731f0 0x7fb2c4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 0x7fb2c4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c41a0440 con 0x7fb2c4108610 2023-10-18T09:18:45.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c41a0580 con 0x7fb2c40731f0 2023-10-18T09:18:45.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2cbdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c41a0710 con 0x7fb2c40747d0 2023-10-18T09:18:45.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2ca39c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2c939a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 0x7fb2c4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2ca39c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34630/0 (socket says 172.21.15.61:34630) 2023-10-18T09:18:45.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2ca39c700 1 -- 172.21.15.61:0/3125950466 learned_addr learned my addr 172.21.15.61:0/3125950466 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:45.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.015+0000 7fb2c9b9b700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c40731f0 0x7fb2c4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 -- 172.21.15.61:0/3125950466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 msgr2=0x7fb2c4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 0x7fb2c4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 -- 172.21.15.61:0/3125950466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c40731f0 msgr2=0x7fb2c4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c40731f0 0x7fb2c4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 -- 172.21.15.61:0/3125950466 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40095f0 con 0x7fb2c4108610 2023-10-18T09:18:45.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2ca39c700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00ef30 tx=0x7fb2bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc018070 con 0x7fb2c4108610 2023-10-18T09:18:45.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2bc00f040 con 0x7fb2c4108610 2023-10-18T09:18:45.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c41a0990 con 0x7fb2c4108610 2023-10-18T09:18:45.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.016+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc00cd00 con 0x7fb2c4108610 2023-10-18T09:18:45.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.017+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c41a0ea0 con 0x7fb2c4108610 2023-10-18T09:18:45.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.017+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c4107420 con 0x7fb2c4108610 2023-10-18T09:18:45.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.019+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb2bc003a00 con 0x7fb2c4108610 2023-10-18T09:18:45.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.021+0000 7fb2baffd700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 0x7fb2b00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.021+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb2bc0ccb50 con 0x7fb2c4108610 2023-10-18T09:18:45.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.021+0000 7fb2c9b9b700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 0x7fb2b00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.021+0000 7fb2c9b9b700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 0x7fb2b00a39c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005950 tx=0x7fb2c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.025+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb2bc095e90 con 0x7fb2c4108610 2023-10-18T09:18:45.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:44 smithi170 ceph-mon[135150]: from='client.25622 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:45.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:44 smithi110 ceph-mon[134818]: from='client.25622 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:45.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.238+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2c4067530 con 0x7fb2c4108610 2023-10-18T09:18:45.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.240+0000 7fb2baffd700 1 -- 172.21.15.61:0/3125950466 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fb2bc0994b0 con 0x7fb2c4108610 2023-10-18T09:18:45.242 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:45.243 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:45.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 msgr2=0x7fb2b00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 0x7fb2b00a39c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005950 tx=0x7fb2c000b410 comp rx=0 tx=0).stop 2023-10-18T09:18:45.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 msgr2=0x7fb2c419fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00ef30 tx=0x7fb2bc00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 shutdown_connections 2023-10-18T09:18:45.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2b00a1500 0x7fb2b00a39c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb2c4108610 0x7fb2c419fe70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c40731f0 0x7fb2c4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 --2- 172.21.15.61:0/3125950466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2c40747d0 0x7fb2c4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.242+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 >> 172.21.15.61:0/3125950466 conn(0x7fb2c40fbc90 msgr2=0x7fb2c4104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:45.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.243+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 shutdown_connections 2023-10-18T09:18:45.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.243+0000 7fb2cbdff700 1 -- 172.21.15.61:0/3125950466 wait complete. 2023-10-18T09:18:45.264 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:18:45.265 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:18:45.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.471+0000 7f2a2bbff700 1 -- 172.21.15.61:0/3360112791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a241007a0 msgr2=0x7f2a24100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.471+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/3360112791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a241007a0 0x7f2a24100bc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009990 tx=0x7f2a20009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 -- 172.21.15.61:0/3360112791 shutdown_connections 2023-10-18T09:18:45.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/3360112791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a24102330 0x7f2a24108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/3360112791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a241007a0 0x7f2a24100bc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/3360112791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 -- 172.21.15.61:0/3360112791 >> 172.21.15.61:0/3360112791 conn(0x7f2a240fbcf0 msgr2=0x7f2a240fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:45.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 -- 172.21.15.61:0/3360112791 shutdown_connections 2023-10-18T09:18:45.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.472+0000 7f2a2bbff700 1 -- 172.21.15.61:0/3360112791 wait complete. 2023-10-18T09:18:45.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 Processor -- start 2023-10-18T09:18:45.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 -- start start 2023-10-18T09:18:45.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a241007a0 0x7f2a24074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102330 0x7f2a24073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240738c0 con 0x7f2a24101990 2023-10-18T09:18:45.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24073a30 con 0x7f2a241007a0 2023-10-18T09:18:45.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2bbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241a07b0 con 0x7f2a24102330 2023-10-18T09:18:45.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2919a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2919a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34648/0 (socket says 172.21.15.61:34648) 2023-10-18T09:18:45.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2919a700 1 -- 172.21.15.61:0/1596758901 learned_addr learned my addr 172.21.15.61:0/1596758901 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:45.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.473+0000 7f2a2999b700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a241007a0 0x7f2a24074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2a19c700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102330 0x7f2a24073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 -- 172.21.15.61:0/1596758901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102330 msgr2=0x7f2a24073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102330 0x7f2a24073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 -- 172.21.15.61:0/1596758901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a241007a0 msgr2=0x7f2a24074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a241007a0 0x7f2a24074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 -- 172.21.15.61:0/1596758901 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14009520 con 0x7f2a24101990 2023-10-18T09:18:45.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2919a700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400eb30 tx=0x7f2a1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14009880 con 0x7f2a24101990 2023-10-18T09:18:45.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a200095f0 con 0x7f2a24101990 2023-10-18T09:18:45.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a1400ce30 con 0x7f2a24101990 2023-10-18T09:18:45.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.474+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14003cc0 con 0x7f2a24101990 2023-10-18T09:18:45.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.475+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a241a0dc0 con 0x7f2a24101990 2023-10-18T09:18:45.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.476+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a24107bc0 con 0x7f2a24101990 2023-10-18T09:18:45.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.478+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2a1400c970 con 0x7f2a24101990 2023-10-18T09:18:45.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.479+0000 7f2a1affd700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 0x7f2a100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.479+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2a14014070 con 0x7f2a24101990 2023-10-18T09:18:45.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.479+0000 7f2a2999b700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 0x7f2a100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.480+0000 7f2a2999b700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 0x7f2a100a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a20005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.483+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2a14096820 con 0x7f2a24101990 2023-10-18T09:18:45.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.695+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a241075a0 con 0x7f2a24101990 2023-10-18T09:18:45.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.696+0000 7f2a1affd700 1 -- 172.21.15.61:0/1596758901 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a14099e40 con 0x7f2a24101990 2023-10-18T09:18:45.698 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:18:45.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 msgr2=0x7f2a100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 0x7f2a100a3940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a20000c00 tx=0x7f2a20005fb0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 msgr2=0x7f2a24072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400eb30 tx=0x7f2a1400ee40 comp rx=0 tx=0).stop 2023-10-18T09:18:45.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 shutdown_connections 2023-10-18T09:18:45.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2a100a1480 0x7f2a100a3940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102330 0x7f2a24073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a241007a0 0x7f2a24074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 --2- 172.21.15.61:0/1596758901 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2a24101990 0x7f2a24072be0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 >> 172.21.15.61:0/1596758901 conn(0x7f2a240fbcf0 msgr2=0x7f2a24104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:45.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 shutdown_connections 2023-10-18T09:18:45.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.698+0000 7f2a2bbff700 1 -- 172.21.15.61:0/1596758901 wait complete. 2023-10-18T09:18:45.720 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:18:45.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.926+0000 7f4b619ff700 1 -- 172.21.15.61:0/2794009952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 msgr2=0x7f4b5c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.926+0000 7f4b619ff700 1 --2- 172.21.15.61:0/2794009952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c105ff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009990 tx=0x7f4b50009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 -- 172.21.15.61:0/2794009952 shutdown_connections 2023-10-18T09:18:45.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 --2- 172.21.15.61:0/2794009952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c106530 0x7f4b5c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 --2- 172.21.15.61:0/2794009952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c1012d0 0x7f4b5c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 --2- 172.21.15.61:0/2794009952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c105ff0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 -- 172.21.15.61:0/2794009952 >> 172.21.15.61:0/2794009952 conn(0x7f4b5c0fac00 msgr2=0x7f4b5c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:45.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 -- 172.21.15.61:0/2794009952 shutdown_connections 2023-10-18T09:18:45.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.927+0000 7f4b619ff700 1 -- 172.21.15.61:0/2794009952 wait complete. 2023-10-18T09:18:45.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.928+0000 7f4b619ff700 1 Processor -- start 2023-10-18T09:18:45.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.928+0000 7f4b619ff700 1 -- start start 2023-10-18T09:18:45.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c1012d0 0x7f4b5c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 0x7f4b5c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a0420 con 0x7f4b5c103c00 2023-10-18T09:18:45.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a0560 con 0x7f4b5c106530 2023-10-18T09:18:45.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b619ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1a06f0 con 0x7f4b5c1012d0 2023-10-18T09:18:45.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b5a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b5b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 0x7f4b5c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b5affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c1012d0 0x7f4b5c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b5a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34656/0 (socket says 172.21.15.61:34656) 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.929+0000 7f4b5a7fc700 1 -- 172.21.15.61:0/3142750825 learned_addr learned my addr 172.21.15.61:0/3142750825 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 -- 172.21.15.61:0/3142750825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c1012d0 msgr2=0x7f4b5c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c1012d0 0x7f4b5c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 -- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 msgr2=0x7f4b5c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:45.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:45.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 -- 172.21.15.61:0/3142750825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500095f0 con 0x7f4b5c106530 2023-10-18T09:18:45.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5a7fc700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:45.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b5b7fe700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 0x7f4b5c19fe50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400efc0 tx=0x7f4b5400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54018070 con 0x7f4b5c106530 2023-10-18T09:18:45.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5400f040 con 0x7f4b5c106530 2023-10-18T09:18:45.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c1a0970 con 0x7f4b5c106530 2023-10-18T09:18:45.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400cd40 con 0x7f4b5c106530 2023-10-18T09:18:45.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.930+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c1a0e30 con 0x7f4b5c106530 2023-10-18T09:18:45.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.931+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c1080b0 con 0x7f4b5c106530 2023-10-18T09:18:45.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.933+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4b54003a00 con 0x7f4b5c106530 2023-10-18T09:18:45.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.934+0000 7f4b609fd700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 0x7f4b440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:45.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.935+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b540ccc90 con 0x7f4b5c106530 2023-10-18T09:18:45.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.935+0000 7f4b5affd700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 0x7f4b440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:45.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.935+0000 7f4b5affd700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 0x7f4b440a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009c60 tx=0x7f4b4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:45.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:45.939+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4b54095fd0 con 0x7f4b5c106530 2023-10-18T09:18:46.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:45 smithi061 ceph-mon[98573]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:45 smithi061 ceph-mon[98573]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:45 smithi061 ceph-mon[98573]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:18:46.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3125950466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:46.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:45 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1596758901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:45 smithi170 ceph-mon[135150]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:45 smithi170 ceph-mon[135150]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:45 smithi170 ceph-mon[135150]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:18:46.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3125950466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:46.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:45 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1596758901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:46.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.151+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b5c10bfd0 con 0x7f4b5c106530 2023-10-18T09:18:46.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.152+0000 7f4b609fd700 1 -- 172.21.15.61:0/3142750825 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b540995f0 con 0x7f4b5c106530 2023-10-18T09:18:46.154 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:46.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 msgr2=0x7f4b440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 0x7f4b440a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c009c60 tx=0x7f4b4c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:46.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 msgr2=0x7f4b5c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 0x7f4b5c19fe50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400efc0 tx=0x7f4b5400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 shutdown_connections 2023-10-18T09:18:46.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f4b440a1480 0x7f4b440a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b5c106530 0x7f4b5c19fe50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b5c1012d0 0x7f4b5c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 --2- 172.21.15.61:0/3142750825 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4b5c103c00 0x7f4b5c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 >> 172.21.15.61:0/3142750825 conn(0x7f4b5c0fac00 msgr2=0x7f4b5c1009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:46.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 shutdown_connections 2023-10-18T09:18:46.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.156+0000 7f4b619ff700 1 -- 172.21.15.61:0/3142750825 wait complete. 2023-10-18T09:18:46.183 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:18:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:45 smithi110 ceph-mon[134818]: from='client.15648 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:45 smithi110 ceph-mon[134818]: from='client.15654 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-18T09:18:46.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:45 smithi110 ceph-mon[134818]: pgmap v364: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:18:46.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3125950466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:46.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:45 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1596758901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:18:46.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 -- 172.21.15.61:0/1642558413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30101210 msgr2=0x7fdd30103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 --2- 172.21.15.61:0/1642558413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30101210 0x7fdd30103600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd24009990 tx=0x7fdd24009ca0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 -- 172.21.15.61:0/1642558413 shutdown_connections 2023-10-18T09:18:46.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 --2- 172.21.15.61:0/1642558413 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30106470 0x7fdd301088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 --2- 172.21.15.61:0/1642558413 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30101210 0x7fdd30103600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 --2- 172.21.15.61:0/1642558413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 -- 172.21.15.61:0/1642558413 >> 172.21.15.61:0/1642558413 conn(0x7fdd300fac00 msgr2=0x7fdd300fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:46.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 -- 172.21.15.61:0/1642558413 shutdown_connections 2023-10-18T09:18:46.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.397+0000 7fdd36fff700 1 -- 172.21.15.61:0/1642558413 wait complete. 2023-10-18T09:18:46.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.398+0000 7fdd36fff700 1 Processor -- start 2023-10-18T09:18:46.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.398+0000 7fdd36fff700 1 -- start start 2023-10-18T09:18:46.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.398+0000 7fdd36fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 0x7fdd30198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.398+0000 7fdd36fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd36fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 0x7fdd3019feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd36fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a0480 con 0x7fdd30101210 2023-10-18T09:18:46.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd36fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a05c0 con 0x7fdd30106470 2023-10-18T09:18:46.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd36fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd301a0750 con 0x7fdd30103b40 2023-10-18T09:18:46.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd34d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 0x7fdd30198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd3559c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 0x7fdd3019feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43474/0 (socket says 172.21.15.61:43474) 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 -- 172.21.15.61:0/3834382477 learned_addr learned my addr 172.21.15.61:0/3834382477 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 -- 172.21.15.61:0/3834382477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 msgr2=0x7fdd3019feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 0x7fdd3019feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 -- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 msgr2=0x7fdd30198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 0x7fdd30198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd2ffff700 1 -- 172.21.15.61:0/3834382477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd240095f0 con 0x7fdd30103b40 2023-10-18T09:18:46.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd3559c700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 0x7fdd3019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:46.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.399+0000 7fdd34d9b700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 0x7fdd30198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd2ffff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00d950 tx=0x7fdd1c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00f890 con 0x7fdd30103b40 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd301a0a30 con 0x7fdd30103b40 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd1c010460 con 0x7fdd30103b40 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00e610 con 0x7fdd30103b40 2023-10-18T09:18:46.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.400+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd301a0f90 con 0x7fdd30103b40 2023-10-18T09:18:46.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.401+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd30108070 con 0x7fdd30103b40 2023-10-18T09:18:46.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.403+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdd1c0108d0 con 0x7fdd30103b40 2023-10-18T09:18:46.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.405+0000 7fdd2dffb700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 0x7fdd200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.405+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdd1c0ccda0 con 0x7fdd30103b40 2023-10-18T09:18:46.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.405+0000 7fdd34d9b700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 0x7fdd200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.406+0000 7fdd34d9b700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 0x7fdd200a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400b2e0 tx=0x7fdd24005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:46.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.406+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd1c096050 con 0x7fdd30103b40 2023-10-18T09:18:46.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.626+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd3010bfd0 con 0x7fdd30103b40 2023-10-18T09:18:46.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.627+0000 7fdd2dffb700 1 -- 172.21.15.61:0/3834382477 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd1c0200a0 con 0x7fdd30103b40 2023-10-18T09:18:46.629 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:18:46.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 msgr2=0x7fdd200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 0x7fdd200a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400b2e0 tx=0x7fdd24005fd0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 msgr2=0x7fdd30198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c00d950 tx=0x7fdd1c00dc60 comp rx=0 tx=0).stop 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 shutdown_connections 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdd200a1500 0x7fdd200a39c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd30106470 0x7fdd3019feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdd30101210 0x7fdd30198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 --2- 172.21.15.61:0/3834382477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd30103b40 0x7fdd30198e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.629+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 >> 172.21.15.61:0/3834382477 conn(0x7fdd300fac00 msgr2=0x7fdd30100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:46.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.630+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 shutdown_connections 2023-10-18T09:18:46.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.630+0000 7fdd36fff700 1 -- 172.21.15.61:0/3834382477 wait complete. 2023-10-18T09:18:46.652 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:18:46.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.869+0000 7f66047ff700 1 -- 172.21.15.61:0/1627195574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 msgr2=0x7f65fc108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.869+0000 7f66047ff700 1 --2- 172.21.15.61:0/1627195574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc108be0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f65f400b0a0 tx=0x7f65f400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 -- 172.21.15.61:0/1627195574 shutdown_connections 2023-10-18T09:18:46.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 --2- 172.21.15.61:0/1627195574 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc108be0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 --2- 172.21.15.61:0/1627195574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc1007a0 0x7f65fc100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 --2- 172.21.15.61:0/1627195574 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc101990 0x7f65fc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 -- 172.21.15.61:0/1627195574 >> 172.21.15.61:0/1627195574 conn(0x7f65fc0fbcf0 msgr2=0x7f65fc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:46.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 -- 172.21.15.61:0/1627195574 shutdown_connections 2023-10-18T09:18:46.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.870+0000 7f66047ff700 1 -- 172.21.15.61:0/1627195574 wait complete. 2023-10-18T09:18:46.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.871+0000 7f66047ff700 1 Processor -- start 2023-10-18T09:18:46.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.871+0000 7f66047ff700 1 -- start start 2023-10-18T09:18:46.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 0x7f65fc198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1a0550 con 0x7f65fc1007a0 2023-10-18T09:18:46.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1a0690 con 0x7f65fc101990 2023-10-18T09:18:46.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f66047ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc1a0820 con 0x7f65fc102330 2023-10-18T09:18:46.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f660259b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f660259b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34706/0 (socket says 172.21.15.61:34706) 2023-10-18T09:18:46.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f660259b700 1 -- 172.21.15.61:0/1358331082 learned_addr learned my addr 172.21.15.61:0/1358331082 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:46.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f6602d9c700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.872+0000 7f6601d9a700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 0x7f65fc198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 -- 172.21.15.61:0/1358331082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 msgr2=0x7f65fc19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 -- 172.21.15.61:0/1358331082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 msgr2=0x7f65fc198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:46.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 0x7f65fc198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 -- 172.21.15.61:0/1358331082 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f8009520 con 0x7f65fc1007a0 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f6601d9a700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 0x7f65fc198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f660259b700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ea50 tx=0x7f65f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800cbd0 con 0x7f65fc1007a0 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65f800cd30 con 0x7f65fc1007a0 2023-10-18T09:18:46.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f4009d00 con 0x7f65fc1007a0 2023-10-18T09:18:46.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f80186c0 con 0x7f65fc1007a0 2023-10-18T09:18:46.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.873+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc1a0e80 con 0x7f65fc1007a0 2023-10-18T09:18:46.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.875+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc0675a0 con 0x7f65fc1007a0 2023-10-18T09:18:46.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.878+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f65f8018820 con 0x7f65fc1007a0 2023-10-18T09:18:46.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.880+0000 7f65f37fe700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 0x7f65e80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:46.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.880+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f65f8014070 con 0x7f65fc1007a0 2023-10-18T09:18:46.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.880+0000 7f6601d9a700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 0x7f65e80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:46.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.881+0000 7f6601d9a700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 0x7f65e80a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009cb0 tx=0x7f65ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:46.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:46.884+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65f80968b0 con 0x7f65fc1007a0 2023-10-18T09:18:47.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:46 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:18:47.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3142750825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:46 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3834382477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.110+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65fc1a1170 con 0x7f65fc1007a0 2023-10-18T09:18:47.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.110+0000 7f65f37fe700 1 -- 172.21.15.61:0/1358331082 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f65f8099ed0 con 0x7f65fc1007a0 2023-10-18T09:18:47.113 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:18:47.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 msgr2=0x7f65e80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 0x7f65e80a3940 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009cb0 tx=0x7f65ec009260 comp rx=0 tx=0).stop 2023-10-18T09:18:47.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 msgr2=0x7f65fc198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65f800ea50 tx=0x7f65f800ed60 comp rx=0 tx=0).stop 2023-10-18T09:18:47.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 shutdown_connections 2023-10-18T09:18:47.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f65e80a1480 0x7f65e80a3940 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65fc102330 0x7f65fc19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f65fc1007a0 0x7f65fc198a70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 --2- 172.21.15.61:0/1358331082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65fc101990 0x7f65fc198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 >> 172.21.15.61:0/1358331082 conn(0x7f65fc0fbcf0 msgr2=0x7f65fc072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:47.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 shutdown_connections 2023-10-18T09:18:47.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.113+0000 7f66047ff700 1 -- 172.21.15.61:0/1358331082 wait complete. 2023-10-18T09:18:47.134 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:18:47.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:46 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:18:47.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:46 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3142750825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:46 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3834382477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:46 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:18:47.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:46 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3142750825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:46 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3834382477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:18:47.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 -- 172.21.15.61:0/1833099893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 msgr2=0x7f8654105f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 --2- 172.21.15.61:0/1833099893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654105f10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f86480098f0 tx=0x7f8648009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:47.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 -- 172.21.15.61:0/1833099893 shutdown_connections 2023-10-18T09:18:47.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 --2- 172.21.15.61:0/1833099893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f8654108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 --2- 172.21.15.61:0/1833099893 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 --2- 172.21.15.61:0/1833099893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654105f10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 -- 172.21.15.61:0/1833099893 >> 172.21.15.61:0/1833099893 conn(0x7f86540faba0 msgr2=0x7f86540fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:47.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.344+0000 7f86595ff700 1 -- 172.21.15.61:0/1833099893 shutdown_connections 2023-10-18T09:18:47.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.345+0000 7f86595ff700 1 -- 172.21.15.61:0/1833099893 wait complete. 2023-10-18T09:18:47.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.345+0000 7f86595ff700 1 Processor -- start 2023-10-18T09:18:47.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 -- start start 2023-10-18T09:18:47.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f865419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a03f0 con 0x7f86541011f0 2023-10-18T09:18:47.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a0530 con 0x7f8654106450 2023-10-18T09:18:47.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86595ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86541a06c0 con 0x7f8654103b20 2023-10-18T09:18:47.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f8652ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86527fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f8652ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34714/0 (socket says 172.21.15.61:34714) 2023-10-18T09:18:47.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f8652ffd700 1 -- 172.21.15.61:0/1476333650 learned_addr learned my addr 172.21.15.61:0/1476333650 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:47.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.346+0000 7f86537fe700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f865419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 -- 172.21.15.61:0/1476333650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 msgr2=0x7f865419fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f865419fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 -- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 msgr2=0x7f86541988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 -- 172.21.15.61:0/1476333650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440095f0 con 0x7f8654103b20 2023-10-18T09:18:47.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86537fe700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f865419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:47.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f8652ffd700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:47.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86527fc700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654198e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8648009230 tx=0x7f864800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:47.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864801c070 con 0x7f8654103b20 2023-10-18T09:18:47.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f864800f7d0 con 0x7f8654103b20 2023-10-18T09:18:47.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648003990 con 0x7f8654103b20 2023-10-18T09:18:47.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8648009520 con 0x7f8654103b20 2023-10-18T09:18:47.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.347+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86541a0b80 con 0x7f8654103b20 2023-10-18T09:18:47.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.348+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654107ff0 con 0x7f8654103b20 2023-10-18T09:18:47.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.350+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8648020030 con 0x7f8654103b20 2023-10-18T09:18:47.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.352+0000 7f863bfff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 0x7f863c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.352+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f86480cdee0 con 0x7f8654103b20 2023-10-18T09:18:47.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.352+0000 7f8652ffd700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 0x7f863c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.353+0000 7f8652ffd700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 0x7f863c0a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8644005d10 tx=0x7f8644009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:47.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.354+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8648097220 con 0x7f8654103b20 2023-10-18T09:18:47.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.639+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8654067530 con 0x7f8654103b20 2023-10-18T09:18:47.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.640+0000 7f863bfff700 1 -- 172.21.15.61:0/1476333650 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f864809a840 con 0x7f8654103b20 2023-10-18T09:18:47.642 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:47.643 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:16:59.542245+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:18:47.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.642+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 msgr2=0x7f863c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.642+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 0x7f863c0a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8644005d10 tx=0x7f8644009310 comp rx=0 tx=0).stop 2023-10-18T09:18:47.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 msgr2=0x7f8654198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654198e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8648009230 tx=0x7f864800b790 comp rx=0 tx=0).stop 2023-10-18T09:18:47.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 shutdown_connections 2023-10-18T09:18:47.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f863c0a1500 0x7f863c0a39c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8654106450 0x7f865419fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f86541011f0 0x7f86541988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 --2- 172.21.15.61:0/1476333650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8654103b20 0x7f8654198e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 >> 172.21.15.61:0/1476333650 conn(0x7f86540faba0 msgr2=0x7f8654100840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:47.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 shutdown_connections 2023-10-18T09:18:47.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.643+0000 7f86595ff700 1 -- 172.21.15.61:0/1476333650 wait complete. 2023-10-18T09:18:47.648 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 28 2023-10-18T09:18:47.666 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:18:47.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/489685582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440747d0 msgr2=0x7f1f44072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/489685582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440747d0 0x7f1f44072c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f400098f0 tx=0x7f1f40009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:47.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/489685582 shutdown_connections 2023-10-18T09:18:47.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/489685582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f44108610 0x7f1f4410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/489685582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440747d0 0x7f1f44072c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/489685582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f440731f0 0x7f1f44073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/489685582 >> 172.21.15.61:0/489685582 conn(0x7f1f440fbc90 msgr2=0x7f1f440fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:47.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/489685582 shutdown_connections 2023-10-18T09:18:47.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.881+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/489685582 wait complete. 2023-10-18T09:18:47.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.882+0000 7f1f4b9ff700 1 Processor -- start 2023-10-18T09:18:47.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.882+0000 7f1f4b9ff700 1 -- start start 2023-10-18T09:18:47.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f440747d0 0x7f1f44198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f44108610 0x7f1f4419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a0400 con 0x7f1f440747d0 2023-10-18T09:18:47.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a0540 con 0x7f1f440731f0 2023-10-18T09:18:47.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4b9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a06d0 con 0x7f1f44108610 2023-10-18T09:18:47.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50534/0 (socket says 172.21.15.61:50534) 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 -- 172.21.15.61:0/918565589 learned_addr learned my addr 172.21.15.61:0/918565589 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f48f9a700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f440747d0 0x7f1f44198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f49f9c700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f44108610 0x7f1f4419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 -- 172.21.15.61:0/918565589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f44108610 msgr2=0x7f1f4419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f44108610 0x7f1f4419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 -- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f440747d0 msgr2=0x7f1f44198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:47.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f440747d0 0x7f1f44198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:47.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 -- 172.21.15.61:0/918565589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40009520 con 0x7f1f440731f0 2023-10-18T09:18:47.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.883+0000 7f1f4979b700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009810 tx=0x7f1f4000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:47.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.884+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4001c070 con 0x7f1f440731f0 2023-10-18T09:18:47.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.884+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f441a0950 con 0x7f1f440731f0 2023-10-18T09:18:47.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.884+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f4000f7d0 con 0x7f1f440731f0 2023-10-18T09:18:47.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.884+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40003970 con 0x7f1f440731f0 2023-10-18T09:18:47.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.884+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f441a0dd0 con 0x7f1f440731f0 2023-10-18T09:18:47.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.885+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44107420 con 0x7f1f440731f0 2023-10-18T09:18:47.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.887+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1f40020030 con 0x7f1f440731f0 2023-10-18T09:18:47.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.889+0000 7f1f3a7fc700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 0x7f1f300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:47.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.889+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1f400cdf50 con 0x7f1f440731f0 2023-10-18T09:18:47.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.889+0000 7f1f48f9a700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 0x7f1f300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:47.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.890+0000 7f1f48f9a700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 0x7f1f300a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005fd0 tx=0x7f1f34009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:47.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:47.890+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f40097290 con 0x7f1f440731f0 2023-10-18T09:18:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:47 smithi061 ceph-mon[98573]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-18T09:18:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:47 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1358331082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:48.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:47 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1476333650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:48.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.122+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f4410e170 con 0x7f1f440731f0 2023-10-18T09:18:48.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.124+0000 7f1f3a7fc700 1 -- 172.21.15.61:0/918565589 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f1f40079e60 con 0x7f1f440731f0 2023-10-18T09:18:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:48.127 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:18:48.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 msgr2=0x7f1f300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 0x7f1f300a3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f34005fd0 tx=0x7f1f34009310 comp rx=0 tx=0).stop 2023-10-18T09:18:48.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 msgr2=0x7f1f441988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009810 tx=0x7f1f4000b790 comp rx=0 tx=0).stop 2023-10-18T09:18:48.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 shutdown_connections 2023-10-18T09:18:48.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1f300a1480 0x7f1f300a3940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f44108610 0x7f1f4419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f440731f0 0x7f1f441988d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 --2- 172.21.15.61:0/918565589 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1f440747d0 0x7f1f44198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 >> 172.21.15.61:0/918565589 conn(0x7f1f440fbc90 msgr2=0x7f1f44104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 shutdown_connections 2023-10-18T09:18:48.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.126+0000 7f1f4b9ff700 1 -- 172.21.15.61:0/918565589 wait complete. 2023-10-18T09:18:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:47 smithi170 ceph-mon[135150]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-18T09:18:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:47 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1358331082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:47 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1476333650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:48.149 DEBUG:tasks.fwd_scrub:Yielding 2023-10-18T09:18:48.149 INFO:teuthology.run_tasks:Running task exec... 2023-10-18T09:18:48.184 INFO:teuthology.task.exec:Executing custom commands... 2023-10-18T09:18:48.184 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:18:48.185 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-10-18T09:18:48.185 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-18T09:18:48.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T09:18:48.186 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T09:18:48.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:47 smithi110 ceph-mon[134818]: pgmap v365: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 938 B/s wr, 12 op/s 2023-10-18T09:18:48.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:47 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1358331082' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:18:48.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:47 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1476333650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:18:48.288 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:18:48.343 INFO:teuthology.orchestra.run.smithi061.stdout:Filesystem Size Used Avail Use% Mounted on 2023-10-18T09:18:48.343 INFO:teuthology.orchestra.run.smithi061.stdout:0@c578e0ac-6d95-11ee-8db6-212e2dc638e7.cephfs=/volumes/_nogroup/sv_1/916b255b-5c6f-497a-bbc4-d85b7266a4d4 1.1T 332M 1.1T 1% /tmp/cephfs 2023-10-18T09:18:48.348 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-10-18T09:18:48.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 -- 172.21.15.61:0/3430687680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91880ff690 msgr2=0x7f91880ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3430687680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91880ff690 0x7f91880ffab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f91780099c0 tx=0x7f9178009cd0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 -- 172.21.15.61:0/3430687680 shutdown_connections 2023-10-18T09:18:48.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3430687680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188101220 0x7f9188107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3430687680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f91880ff690 0x7f91880ffab0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3430687680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9188100880 0x7f9188100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 -- 172.21.15.61:0/3430687680 >> 172.21.15.61:0/3430687680 conn(0x7f91880fac00 msgr2=0x7f91880fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 -- 172.21.15.61:0/3430687680 shutdown_connections 2023-10-18T09:18:48.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.422+0000 7f918e1ff700 1 -- 172.21.15.61:0/3430687680 wait complete. 2023-10-18T09:18:48.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 Processor -- start 2023-10-18T09:18:48.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 -- start start 2023-10-18T09:18:48.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188100880 0x7f9188196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9188101220 0x7f918819dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e270 con 0x7f9188101220 2023-10-18T09:18:48.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e3b0 con 0x7f91880ff690 2023-10-18T09:18:48.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.423+0000 7f918e1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918819e540 con 0x7f9188100880 2023-10-18T09:18:48.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50550/0 (socket says 172.21.15.61:50550) 2023-10-18T09:18:48.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 -- 172.21.15.61:0/3663726229 learned_addr learned my addr 172.21.15.61:0/3663726229 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f9187fff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9188101220 0x7f918819dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f9186ffd700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188100880 0x7f9188196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 -- 172.21.15.61:0/3663726229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188100880 msgr2=0x7f9188196c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188100880 0x7f9188196c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 -- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9188101220 msgr2=0x7f918819dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9188101220 0x7f918819dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.424+0000 7f91877fe700 1 -- 172.21.15.61:0/3663726229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009520 con 0x7f91880ff690 2023-10-18T09:18:48.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f91877fe700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9178003730 tx=0x7f917800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f9184ff9700 1 -- 172.21.15.61:0/3663726229 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917801d070 con 0x7f91880ff690 2023-10-18T09:18:48.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f9184ff9700 1 -- 172.21.15.61:0/3663726229 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9178005050 con 0x7f91880ff690 2023-10-18T09:18:48.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f9184ff9700 1 -- 172.21.15.61:0/3663726229 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178004a30 con 0x7f91880ff690 2023-10-18T09:18:48.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9180009c40 con 0x7f91880ff690 2023-10-18T09:18:48.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.425+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91780095f0 con 0x7f91880ff690 2023-10-18T09:18:48.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.428+0000 7f9184ff9700 1 -- 172.21.15.61:0/3663726229 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f91780051c0 con 0x7f91880ff690 2023-10-18T09:18:48.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.431+0000 7f9184ff9700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 0x7f91700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.431+0000 7f9184ff9700 1 -- 172.21.15.61:0/3663726229 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f91780cdd40 con 0x7f91880ff690 2023-10-18T09:18:48.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.431+0000 7f9186ffd700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 0x7f91700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f9186ffd700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 0x7f91700a39c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f917c009df0 tx=0x7f917c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 Processor -- start 2023-10-18T09:18:48.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 -- start start 2023-10-18T09:18:48.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 0x555de0917520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de0917ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0886760 con 0x555de09190c0 2023-10-18T09:18:48.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de086cce0 con 0x555de0917a60 2023-10-18T09:18:48.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.432+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de086a600 con 0x555de0918760 2023-10-18T09:18:48.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43560/0 (socket says 172.21.15.61:43560) 2023-10-18T09:18:48.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 -- 172.21.15.61:0/99815578 learned_addr learned my addr 172.21.15.61:0/99815578 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f9186ffd700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 0x555de0917520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f9187fff700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de0917ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 -- 172.21.15.61:0/99815578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 msgr2=0x555de0917ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de0917ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 -- 172.21.15.61:0/99815578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 msgr2=0x555de0917520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 0x555de0917520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 -- 172.21.15.61:0/99815578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555de0823470 con 0x555de0918760 2023-10-18T09:18:48.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f9186ffd700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 0x555de0917520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:48.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f91877fe700 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91780a2b10 tx=0x7f917809b440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b87583b7747baa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f916dffb700 1 -- 172.21.15.61:0/99815578 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780a3040 con 0x555de0918760 2023-10-18T09:18:48.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f916dffb700 1 -- 172.21.15.61:0/99815578 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91780990e0 con 0x555de0918760 2023-10-18T09:18:48.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.433+0000 7f916dffb700 1 -- 172.21.15.61:0/99815578 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917809a230 con 0x555de0918760 2023-10-18T09:18:48.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.434+0000 7f91a1c04b80 1 -- 172.21.15.61:0/99815578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 msgr2=0x555de0918b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.434+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f91780a2b10 tx=0x7f917809b440 comp rx=0 tx=0).stop 2023-10-18T09:18:48.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 -- 172.21.15.61:0/99815578 shutdown_connections 2023-10-18T09:18:48.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de0917ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0918b80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/99815578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de09190c0 0x555de0917520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 -- 172.21.15.61:0/99815578 >> 172.21.15.61:0/99815578 conn(0x555de08877a0 msgr2=0x555de090dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 -- 172.21.15.61:0/99815578 shutdown_connections 2023-10-18T09:18:48.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 -- 172.21.15.61:0/99815578 wait complete. 2023-10-18T09:18:48.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.435+0000 7f91a1c04b80 1 -- start start 2023-10-18T09:18:48.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.436+0000 7f91a1c04b80 1 Processor -- start 2023-10-18T09:18:48.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.436+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.436+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0921150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0865ef0 con 0x555de0918760 2023-10-18T09:18:48.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0802790 con 0x555de09190c0 2023-10-18T09:18:48.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0802430 con 0x555de0917a60 2023-10-18T09:18:48.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91a1c04b80 10 client.0 _ll_register_callbacks cb 0x555de074cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:48.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0921150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50572/0 (socket says 172.21.15.61:50572) 2023-10-18T09:18:48.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43566/0 (socket says 172.21.15.61:43566) 2023-10-18T09:18:48.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 -- 172.21.15.61:0/1452180891 learned_addr learned my addr 172.21.15.61:0/1452180891 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 -- 172.21.15.61:0/1452180891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 msgr2=0x555de0920c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 -- 172.21.15.61:0/1452180891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 msgr2=0x555de0921150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0921150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f9187fff700 1 -- 172.21.15.61:0/1452180891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91780095f0 con 0x555de09190c0 2023-10-18T09:18:48.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.437+0000 7f91877fe700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:48.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9187fff700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9180007b50 tx=0x7f9180007b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97a00876b8c38b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:48.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000e040 con 0x555de09190c0 2023-10-18T09:18:48.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9180004db0 con 0x555de09190c0 2023-10-18T09:18:48.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555de086af70 con 0x555de09190c0 2023-10-18T09:18:48.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9180003a00 con 0x555de09190c0 2023-10-18T09:18:48.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f91a1c04b80 10 client.25688 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:48.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555de0690e60 con 0x555de09190c0 2023-10-18T09:18:48.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555de086af70 con 0x555de09190c0 2023-10-18T09:18:48.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.438+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9180004020 con 0x555de09190c0 2023-10-18T09:18:48.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 10 client.25688 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:48.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:18:48.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 20 client.25688 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:48.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de092ca80 0x555de092eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 4 client.25688 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:48.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555de08e2930 con 0x555de092ca80 2023-10-18T09:18:48.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91877fe700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de092ca80 0x555de092eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f91877fe700 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de092ca80 0x555de092eea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7761f535a2cd4650 server_cookie=e1e888d74b414d02 in_seq=0 out_seq=0 2023-10-18T09:18:48.454 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.439+0000 7f9166ffd700 10 client.25688 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f9166ffd700 1 -- 172.21.15.61:0/1452180891 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555de08e2930 con 0x555de092ca80 2023-10-18T09:18:48.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f9166ffd700 10 client.25688 handle_command_reply: tid=0 2023-10-18T09:18:48.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 client.25688 shutdown 2023-10-18T09:18:48.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 msgr2=0x555de0928140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.455 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9180007b50 tx=0x7f9180007b80 comp rx=0 tx=0).stop 2023-10-18T09:18:48.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 shutdown_connections 2023-10-18T09:18:48.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de092ca80 0x555de092eea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0928140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.456 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0917a60 0x555de0920c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f9166ffd700 0 client.25688 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1452180891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0921150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 >> 172.21.15.61:0/1452180891 conn(0x555de08877a0 msgr2=0x555de0912100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.440+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 shutdown_connections 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.441+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1452180891 wait complete. 2023-10-18T09:18:48.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.441+0000 7f91a1c04b80 20 client.25688 trim_cache size 0 max 16384 2023-10-18T09:18:48.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.459+0000 7f91a1c04b80 1 Processor -- start 2023-10-18T09:18:48.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.459+0000 7f91a1c04b80 1 -- start start 2023-10-18T09:18:48.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.459+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0917a60 0x555de0912100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.459+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0928100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de06e69a0 con 0x555de0917a60 2023-10-18T09:18:48.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de086af70 con 0x555de09190c0 2023-10-18T09:18:48.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de090f690 con 0x555de0918760 2023-10-18T09:18:48.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0928100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f91877fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0917a60 0x555de0912100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50584/0 (socket says 172.21.15.61:50584) 2023-10-18T09:18:48.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 -- 172.21.15.61:0/1589756750 learned_addr learned my addr 172.21.15.61:0/1589756750 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0928100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43582/0 (socket says 172.21.15.61:43582) 2023-10-18T09:18:48.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 -- 172.21.15.61:0/1589756750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 msgr2=0x555de0928100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0928100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 -- 172.21.15.61:0/1589756750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0917a60 msgr2=0x555de0912100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0917a60 0x555de0912100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 -- 172.21.15.61:0/1589756750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555de0933ad0 con 0x555de09190c0 2023-10-18T09:18:48.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.460+0000 7f9187fff700 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9180004380 tx=0x7f9180008fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3627fb7e1d40e745 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f916e7fc700 1 -- 172.21.15.61:0/1589756750 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180003c00 con 0x555de09190c0 2023-10-18T09:18:48.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f916e7fc700 1 -- 172.21.15.61:0/1589756750 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f918000e030 con 0x555de09190c0 2023-10-18T09:18:48.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f916e7fc700 1 -- 172.21.15.61:0/1589756750 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180015680 con 0x555de09190c0 2023-10-18T09:18:48.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1589756750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 msgr2=0x555de0912640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9180004380 tx=0x7f9180008fa0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1589756750 shutdown_connections 2023-10-18T09:18:48.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de09190c0 0x555de0912640 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0917a60 0x555de0912100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/1589756750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de0918760 0x555de0928100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1589756750 >> 172.21.15.61:0/1589756750 conn(0x555de08877a0 msgr2=0x555de091ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.461+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1589756750 shutdown_connections 2023-10-18T09:18:48.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.462+0000 7f91a1c04b80 1 -- 172.21.15.61:0/1589756750 wait complete. 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.462+0000 7f91a1c04b80 1 -- start start 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.462+0000 7f91a1c04b80 1 Processor -- start 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de09273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0927930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0980fe0 con 0x555de0918760 2023-10-18T09:18:48.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0981150 con 0x555de0917a60 2023-10-18T09:18:48.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0981310 con 0x555de09190c0 2023-10-18T09:18:48.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91a1c04b80 10 client.0 _ll_register_callbacks cb 0x555de074cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:48.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f9187fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f9186ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0927930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de09273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f9187fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43592/0 (socket says 172.21.15.61:43592) 2023-10-18T09:18:48.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f9187fff700 1 -- 172.21.15.61:0/3783537539 learned_addr learned my addr 172.21.15.61:0/3783537539 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 -- 172.21.15.61:0/3783537539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 msgr2=0x555de0980aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 -- 172.21.15.61:0/3783537539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 msgr2=0x555de0927930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0927930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f91877fe700 1 -- 172.21.15.61:0/3783537539 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555de0690e60 con 0x555de0917a60 2023-10-18T09:18:48.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.463+0000 7f9187fff700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:48.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91877fe700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de09273f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91780a2b10 tx=0x7f917809a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62ae17c99ca69f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:48.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917809afa0 con 0x555de0917a60 2023-10-18T09:18:48.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f917809b100 con 0x555de0917a60 2023-10-18T09:18:48.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91780990e0 con 0x555de0917a60 2023-10-18T09:18:48.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f917809a230 con 0x555de0917a60 2023-10-18T09:18:48.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 10 client.25694 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:48.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555de0679ce0 con 0x555de0917a60 2023-10-18T09:18:48.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91780990e0 con 0x555de0917a60 2023-10-18T09:18:48.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f91780a2bb0 con 0x555de0917a60 2023-10-18T09:18:48.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 10 client.25694 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:48.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:18:48.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 20 client.25694 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:48.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de09829a0 0x555de0984dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 4 client.25694 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T09:18:48.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.464+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555de085eed0 con 0x555de09829a0 2023-10-18T09:18:48.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.465+0000 7f9186ffd700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de09829a0 0x555de0984dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.465+0000 7f9186ffd700 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de09829a0 0x555de0984dc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0079168219bf7ff server_cookie=ea623b0b049f74bc in_seq=0 out_seq=0 2023-10-18T09:18:48.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.465+0000 7f9166ffd700 10 client.25694 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.466+0000 7f9166ffd700 1 -- 172.21.15.61:0/3783537539 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555de085eed0 con 0x555de09829a0 2023-10-18T09:18:48.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.466+0000 7f9166ffd700 10 client.25694 handle_command_reply: tid=0 2023-10-18T09:18:48.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.466+0000 7f91a1c04b80 1 client.25694 shutdown 2023-10-18T09:18:48.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.466+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 msgr2=0x555de09273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.466+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de09273f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91780a2b10 tx=0x7f917809a750 comp rx=0 tx=0).stop 2023-10-18T09:18:48.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 shutdown_connections 2023-10-18T09:18:48.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555de09829a0 0x555de0984dc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555de09190c0 0x555de0980aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555de0917a60 0x555de09273f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 --2- 172.21.15.61:0/3783537539 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555de0918760 0x555de0927930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 >> 172.21.15.61:0/3783537539 conn(0x555de08877a0 msgr2=0x555de0912100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f9166ffd700 0 client.25694 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 shutdown_connections 2023-10-18T09:18:48.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 1 -- 172.21.15.61:0/3783537539 wait complete. 2023-10-18T09:18:48.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f91a1c04b80 20 client.25694 trim_cache size 0 max 16384 2023-10-18T09:18:48.483 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:48.484 INFO:teuthology.orchestra.run.smithi061.stdout: "return_code": 0, 2023-10-18T09:18:48.484 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_tag": "b26da0d4-5ffe-4029-b83b-4384758461fe", 2023-10-18T09:18:48.484 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": "asynchronous" 2023-10-18T09:18:48.484 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:48.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 msgr2=0x7f91700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.467+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 0x7f91700a39c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f917c009df0 tx=0x7f917c009260 comp rx=0 tx=0).stop 2023-10-18T09:18:48.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 msgr2=0x7f9188196740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9178003730 tx=0x7f917800f690 comp rx=0 tx=0).stop 2023-10-18T09:18:48.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 shutdown_connections 2023-10-18T09:18:48.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f91700a1500 0x7f91700a39c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9188101220 0x7f918819dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91880ff690 0x7f9188196740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 --2- 172.21.15.61:0/3663726229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9188100880 0x7f9188196c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 >> 172.21.15.61:0/3663726229 conn(0x7f91880fac00 msgr2=0x7f91880fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 shutdown_connections 2023-10-18T09:18:48.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.468+0000 7f918e1ff700 1 -- 172.21.15.61:0/3663726229 wait complete. 2023-10-18T09:18:48.489 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:18:48.502 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-10-18T09:18:48.660 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-10-18T09:18:48.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.719+0000 7f6ae1bff700 1 -- 172.21.15.61:0/619932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc103ba0 msgr2=0x7f6adc105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.719+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/619932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc103ba0 0x7f6adc105f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00098f0 tx=0x7f6ad0009c00 comp rx=0 tx=0).stop 2023-10-18T09:18:48.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 -- 172.21.15.61:0/619932966 shutdown_connections 2023-10-18T09:18:48.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/619932966 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/619932966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc101270 0x7f6adc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/619932966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc103ba0 0x7f6adc105f90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 -- 172.21.15.61:0/619932966 >> 172.21.15.61:0/619932966 conn(0x7f6adc0fac00 msgr2=0x7f6adc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 -- 172.21.15.61:0/619932966 shutdown_connections 2023-10-18T09:18:48.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.720+0000 7f6ae1bff700 1 -- 172.21.15.61:0/619932966 wait complete. 2023-10-18T09:18:48.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.721+0000 7f6ae1bff700 1 Processor -- start 2023-10-18T09:18:48.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.721+0000 7f6ae1bff700 1 -- start start 2023-10-18T09:18:48.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 0x7f6adc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a04e0 con 0x7f6adc1064d0 2023-10-18T09:18:48.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a0620 con 0x7f6adc103ba0 2023-10-18T09:18:48.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6ae1bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1a07b0 con 0x7f6adc101270 2023-10-18T09:18:48.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 0x7f6adc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43602/0 (socket says 172.21.15.61:43602) 2023-10-18T09:18:48.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34798/0 (socket says 172.21.15.61:34798) 2023-10-18T09:18:48.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.722+0000 7f6adb7fe700 1 -- 172.21.15.61:0/3775592494 learned_addr learned my addr 172.21.15.61:0/3775592494 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 -- 172.21.15.61:0/3775592494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 msgr2=0x7f6adc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 -- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 msgr2=0x7f6adc19ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 -- 172.21.15.61:0/3775592494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0095f0 con 0x7f6adc103ba0 2023-10-18T09:18:48.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:48.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:48.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 0x7f6adc198ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009230 tx=0x7f6ad000b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6ad8ff9700 1 -- 172.21.15.61:0/3775592494 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad001c070 con 0x7f6adc103ba0 2023-10-18T09:18:48.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6ad8ff9700 1 -- 172.21.15.61:0/3775592494 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad000f7d0 con 0x7f6adc103ba0 2023-10-18T09:18:48.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6ad8ff9700 1 -- 172.21.15.61:0/3775592494 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00038e0 con 0x7f6adc103ba0 2023-10-18T09:18:48.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad0009520 con 0x7f6adc103ba0 2023-10-18T09:18:48.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.723+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a0c70 con 0x7f6adc103ba0 2023-10-18T09:18:48.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.726+0000 7f6ad8ff9700 1 -- 172.21.15.61:0/3775592494 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6ad0020030 con 0x7f6adc103ba0 2023-10-18T09:18:48.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 Processor -- start 2023-10-18T09:18:48.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 -- start start 2023-10-18T09:18:48.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57157e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57157ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5714f16e0 con 0x55f57157e5d0 2023-10-18T09:18:48.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5714d7c60 con 0x55f57157f2d0 2023-10-18T09:18:48.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.728+0000 7f6af5595b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5714d5580 con 0x55f57157fc30 2023-10-18T09:18:48.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57157ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50612/0 (socket says 172.21.15.61:50612) 2023-10-18T09:18:48.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 -- 172.21.15.61:0/3245598582 learned_addr learned my addr 172.21.15.61:0/3245598582 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57157e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6ad8ff9700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 0x7f6ac40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 -- 172.21.15.61:0/3245598582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 msgr2=0x55f57157e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57157e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 -- 172.21.15.61:0/3245598582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 msgr2=0x55f57157ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57157ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6adbfff700 1 -- 172.21.15.61:0/3245598582 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f57148e3f0 con 0x55f57157f2d0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.729+0000 7f6ad8ff9700 1 -- 172.21.15.61:0/3775592494 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6ad00cde70 con 0x7f6adc103ba0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.730+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009520 tx=0x7f6ad4004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e161db60398f4312 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.730+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 0x7f6ac40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.730+0000 7f6ac1ffb700 1 -- 172.21.15.61:0/3245598582 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4007500 con 0x55f57157f2d0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.730+0000 7f6ac1ffb700 1 -- 172.21.15.61:0/3245598582 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad400f040 con 0x55f57157f2d0 2023-10-18T09:18:48.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.730+0000 7f6ac1ffb700 1 -- 172.21.15.61:0/3245598582 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad400ec60 con 0x55f57157f2d0 2023-10-18T09:18:48.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 -- 172.21.15.61:0/3245598582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 msgr2=0x55f57157f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 0x7f6ac40a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401c000 tx=0x7f6ad400ee40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009520 tx=0x7f6ad4004320 comp rx=0 tx=0).stop 2023-10-18T09:18:48.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 -- 172.21.15.61:0/3245598582 shutdown_connections 2023-10-18T09:18:48.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57157ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57157f6f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3245598582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57157e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 -- 172.21.15.61:0/3245598582 >> 172.21.15.61:0/3245598582 conn(0x55f5714f2720 msgr2=0x55f571578b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 -- 172.21.15.61:0/3245598582 shutdown_connections 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.731+0000 7f6af5595b80 1 -- 172.21.15.61:0/3245598582 wait complete. 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.732+0000 7f6af5595b80 1 -- start start 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.732+0000 7f6af5595b80 1 Processor -- start 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 0x55f571587ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57158f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5714d0e70 con 0x55f57157f2d0 2023-10-18T09:18:48.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57146d710 con 0x55f57157e5d0 2023-10-18T09:18:48.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57146d3b0 con 0x55f57157fc30 2023-10-18T09:18:48.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6af5595b80 10 client.0 _ll_register_callbacks cb 0x55f5713b7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:48.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50614/0 (socket says 172.21.15.61:50614) 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 -- 172.21.15.61:0/2674544183 learned_addr learned my addr 172.21.15.61:0/2674544183 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adaffd700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 0x55f571587ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adbfff700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57158f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 -- 172.21.15.61:0/2674544183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 msgr2=0x55f57158f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57158f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 -- 172.21.15.61:0/2674544183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 msgr2=0x55f571587ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 0x55f571587ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adb7fe700 1 -- 172.21.15.61:0/2674544183 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad4003be0 con 0x55f57157e5d0 2023-10-18T09:18:48.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6adaffd700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 0x55f571587ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:18:48.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.733+0000 7f6adbfff700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57158f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6acc012e60 tx=0x7f6acc00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1e0c7fc117eb27 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc017070 con 0x55f57157e5d0 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6acc0120d0 con 0x55f57157e5d0 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5714d5ef0 con 0x55f57157e5d0 2023-10-18T09:18:48.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6acc012290 con 0x55f57157e5d0 2023-10-18T09:18:48.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6af5595b80 10 client.25706 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:48.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.734+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f5712fbde0 con 0x55f57157e5d0 2023-10-18T09:18:48.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f5714d5ef0 con 0x55f57157e5d0 2023-10-18T09:18:48.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6acc010430 con 0x55f57157e5d0 2023-10-18T09:18:48.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 10 client.25706 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:48.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:18:48.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 20 client.25706 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:48.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715938d0 0x55f571595cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 4 client.25706 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:18:48.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f57154d8b0 con 0x55f5715938d0 2023-10-18T09:18:48.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.735+0000 7f6adaffd700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715938d0 0x55f571595cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.736+0000 7f6adaffd700 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715938d0 0x55f571595cf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5694c24226698e22 server_cookie=c6d8a835d8b75c6b in_seq=0 out_seq=0 2023-10-18T09:18:48.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.736+0000 7f6abaffd700 10 client.25706 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.736+0000 7f6abaffd700 1 -- 172.21.15.61:0/2674544183 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f57154d8b0 con 0x55f5715938d0 2023-10-18T09:18:48.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.736+0000 7f6abaffd700 10 client.25706 handle_command_reply: tid=0 2023-10-18T09:18:48.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 client.25706 shutdown 2023-10-18T09:18:48.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 msgr2=0x55f571587ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6acc012e60 tx=0x7f6acc00f6a0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 shutdown_connections 2023-10-18T09:18:48.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715938d0 0x55f571595cf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f57158f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f571587ab0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6abaffd700 0 client.25706 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 --2- 172.21.15.61:0/2674544183 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157f2d0 0x55f571587ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 >> 172.21.15.61:0/2674544183 conn(0x55f5714f2720 msgr2=0x55f57157d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 shutdown_connections 2023-10-18T09:18:48.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 1 -- 172.21.15.61:0/2674544183 wait complete. 2023-10-18T09:18:48.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.737+0000 7f6af5595b80 20 client.25706 trim_cache size 0 max 16384 2023-10-18T09:18:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 Processor -- start 2023-10-18T09:18:48.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 -- start start 2023-10-18T09:18:48.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57158f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f571586fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571351920 con 0x55f57157e5d0 2023-10-18T09:18:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5715790d0 con 0x55f57157f2d0 2023-10-18T09:18:48.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.759+0000 7f6af5595b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f571579a20 con 0x55f57157fc30 2023-10-18T09:18:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f571586fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f571586fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:43618/0 (socket says 172.21.15.61:43618) 2023-10-18T09:18:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50616/0 (socket says 172.21.15.61:50616) 2023-10-18T09:18:48.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adbfff700 1 -- 172.21.15.61:0/3836451790 learned_addr learned my addr 172.21.15.61:0/3836451790 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57158f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 -- 172.21.15.61:0/3836451790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 msgr2=0x55f571586fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f571586fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 -- 172.21.15.61:0/3836451790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 msgr2=0x55f57158f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57158f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 -- 172.21.15.61:0/3836451790 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5712e4c60 con 0x55f57157f2d0 2023-10-18T09:18:48.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.760+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ad009ab70 tx=0x7f6ad0019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03647f3b68528d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.761+0000 7f6ac27fc700 1 -- 172.21.15.61:0/3836451790 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad009a9b0 con 0x55f57157f2d0 2023-10-18T09:18:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.761+0000 7f6ac27fc700 1 -- 172.21.15.61:0/3836451790 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad009a7d0 con 0x55f57157f2d0 2023-10-18T09:18:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.761+0000 7f6ac27fc700 1 -- 172.21.15.61:0/3836451790 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad009efc0 con 0x55f57157f2d0 2023-10-18T09:18:48.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 -- 172.21.15.61:0/3836451790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 msgr2=0x55f57158eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ad009ab70 tx=0x7f6ad0019040 comp rx=0 tx=0).stop 2023-10-18T09:18:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 -- 172.21.15.61:0/3836451790 shutdown_connections 2023-10-18T09:18:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157fc30 0x55f571586fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157e5d0 0x55f57158f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3836451790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157f2d0 0x55f57158eff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.762+0000 7f6af5595b80 1 -- 172.21.15.61:0/3836451790 >> 172.21.15.61:0/3836451790 conn(0x55f5714f2720 msgr2=0x55f57157e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.763+0000 7f6af5595b80 1 -- 172.21.15.61:0/3836451790 shutdown_connections 2023-10-18T09:18:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.763+0000 7f6af5595b80 1 -- 172.21.15.61:0/3836451790 wait complete. 2023-10-18T09:18:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.763+0000 7f6af5595b80 1 -- start start 2023-10-18T09:18:48.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.763+0000 7f6af5595b80 1 Processor -- start 2023-10-18T09:18:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f57158cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157f2d0 0x55f57158d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57158ea30 con 0x55f57157fc30 2023-10-18T09:18:48.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57158eba0 con 0x55f57157e5d0 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f57158ed60 con 0x55f57157f2d0 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6af5595b80 10 client.0 _ll_register_callbacks cb 0x55f5713b7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34828/0 (socket says 172.21.15.61:34828) 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6adbfff700 1 -- 172.21.15.61:0/3916622297 learned_addr learned my addr 172.21.15.61:0/3916622297 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:18:48.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f57158cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.764+0000 7f6adaffd700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157f2d0 0x55f57158d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 -- 172.21.15.61:0/3916622297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157f2d0 msgr2=0x55f57158d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157f2d0 0x55f57158d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 -- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 msgr2=0x55f57158cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f57158cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 -- 172.21.15.61:0/3916622297 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a0c70 con 0x55f57157fc30 2023-10-18T09:18:48.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f57158cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:18:48.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6adbfff700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400eaa0 tx=0x7f6ad4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b06c7bea82e92b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:18:48.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.765+0000 7f6abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:18:48.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4025070 con 0x55f57157fc30 2023-10-18T09:18:48.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6ad40049e0 con 0x55f57157fc30 2023-10-18T09:18:48.776 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ad400f040 con 0x55f57157fc30 2023-10-18T09:18:48.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ad400ece0 con 0x55f57157fc30 2023-10-18T09:18:48.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6af5595b80 10 client.15732 fetch_fsmap learned FSMap version 28 2023-10-18T09:18:48.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.767+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55f571322250 con 0x55f57157fc30 2023-10-18T09:18:48.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ad400f040 con 0x55f57157fc30 2023-10-18T09:18:48.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 28) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f6ad4033070 con 0x55f57157fc30 2023-10-18T09:18:48.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2023-10-18T09:18:48.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:18:48.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 20 client.15732 populate_metadata read hostname 'smithi061' 2023-10-18T09:18:48.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715e8950 0x55f5715ead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:18:48.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 4 client.15732 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:18:48.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.768+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5714c9e50 con 0x55f5715e8950 2023-10-18T09:18:48.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715e8950 0x55f5715ead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:18:48.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6adb7fe700 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715e8950 0x55f5715ead70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e09dbd1fa56c3a3 server_cookie=e6bf3c010eff9a1d in_seq=0 out_seq=0 2023-10-18T09:18:48.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6abaffd700 10 client.15732 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6abaffd700 1 -- 172.21.15.61:0/3916622297 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f5714c9e50 con 0x55f5715e8950 2023-10-18T09:18:48.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6abaffd700 10 client.15732 handle_command_reply: tid=0 2023-10-18T09:18:48.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.769+0000 7f6af5595b80 1 client.15732 shutdown 2023-10-18T09:18:48.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 msgr2=0x55f57158e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400eaa0 tx=0x7f6ad4027000 comp rx=0 tx=0).stop 2023-10-18T09:18:48.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 shutdown_connections 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f5715e8950 0x55f5715ead70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f57157fc30 0x55f57158e4f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f57157e5d0 0x55f57158cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 --2- 172.21.15.61:0/3916622297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57157f2d0 0x55f57158d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 >> 172.21.15.61:0/3916622297 conn(0x55f5714f2720 msgr2=0x55f571586fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6abaffd700 0 client.15732 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:18:48.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 shutdown_connections 2023-10-18T09:18:48.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 1 -- 172.21.15.61:0/3916622297 wait complete. 2023-10-18T09:18:48.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.770+0000 7f6af5595b80 20 client.15732 trim_cache size 0 max 16384 2023-10-18T09:18:48.783 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:18:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T09:18:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:18:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "b26da0d4-5ffe-4029-b83b-4384758461fe": { 2023-10-18T09:18:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:18:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "b26da0d4-5ffe-4029-b83b-4384758461fe", 2023-10-18T09:18:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:18:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:18:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:18:48.784 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:18:48.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 msgr2=0x7f6ac40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 0x7f6ac40a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ad401c000 tx=0x7f6ad400ee40 comp rx=0 tx=0).stop 2023-10-18T09:18:48.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 msgr2=0x7f6adc198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:18:48.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 0x7f6adc198ef0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009230 tx=0x7f6ad000b790 comp rx=0 tx=0).stop 2023-10-18T09:18:48.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 shutdown_connections 2023-10-18T09:18:48.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6ac40a1480 0x7f6ac40a3940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6adc1064d0 0x7f6adc19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6adc101270 0x7f6adc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 --2- 172.21.15.61:0/3775592494 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6adc103ba0 0x7f6adc198ef0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 >> 172.21.15.61:0/3775592494 conn(0x7f6adc0fac00 msgr2=0x7f6adc0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 shutdown_connections 2023-10-18T09:18:48.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:18:48.771+0000 7f6ae1bff700 1 -- 172.21.15.61:0/3775592494 wait complete. 2023-10-18T09:18:48.794 INFO:tasks.cephfs.filesystem:scrub status for tag:b26da0d4-5ffe-4029-b83b-4384758461fe - {'path': '/', 'tag': 'b26da0d4-5ffe-4029-b83b-4384758461fe', 'options': 'recursive,force'} 2023-10-18T09:18:49.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:48 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/918565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:49.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:48 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/918565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:49.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:48 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/918565589' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:18:50.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: scrub queued for path: / 2023-10-18T09:18:50.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:18:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-18T09:18:50.144 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:49 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: scrub queued for path: / 2023-10-18T09:18:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:18:50.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-18T09:18:50.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:49 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: scrub queued for path: / 2023-10-18T09:18:50.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:18:50.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:18:50.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:50.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: pgmap v366: 129 pgs: 129 active+clean; 649 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 9 op/s 2023-10-18T09:18:50.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:49 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:18:51.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:18:51.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:50 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:18:51.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:18:51.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:50 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:18:51.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:18:51.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:50 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:18:52.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:51 smithi061 ceph-mon[98573]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-18T09:18:52.066 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-10-18T09:18:52.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:51 smithi170 ceph-mon[135150]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-18T09:18:52.165 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-10-18T09:18:52.170 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-10-18T09:18:52.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:51 smithi110 ceph-mon[134818]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2023-10-18T09:18:52.274 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-18T09:18:52.372 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/: 2023-10-18T09:18:52.373 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:18:52.373 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:18:52.373 INFO:teuthology.orchestra.run.smithi061.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:18:52.373 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 data 2023-10-18T09:18:52.374 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:18:52.374 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data: 2023-10-18T09:18:52.374 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:18:52.374 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:18:52.375 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:18:52.377 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-10-18T09:18:52.501 INFO:teuthology.orchestra.run.smithi061.stdout:[Service] 2023-10-18T09:18:52.502 INFO:teuthology.orchestra.run.smithi061.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-10-18T09:18:52.502 INFO:teuthology.orchestra.run.smithi061.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-10-18T09:18:52.506 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-10-18T09:18:52.704 INFO:teuthology.orchestra.run.smithi061.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-10-18T09:18:54.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:53 smithi170 ceph-mon[135150]: pgmap v368: 129 pgs: 129 active+clean; 659 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-10-18T09:18:54.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:53 smithi110 ceph-mon[134818]: pgmap v368: 129 pgs: 129 active+clean; 659 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-10-18T09:18:54.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:53 smithi061 ceph-mon[98573]: pgmap v368: 129 pgs: 129 active+clean; 659 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2023-10-18T09:18:56.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:55 smithi170 ceph-mon[135150]: pgmap v369: 129 pgs: 129 active+clean; 2.8 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 190 KiB/s wr, 17 op/s 2023-10-18T09:18:56.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:55 smithi110 ceph-mon[134818]: pgmap v369: 129 pgs: 129 active+clean; 2.8 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 190 KiB/s wr, 17 op/s 2023-10-18T09:18:56.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:55 smithi061 ceph-mon[98573]: pgmap v369: 129 pgs: 129 active+clean; 2.8 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 190 KiB/s wr, 17 op/s 2023-10-18T09:18:57.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:57 smithi170 ceph-mon[135150]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.8 MiB/s wr, 30 op/s 2023-10-18T09:18:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:57 smithi110 ceph-mon[134818]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.8 MiB/s wr, 30 op/s 2023-10-18T09:18:58.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:57 smithi061 ceph-mon[98573]: pgmap v370: 129 pgs: 129 active+clean; 23 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.8 MiB/s wr, 30 op/s 2023-10-18T09:19:00.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:18:59 smithi110 ceph-mon[134818]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 30 op/s 2023-10-18T09:19:00.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:18:59 smithi061 ceph-mon[98573]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 30 op/s 2023-10-18T09:19:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:18:59 smithi170 ceph-mon[135150]: pgmap v371: 129 pgs: 129 active+clean; 23 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.9 MiB/s wr, 30 op/s 2023-10-18T09:19:01.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:00 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:01.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:00 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:00 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:01.453 INFO:teuthology.orchestra.run.smithi061.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-10-18T09:19:01.459 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-18T09:19:01.582 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/: 2023-10-18T09:19:01.583 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.583 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.583 INFO:teuthology.orchestra.run.smithi061.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:19:01.583 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 data 2023-10-18T09:19:01.584 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.584 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data: 2023-10-18T09:19:01.584 INFO:teuthology.orchestra.run.smithi061.stdout:total 30 2023-10-18T09:19:01.584 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 . 2023-10-18T09:19:01.584 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:19:01.585 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 base 2023-10-18T09:19:01.585 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 18 09:18 global 2023-10-18T09:19:01.585 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 log 2023-10-18T09:19:01.585 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_commit_ts 2023-10-18T09:19:01.585 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_dynshmem 2023-10-18T09:19:01.586 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 18 09:18 pg_hba.conf 2023-10-18T09:19:01.586 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 18 09:18 pg_ident.conf 2023-10-18T09:19:01.586 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 pg_logical 2023-10-18T09:19:01.586 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 pg_multixact 2023-10-18T09:19:01.586 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 pg_notify 2023-10-18T09:19:01.587 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_replslot 2023-10-18T09:19:01.587 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_serial 2023-10-18T09:19:01.587 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_snapshots 2023-10-18T09:19:01.587 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_stat 2023-10-18T09:19:01.587 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_stat_tmp 2023-10-18T09:19:01.588 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 pg_subtrans 2023-10-18T09:19:01.588 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_tblspc 2023-10-18T09:19:01.588 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_twophase 2023-10-18T09:19:01.588 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:19:01.588 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 pg_wal 2023-10-18T09:19:01.589 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 pg_xact 2023-10-18T09:19:01.589 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 18 09:18 postgresql.auto.conf 2023-10-18T09:19:01.589 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 18 09:18 postgresql.conf 2023-10-18T09:19:01.589 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.589 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base: 2023-10-18T09:19:01.590 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.590 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 . 2023-10-18T09:19:01.590 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.590 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 1 2023-10-18T09:19:01.590 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 12346 2023-10-18T09:19:01.591 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:19 12347 2023-10-18T09:19:01.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.591 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-18T09:19:01.591 INFO:teuthology.orchestra.run.smithi061.stdout:total 7193 2023-10-18T09:19:01.591 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 . 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 .. 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 113 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12182_fsm 2023-10-18T09:19:01.592 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12184 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12187_fsm 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187_vm 2023-10-18T09:19:01.593 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12189 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12191 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12192_fsm 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:19:01.594 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:19:01.595 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:19:01.595 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12197_fsm 2023-10-18T09:19:01.595 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197_vm 2023-10-18T09:19:01.595 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12199 2023-10-18T09:19:01.595 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:19:01.596 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:19:01.596 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:19:01.596 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:19:01.596 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:19:01.596 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12206 2023-10-18T09:19:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207 2023-10-18T09:19:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12207_fsm 2023-10-18T09:19:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:19:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:19:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:19:01.598 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:19:01.598 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:19:01.598 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12216 2023-10-18T09:19:01.598 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:18 1247 2023-10-18T09:19:01.598 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1247_fsm 2023-10-18T09:19:01.599 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1247_vm 2023-10-18T09:19:01.599 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:18 1249 2023-10-18T09:19:01.599 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1249_fsm 2023-10-18T09:19:01.599 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1249_vm 2023-10-18T09:19:01.599 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:19:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1255_fsm 2023-10-18T09:19:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:19:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:18 1259 2023-10-18T09:19:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1259_fsm 2023-10-18T09:19:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1259_vm 2023-10-18T09:19:01.601 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:19:01.601 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417_vm 2023-10-18T09:19:01.601 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418 2023-10-18T09:19:01.601 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418_vm 2023-10-18T09:19:01.601 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:19:01.602 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 175 2023-10-18T09:19:01.602 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2187 2023-10-18T09:19:01.602 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:19:01.602 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:19:01.603 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328 2023-10-18T09:19:01.603 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:19:01.603 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:19:01.603 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:19:01.603 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2337 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2600_fsm 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2601_fsm 2023-10-18T09:19:01.604 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:19:01.605 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:19:01.605 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2602_fsm 2023-10-18T09:19:01.605 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2602_vm 2023-10-18T09:19:01.605 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2603 2023-10-18T09:19:01.605 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2603_fsm 2023-10-18T09:19:01.606 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:19:01.606 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:19:01.606 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:19:01.606 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:19:01.606 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2605_fsm 2023-10-18T09:19:01.607 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2605_vm 2023-10-18T09:19:01.607 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606 2023-10-18T09:19:01.607 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:19:01.607 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606_vm 2023-10-18T09:19:01.607 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:19:01.608 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:19:01.608 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:19:01.608 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 18 09:18 2608 2023-10-18T09:19:01.608 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2608_fsm 2023-10-18T09:19:01.608 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2608_vm 2023-10-18T09:19:01.609 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:19:01.609 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:19:01.609 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:19:01.609 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610 2023-10-18T09:19:01.609 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610_fsm 2023-10-18T09:19:01.610 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2610_vm 2023-10-18T09:19:01.610 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:19:01.610 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611_vm 2023-10-18T09:19:01.610 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:19:01.610 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:19:01.611 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:19:01.611 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613 2023-10-18T09:19:01.611 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613_vm 2023-10-18T09:19:01.611 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:19:01.611 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:19:01.612 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:19:01.612 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:19:01.612 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:19:01.612 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:19:01.612 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:19:01.613 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2617_fsm 2023-10-18T09:19:01.613 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:19:01.613 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:18 2618 2023-10-18T09:19:01.613 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2618_fsm 2023-10-18T09:19:01.613 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2618_vm 2023-10-18T09:19:01.614 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:18 2619 2023-10-18T09:19:01.614 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2619_fsm 2023-10-18T09:19:01.614 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2619_vm 2023-10-18T09:19:01.614 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620_vm 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2651 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2652 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2653 2023-10-18T09:19:01.615 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2654 2023-10-18T09:19:01.616 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2655 2023-10-18T09:19:01.616 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:19:01.616 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:19:01.616 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 18 09:18 2658 2023-10-18T09:19:01.616 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2659 2023-10-18T09:19:01.617 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:19:01.617 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:19:01.617 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2662 2023-10-18T09:19:01.617 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2663 2023-10-18T09:19:01.617 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2664 2023-10-18T09:19:01.618 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2665 2023-10-18T09:19:01.618 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2666 2023-10-18T09:19:01.618 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2667 2023-10-18T09:19:01.618 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2668 2023-10-18T09:19:01.618 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:19:01.619 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:19:01.619 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:18 2673 2023-10-18T09:19:01.619 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:18 2674 2023-10-18T09:19:01.619 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:19:01.619 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2678 2023-10-18T09:19:01.620 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2679 2023-10-18T09:19:01.620 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2680 2023-10-18T09:19:01.620 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:19:01.620 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2682 2023-10-18T09:19:01.620 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:19:01.621 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2684 2023-10-18T09:19:01.621 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2685 2023-10-18T09:19:01.621 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2686 2023-10-18T09:19:01.621 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:19:01.621 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:19:01.622 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:19:01.622 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2690 2023-10-18T09:19:01.622 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:18 2691 2023-10-18T09:19:01.622 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2692 2023-10-18T09:19:01.622 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2693 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2696 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2701 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2702 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2703 2023-10-18T09:19:01.623 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2704 2023-10-18T09:19:01.624 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:19:01.624 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2753_fsm 2023-10-18T09:19:01.624 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:19:01.624 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2754 2023-10-18T09:19:01.624 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2755 2023-10-18T09:19:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:19:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:19:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830 2023-10-18T09:19:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830_vm 2023-10-18T09:19:01.625 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2831 2023-10-18T09:19:01.626 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:19:01.626 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832_vm 2023-10-18T09:19:01.626 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:19:01.626 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834 2023-10-18T09:19:01.626 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:19:01.627 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:19:01.627 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836 2023-10-18T09:19:01.627 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:19:01.627 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:19:01.627 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:18 2838 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840_fsm 2023-10-18T09:19:01.628 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2841 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995_vm 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079 2023-10-18T09:19:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:19:01.630 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079_vm 2023-10-18T09:19:01.630 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3080 2023-10-18T09:19:01.630 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3081 2023-10-18T09:19:01.630 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3085 2023-10-18T09:19:01.630 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118 2023-10-18T09:19:01.631 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:19:01.631 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3119 2023-10-18T09:19:01.631 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3164 2023-10-18T09:19:01.631 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256 2023-10-18T09:19:01.631 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256_vm 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350_vm 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:19:01.632 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3380 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3394 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:19:01.633 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3394_vm 2023-10-18T09:19:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:19:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439 2023-10-18T09:19:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:19:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3440 2023-10-18T09:19:01.634 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 3455 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:19:01.635 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3467 2023-10-18T09:19:01.636 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:19:01.636 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:19:01.636 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501_vm 2023-10-18T09:19:01.636 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3502 2023-10-18T09:19:01.636 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:19:01.637 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3534 2023-10-18T09:19:01.637 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:19:01.637 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3541_fsm 2023-10-18T09:19:01.637 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541_vm 2023-10-18T09:19:01.637 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3542 2023-10-18T09:19:01.638 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3574 2023-10-18T09:19:01.638 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3575 2023-10-18T09:19:01.638 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576 2023-10-18T09:19:01.638 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:19:01.638 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:19:01.639 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:19:01.639 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3597 2023-10-18T09:19:01.639 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:19:01.639 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:19:01.639 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:19:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:19:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:19:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:19:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:19:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:19:01.641 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601_vm 2023-10-18T09:19:01.641 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:19:01.641 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3602_fsm 2023-10-18T09:19:01.641 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602_vm 2023-10-18T09:19:01.641 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3603 2023-10-18T09:19:01.642 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3603_fsm 2023-10-18T09:19:01.642 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3603_vm 2023-10-18T09:19:01.642 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3604 2023-10-18T09:19:01.642 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:19:01.642 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3606 2023-10-18T09:19:01.646 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3607 2023-10-18T09:19:01.646 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:19:01.646 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:19:01.646 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3712 2023-10-18T09:19:01.647 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764 2023-10-18T09:19:01.647 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:19:01.647 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764_vm 2023-10-18T09:19:01.647 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:19:01.647 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:19:01.648 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3997 2023-10-18T09:19:01.648 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:19:01.648 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 548 2023-10-18T09:19:01.648 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 549 2023-10-18T09:19:01.648 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:19:01.649 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102_vm 2023-10-18T09:19:01.649 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:19:01.649 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104_vm 2023-10-18T09:19:01.649 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106 2023-10-18T09:19:01.649 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106_vm 2023-10-18T09:19:01.650 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6110 2023-10-18T09:19:01.650 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6111 2023-10-18T09:19:01.650 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6112 2023-10-18T09:19:01.650 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:19:01.651 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:19:01.651 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:19:01.651 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826_vm 2023-10-18T09:19:01.651 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:19:01.651 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:19:01.652 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:19:01.652 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:19:01.652 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.652 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-18T09:19:01.652 INFO:teuthology.orchestra.run.smithi061.stdout:total 7193 2023-10-18T09:19:01.653 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 . 2023-10-18T09:19:01.653 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 .. 2023-10-18T09:19:01.653 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:19:01.653 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 113 2023-10-18T09:19:01.654 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:19:01.654 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12182_fsm 2023-10-18T09:19:01.654 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:19:01.654 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12184 2023-10-18T09:19:01.654 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:19:01.655 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:19:01.655 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12187_fsm 2023-10-18T09:19:01.655 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187_vm 2023-10-18T09:19:01.655 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12189 2023-10-18T09:19:01.655 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12191 2023-10-18T09:19:01.656 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:19:01.656 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12192_fsm 2023-10-18T09:19:01.656 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:19:01.656 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:19:01.656 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:19:01.657 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:19:01.657 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12197_fsm 2023-10-18T09:19:01.657 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197_vm 2023-10-18T09:19:01.657 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12199 2023-10-18T09:19:01.657 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:19:01.658 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:19:01.658 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:19:01.658 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:19:01.658 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:19:01.658 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12206 2023-10-18T09:19:01.659 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207 2023-10-18T09:19:01.659 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12207_fsm 2023-10-18T09:19:01.659 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:19:01.659 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:19:01.660 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:19:01.660 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:19:01.660 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:19:01.660 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12216 2023-10-18T09:19:01.660 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:18 1247 2023-10-18T09:19:01.661 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1247_fsm 2023-10-18T09:19:01.661 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1247_vm 2023-10-18T09:19:01.661 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:18 1249 2023-10-18T09:19:01.661 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1249_fsm 2023-10-18T09:19:01.661 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1249_vm 2023-10-18T09:19:01.662 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:19:01.662 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1255_fsm 2023-10-18T09:19:01.662 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:19:01.662 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:18 1259 2023-10-18T09:19:01.662 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1259_fsm 2023-10-18T09:19:01.663 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1259_vm 2023-10-18T09:19:01.663 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:19:01.663 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417_vm 2023-10-18T09:19:01.663 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418 2023-10-18T09:19:01.663 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418_vm 2023-10-18T09:19:01.664 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:19:01.664 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 175 2023-10-18T09:19:01.664 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2187 2023-10-18T09:19:01.664 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:19:01.664 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:19:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328 2023-10-18T09:19:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:19:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:19:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:19:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2337 2023-10-18T09:19:01.666 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:19:01.666 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2600_fsm 2023-10-18T09:19:01.666 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:19:01.666 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2601_fsm 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2602_fsm 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2602_vm 2023-10-18T09:19:01.667 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2603 2023-10-18T09:19:01.668 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2603_fsm 2023-10-18T09:19:01.669 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:19:01.669 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:19:01.669 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:19:01.669 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:19:01.670 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2605_fsm 2023-10-18T09:19:01.670 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2605_vm 2023-10-18T09:19:01.670 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606 2023-10-18T09:19:01.670 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:19:01.670 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606_vm 2023-10-18T09:19:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:19:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:19:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:19:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 18 09:18 2608 2023-10-18T09:19:01.671 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2608_fsm 2023-10-18T09:19:01.672 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2608_vm 2023-10-18T09:19:01.672 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:19:01.672 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:19:01.672 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:19:01.672 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610 2023-10-18T09:19:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610_fsm 2023-10-18T09:19:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2610_vm 2023-10-18T09:19:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:19:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611_vm 2023-10-18T09:19:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:19:01.674 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:19:01.674 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:19:01.674 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613 2023-10-18T09:19:01.674 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613_vm 2023-10-18T09:19:01.674 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:19:01.675 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:19:01.675 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:19:01.675 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:19:01.675 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:19:01.675 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2617_fsm 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:18 2618 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2618_fsm 2023-10-18T09:19:01.676 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2618_vm 2023-10-18T09:19:01.677 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:18 2619 2023-10-18T09:19:01.677 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2619_fsm 2023-10-18T09:19:01.677 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2619_vm 2023-10-18T09:19:01.677 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:19:01.677 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620_vm 2023-10-18T09:19:01.678 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:19:01.678 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2651 2023-10-18T09:19:01.678 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2652 2023-10-18T09:19:01.678 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2653 2023-10-18T09:19:01.678 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2654 2023-10-18T09:19:01.679 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2655 2023-10-18T09:19:01.679 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:19:01.679 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:19:01.679 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 18 09:18 2658 2023-10-18T09:19:01.679 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2659 2023-10-18T09:19:01.680 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:19:01.680 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:19:01.680 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2662 2023-10-18T09:19:01.680 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2663 2023-10-18T09:19:01.680 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2664 2023-10-18T09:19:01.681 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2665 2023-10-18T09:19:01.681 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2666 2023-10-18T09:19:01.681 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2667 2023-10-18T09:19:01.681 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2668 2023-10-18T09:19:01.681 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:19:01.682 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:19:01.682 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:18 2673 2023-10-18T09:19:01.682 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:18 2674 2023-10-18T09:19:01.682 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:19:01.682 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2678 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2679 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2680 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2682 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:19:01.683 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2684 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2685 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2686 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:19:01.684 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2690 2023-10-18T09:19:01.685 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:18 2691 2023-10-18T09:19:01.685 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2692 2023-10-18T09:19:01.685 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2693 2023-10-18T09:19:01.685 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2696 2023-10-18T09:19:01.685 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:19:01.686 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2701 2023-10-18T09:19:01.686 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2702 2023-10-18T09:19:01.686 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2703 2023-10-18T09:19:01.686 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2704 2023-10-18T09:19:01.686 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:19:01.687 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2753_fsm 2023-10-18T09:19:01.687 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:19:01.687 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2754 2023-10-18T09:19:01.687 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2755 2023-10-18T09:19:01.687 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:19:01.688 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:19:01.688 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830 2023-10-18T09:19:01.688 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830_vm 2023-10-18T09:19:01.688 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2831 2023-10-18T09:19:01.688 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:19:01.689 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832_vm 2023-10-18T09:19:01.689 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:19:01.689 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834 2023-10-18T09:19:01.689 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:19:01.689 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:19:01.690 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836 2023-10-18T09:19:01.690 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:19:01.690 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:19:01.690 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:18 2838 2023-10-18T09:19:01.690 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:19:01.691 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:19:01.691 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:19:01.691 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840 2023-10-18T09:19:01.691 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840_fsm 2023-10-18T09:19:01.691 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2841 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995_vm 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079 2023-10-18T09:19:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:19:01.693 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079_vm 2023-10-18T09:19:01.693 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3080 2023-10-18T09:19:01.693 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3081 2023-10-18T09:19:01.693 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3085 2023-10-18T09:19:01.693 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3119 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3164 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256_vm 2023-10-18T09:19:01.694 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:19:01.695 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:19:01.695 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:19:01.695 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350_vm 2023-10-18T09:19:01.695 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:19:01.696 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:19:01.696 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3380 2023-10-18T09:19:01.696 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381 2023-10-18T09:19:01.696 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:19:01.696 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3394 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3394_vm 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:19:01.697 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3440 2023-10-18T09:19:01.698 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 3455 2023-10-18T09:19:01.698 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:19:01.698 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:19:01.698 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:19:01.699 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466 2023-10-18T09:19:01.699 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:19:01.699 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3467 2023-10-18T09:19:01.699 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:19:01.699 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501_vm 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3502 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3534 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:19:01.700 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3541_fsm 2023-10-18T09:19:01.701 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541_vm 2023-10-18T09:19:01.701 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3542 2023-10-18T09:19:01.701 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3574 2023-10-18T09:19:01.701 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3575 2023-10-18T09:19:01.701 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576 2023-10-18T09:19:01.702 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:19:01.702 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:19:01.702 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:19:01.702 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3597 2023-10-18T09:19:01.702 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:19:01.703 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:19:01.703 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:19:01.703 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:19:01.703 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:19:01.703 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:19:01.704 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:19:01.704 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:19:01.704 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601_vm 2023-10-18T09:19:01.704 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:19:01.704 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3602_fsm 2023-10-18T09:19:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602_vm 2023-10-18T09:19:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3603 2023-10-18T09:19:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3603_fsm 2023-10-18T09:19:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3603_vm 2023-10-18T09:19:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3604 2023-10-18T09:19:01.706 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:19:01.706 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3606 2023-10-18T09:19:01.706 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3607 2023-10-18T09:19:01.706 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:19:01.706 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:19:01.707 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3712 2023-10-18T09:19:01.707 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764 2023-10-18T09:19:01.707 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:19:01.707 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764_vm 2023-10-18T09:19:01.707 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3997 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 548 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 549 2023-10-18T09:19:01.708 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:19:01.709 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102_vm 2023-10-18T09:19:01.709 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:19:01.709 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104_vm 2023-10-18T09:19:01.709 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106 2023-10-18T09:19:01.709 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106_vm 2023-10-18T09:19:01.710 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6110 2023-10-18T09:19:01.710 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6111 2023-10-18T09:19:01.710 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6112 2023-10-18T09:19:01.710 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:19:01.710 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:19:01.711 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:19:01.711 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826_vm 2023-10-18T09:19:01.711 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:19:01.711 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:19:01.711 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:19:01.712 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:19:01.712 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.712 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-18T09:19:01.712 INFO:teuthology.orchestra.run.smithi061.stdout:total 7193 2023-10-18T09:19:01.712 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:19 . 2023-10-18T09:19:01.713 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 .. 2023-10-18T09:19:01.713 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:19:01.713 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 113 2023-10-18T09:19:01.713 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:19:01.713 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12182_fsm 2023-10-18T09:19:01.714 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:19:01.714 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12184 2023-10-18T09:19:01.714 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:19:01.714 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:19:01.714 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12187_fsm 2023-10-18T09:19:01.715 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12187_vm 2023-10-18T09:19:01.715 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12189 2023-10-18T09:19:01.715 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12191 2023-10-18T09:19:01.715 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:19:01.715 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12192_fsm 2023-10-18T09:19:01.716 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:19:01.716 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:19:01.716 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:19:01.716 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:19:01.716 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12197_fsm 2023-10-18T09:19:01.717 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12197_vm 2023-10-18T09:19:01.717 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12199 2023-10-18T09:19:01.717 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:19:01.717 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:19:01.717 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:19:01.718 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:19:01.718 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:19:01.718 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12206 2023-10-18T09:19:01.718 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12207 2023-10-18T09:19:01.718 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12207_fsm 2023-10-18T09:19:01.719 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:19:01.719 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:19:01.719 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:19:01.719 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:19:01.719 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12216 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:19 1247 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1247_fsm 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1247_vm 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:19 1249 2023-10-18T09:19:01.720 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1249_fsm 2023-10-18T09:19:01.721 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1249_vm 2023-10-18T09:19:01.721 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:19:01.721 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1255_fsm 2023-10-18T09:19:01.721 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:19:01.721 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:19 1259 2023-10-18T09:19:01.722 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1259_fsm 2023-10-18T09:19:01.722 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1259_vm 2023-10-18T09:19:01.722 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:19:01.722 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1417_vm 2023-10-18T09:19:01.722 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1418 2023-10-18T09:19:01.723 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1418_vm 2023-10-18T09:19:01.723 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:19:01.723 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 175 2023-10-18T09:19:01.723 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2187 2023-10-18T09:19:01.723 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2328 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:19:01.724 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2337 2023-10-18T09:19:01.725 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:19:01.725 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2600_fsm 2023-10-18T09:19:01.725 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:19:01.725 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:19:01.725 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2601_fsm 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2602_fsm 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2602_vm 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2603 2023-10-18T09:19:01.726 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2603_fsm 2023-10-18T09:19:01.727 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:19:01.727 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:19:01.727 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:19:01.727 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:19:01.727 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2605_fsm 2023-10-18T09:19:01.728 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2605_vm 2023-10-18T09:19:01.728 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2606 2023-10-18T09:19:01.728 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:19:01.728 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2606_vm 2023-10-18T09:19:01.728 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:19:01.729 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:19:01.729 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:19:01.729 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 18 09:19 2608 2023-10-18T09:19:01.729 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2608_fsm 2023-10-18T09:19:01.729 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2608_vm 2023-10-18T09:19:01.730 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:19:01.730 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:19:01.730 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:19:01.730 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2610 2023-10-18T09:19:01.730 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610_fsm 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2610_vm 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2611_vm 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:19:01.731 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:19:01.732 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2613 2023-10-18T09:19:01.732 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2613_vm 2023-10-18T09:19:01.732 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:19:01.732 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:19:01.732 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:19:01.733 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:19:01.733 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:19:01.733 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:19:01.733 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:19:01.733 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2617_fsm 2023-10-18T09:19:01.734 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:19:01.734 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:19 2618 2023-10-18T09:19:01.734 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2618_fsm 2023-10-18T09:19:01.734 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2618_vm 2023-10-18T09:19:01.734 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:18 2619 2023-10-18T09:19:01.735 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2619_fsm 2023-10-18T09:19:01.735 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2619_vm 2023-10-18T09:19:01.735 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:19:01.735 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2620_vm 2023-10-18T09:19:01.735 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:19:01.736 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2651 2023-10-18T09:19:01.736 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2652 2023-10-18T09:19:01.736 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2653 2023-10-18T09:19:01.736 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2654 2023-10-18T09:19:01.736 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2655 2023-10-18T09:19:01.737 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:19:01.737 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:19:01.737 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 18 09:18 2658 2023-10-18T09:19:01.737 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:19 2659 2023-10-18T09:19:01.737 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:19:01.738 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:19:01.738 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2662 2023-10-18T09:19:01.738 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:19 2663 2023-10-18T09:19:01.738 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2664 2023-10-18T09:19:01.739 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2665 2023-10-18T09:19:01.739 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2666 2023-10-18T09:19:01.739 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2667 2023-10-18T09:19:01.739 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2668 2023-10-18T09:19:01.739 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:19:01.740 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:19:01.740 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:19 2673 2023-10-18T09:19:01.740 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:19 2674 2023-10-18T09:19:01.740 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:19:01.740 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2678 2023-10-18T09:19:01.741 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2679 2023-10-18T09:19:01.741 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2680 2023-10-18T09:19:01.741 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:19:01.741 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2682 2023-10-18T09:19:01.741 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:19:01.742 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2684 2023-10-18T09:19:01.742 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2685 2023-10-18T09:19:01.742 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2686 2023-10-18T09:19:01.742 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:19:01.742 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:19:01.743 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:19:01.743 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:19 2690 2023-10-18T09:19:01.743 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:19 2691 2023-10-18T09:19:01.743 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2692 2023-10-18T09:19:01.743 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2693 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2696 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2701 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2702 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2703 2023-10-18T09:19:01.744 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2704 2023-10-18T09:19:01.745 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:19:01.745 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2753_fsm 2023-10-18T09:19:01.745 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:19:01.745 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2754 2023-10-18T09:19:01.745 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2755 2023-10-18T09:19:01.746 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:19:01.746 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:19:01.746 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2830 2023-10-18T09:19:01.746 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2830_vm 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2831 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2832_vm 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2834 2023-10-18T09:19:01.747 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:19:01.748 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:19:01.748 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2836 2023-10-18T09:19:01.748 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:19:01.748 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:19:01.748 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:19 2838 2023-10-18T09:19:01.749 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:19:01.749 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:19:01.749 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:19:01.749 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2840 2023-10-18T09:19:01.749 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2840_fsm 2023-10-18T09:19:01.750 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:19:01.750 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2841 2023-10-18T09:19:01.750 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:19:01.750 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2995_vm 2023-10-18T09:19:01.750 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:19:01.751 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3079 2023-10-18T09:19:01.751 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:19:01.751 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3079_vm 2023-10-18T09:19:01.751 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3080 2023-10-18T09:19:01.751 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3081 2023-10-18T09:19:01.752 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3085 2023-10-18T09:19:01.752 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3118 2023-10-18T09:19:01.752 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:19:01.752 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3119 2023-10-18T09:19:01.752 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3164 2023-10-18T09:19:01.753 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3256 2023-10-18T09:19:01.753 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3256_vm 2023-10-18T09:19:01.753 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:19:01.753 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:19:01.753 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3350_vm 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3380 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3381 2023-10-18T09:19:01.754 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:19:01.755 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3394 2023-10-18T09:19:01.755 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:19:01.755 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3394_vm 2023-10-18T09:19:01.755 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3439 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3440 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 3455 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:19:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:19:01.757 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:19:01.757 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3466 2023-10-18T09:19:01.757 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:19:01.757 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3467 2023-10-18T09:19:01.757 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:19:01.758 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:19:01.758 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3501_vm 2023-10-18T09:19:01.758 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3502 2023-10-18T09:19:01.758 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:19:01.758 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3534 2023-10-18T09:19:01.759 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:19:01.759 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3541_fsm 2023-10-18T09:19:01.759 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3541_vm 2023-10-18T09:19:01.759 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3542 2023-10-18T09:19:01.759 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3574 2023-10-18T09:19:01.760 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3575 2023-10-18T09:19:01.760 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3576 2023-10-18T09:19:01.760 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:19:01.760 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:19:01.760 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:19:01.761 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3597 2023-10-18T09:19:01.761 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:19:01.761 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:19:01.761 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:19:01.761 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:19:01.762 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:19:01.762 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:19:01.762 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:19:01.762 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3601_vm 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3602_fsm 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3602_vm 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3603 2023-10-18T09:19:01.763 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3603_fsm 2023-10-18T09:19:01.764 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3603_vm 2023-10-18T09:19:01.764 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3604 2023-10-18T09:19:01.764 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:19:01.764 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3606 2023-10-18T09:19:01.764 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3607 2023-10-18T09:19:01.765 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:19:01.765 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:19:01.765 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3712 2023-10-18T09:19:01.765 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3764 2023-10-18T09:19:01.765 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:19:01.766 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3764_vm 2023-10-18T09:19:01.766 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:19:01.766 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:19:01.766 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3997 2023-10-18T09:19:01.766 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:19:01.767 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 548 2023-10-18T09:19:01.767 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 549 2023-10-18T09:19:01.767 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:19:01.767 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6102_vm 2023-10-18T09:19:01.767 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:19:01.768 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6104_vm 2023-10-18T09:19:01.768 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6106 2023-10-18T09:19:01.768 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6106_vm 2023-10-18T09:19:01.768 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6110 2023-10-18T09:19:01.768 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6111 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6112 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 826_vm 2023-10-18T09:19:01.769 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:19:01.770 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:19:01.770 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:19 pg_filenode.map 2023-10-18T09:19:01.770 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 PG_VERSION 2023-10-18T09:19:01.770 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.770 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/global: 2023-10-18T09:19:01.771 INFO:teuthology.orchestra.run.smithi061.stdout:total 553 2023-10-18T09:19:01.771 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Oct 18 09:18 . 2023-10-18T09:19:01.771 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.771 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1136 2023-10-18T09:19:01.771 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1136_fsm 2023-10-18T09:19:01.772 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1136_vm 2023-10-18T09:19:01.772 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1137 2023-10-18T09:19:01.772 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1213 2023-10-18T09:19:01.772 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1213_fsm 2023-10-18T09:19:01.772 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1213_vm 2023-10-18T09:19:01.773 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1214 2023-10-18T09:19:01.773 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1214_fsm 2023-10-18T09:19:01.773 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1214_vm 2023-10-18T09:19:01.773 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1232 2023-10-18T09:19:01.773 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1233 2023-10-18T09:19:01.774 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1260 2023-10-18T09:19:01.774 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1260_fsm 2023-10-18T09:19:01.774 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1260_vm 2023-10-18T09:19:01.774 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1261 2023-10-18T09:19:01.774 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1261_fsm 2023-10-18T09:19:01.775 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1261_vm 2023-10-18T09:19:01.775 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 1262 2023-10-18T09:19:01.775 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1262_fsm 2023-10-18T09:19:01.775 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1262_vm 2023-10-18T09:19:01.775 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2396 2023-10-18T09:19:01.776 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2396_fsm 2023-10-18T09:19:01.776 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2396_vm 2023-10-18T09:19:01.776 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2397 2023-10-18T09:19:01.776 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2671 2023-10-18T09:19:01.776 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2672 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2676 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2677 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2694 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2695 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2697 2023-10-18T09:19:01.777 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2698 2023-10-18T09:19:01.778 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2846 2023-10-18T09:19:01.778 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2846_vm 2023-10-18T09:19:01.778 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2847 2023-10-18T09:19:01.778 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2964 2023-10-18T09:19:01.778 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2964_vm 2023-10-18T09:19:01.779 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2965 2023-10-18T09:19:01.779 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2966 2023-10-18T09:19:01.779 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2966_vm 2023-10-18T09:19:01.779 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2967 2023-10-18T09:19:01.779 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3592 2023-10-18T09:19:01.780 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3592_vm 2023-10-18T09:19:01.780 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3593 2023-10-18T09:19:01.780 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 4060 2023-10-18T09:19:01.780 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 4060_vm 2023-10-18T09:19:01.780 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 4061 2023-10-18T09:19:01.781 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6000 2023-10-18T09:19:01.781 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6000_vm 2023-10-18T09:19:01.781 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6001 2023-10-18T09:19:01.781 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6002 2023-10-18T09:19:01.781 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6100 2023-10-18T09:19:01.782 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6100_vm 2023-10-18T09:19:01.782 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6114 2023-10-18T09:19:01.782 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6115 2023-10-18T09:19:01.782 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 pg_control 2023-10-18T09:19:01.782 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:19:01.783 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.783 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/log: 2023-10-18T09:19:01.783 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.783 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 . 2023-10-18T09:19:01.783 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.784 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.785 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-18T09:19:01.785 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.785 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.785 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.785 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-18T09:19:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:total 1 2023-10-18T09:19:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 . 2023-10-18T09:19:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 mappings 2023-10-18T09:19:01.787 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 18 09:19 replorigin_checkpoint 2023-10-18T09:19:01.787 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 snapshots 2023-10-18T09:19:01.787 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.787 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-18T09:19:01.787 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.788 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.788 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 .. 2023-10-18T09:19:01.788 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.788 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-18T09:19:01.788 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.789 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.789 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 .. 2023-10-18T09:19:01.789 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.789 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 . 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 members 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 offsets 2023-10-18T09:19:01.790 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.791 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-18T09:19:01.791 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:19:01.791 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.791 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 .. 2023-10-18T09:19:01.791 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 0000 2023-10-18T09:19:01.792 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.792 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-18T09:19:01.792 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:19:01.792 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.792 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 .. 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 0000 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.793 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.794 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 0000 2023-10-18T09:19:01.794 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.794 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-18T09:19:01.794 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.794 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.795 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.795 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.795 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-18T09:19:01.795 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.795 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.796 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.797 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.797 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-18T09:19:01.797 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.797 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.797 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.798 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.798 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-18T09:19:01.798 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.798 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.798 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.799 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 0000 2023-10-18T09:19:01.800 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.800 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-18T09:19:01.800 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.800 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.800 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.801 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.802 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-18T09:19:01.802 INFO:teuthology.orchestra.run.smithi061.stdout:total 16384 2023-10-18T09:19:01.802 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 . 2023-10-18T09:19:01.802 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.802 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:19 000000010000000000000001 2023-10-18T09:19:01.803 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 archive_status 2023-10-18T09:19:01.803 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.803 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-18T09:19:01.803 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:19:01.803 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 .. 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:19:01.804 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Oct 18 09:19 .. 2023-10-18T09:19:01.805 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 0000 2023-10-18T09:19:01.805 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-10-18T09:19:02.045 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2023-10-18T09:19:02.134 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:01 smithi061 ceph-mon[98573]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2023-10-18T09:19:02.135 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:01 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:02.163 INFO:teuthology.orchestra.run.smithi061.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-10-18T09:19:02.172 INFO:teuthology.orchestra.run.smithi061.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-10-18T09:19:02.177 INFO:teuthology.orchestra.run.smithi061.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-10-18T09:19:02.183 INFO:teuthology.orchestra.run.smithi061.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-10-18T09:19:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:01 smithi110 ceph-mon[134818]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2023-10-18T09:19:02.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:01 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:02.213 INFO:teuthology.orchestra.run.smithi061.stderr:creating tables... 2023-10-18T09:19:02.324 INFO:teuthology.orchestra.run.smithi061.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.11 s, remaining 3.37 s) 2023-10-18T09:19:02.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:01 smithi170 ceph-mon[135150]: pgmap v372: 129 pgs: 129 active+clean; 23 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.9 MiB/s wr, 33 op/s 2023-10-18T09:19:02.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:01 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:02.666 INFO:teuthology.orchestra.run.smithi061.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.45 s, remaining 6.75 s) 2023-10-18T09:19:02.979 INFO:teuthology.orchestra.run.smithi061.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.76 s, remaining 7.38 s) 2023-10-18T09:19:03.167 INFO:teuthology.orchestra.run.smithi061.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.95 s, remaining 6.66 s) 2023-10-18T09:19:03.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:02 smithi110 ceph-mon[134818]: pgmap v373: 129 pgs: 129 active+clean; 28 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.4 MiB/s wr, 60 op/s 2023-10-18T09:19:03.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:02 smithi061 ceph-mon[98573]: pgmap v373: 129 pgs: 129 active+clean; 28 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.4 MiB/s wr, 60 op/s 2023-10-18T09:19:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:02 smithi170 ceph-mon[135150]: pgmap v373: 129 pgs: 129 active+clean; 28 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.4 MiB/s wr, 60 op/s 2023-10-18T09:19:03.493 INFO:teuthology.orchestra.run.smithi061.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.28 s, remaining 6.90 s) 2023-10-18T09:19:03.841 INFO:teuthology.orchestra.run.smithi061.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.63 s, remaining 7.04 s) 2023-10-18T09:19:03.948 INFO:teuthology.orchestra.run.smithi061.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.73 s, remaining 6.19 s) 2023-10-18T09:19:04.331 INFO:teuthology.orchestra.run.smithi061.stderr:800000 of 3200000 tuples (25%) done (elapsed 2.12 s, remaining 6.35 s) 2023-10-18T09:19:04.618 INFO:teuthology.orchestra.run.smithi061.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.40 s, remaining 6.14 s) 2023-10-18T09:19:04.729 INFO:teuthology.orchestra.run.smithi061.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.51 s, remaining 5.53 s) 2023-10-18T09:19:05.164 INFO:teuthology.orchestra.run.smithi061.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.95 s, remaining 5.63 s) 2023-10-18T09:19:05.296 INFO:teuthology.orchestra.run.smithi061.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.08 s, remaining 5.13 s) 2023-10-18T09:19:05.630 INFO:teuthology.orchestra.run.smithi061.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.41 s, remaining 4.99 s) 2023-10-18T09:19:05.931 INFO:teuthology.orchestra.run.smithi061.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.72 s, remaining 4.78 s) 2023-10-18T09:19:06.087 INFO:teuthology.orchestra.run.smithi061.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.87 s, remaining 4.39 s) 2023-10-18T09:19:06.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:05 smithi110 ceph-mon[134818]: pgmap v374: 129 pgs: 129 active+clean; 63 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.8 MiB/s wr, 94 op/s 2023-10-18T09:19:06.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:05 smithi061 ceph-mon[98573]: pgmap v374: 129 pgs: 129 active+clean; 63 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.8 MiB/s wr, 94 op/s 2023-10-18T09:19:06.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:05 smithi170 ceph-mon[135150]: pgmap v374: 129 pgs: 129 active+clean; 63 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.8 MiB/s wr, 94 op/s 2023-10-18T09:19:06.419 INFO:teuthology.orchestra.run.smithi061.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.20 s, remaining 4.20 s) 2023-10-18T09:19:06.630 INFO:teuthology.orchestra.run.smithi061.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.41 s, remaining 3.90 s) 2023-10-18T09:19:06.986 INFO:teuthology.orchestra.run.smithi061.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.77 s, remaining 3.71 s) 2023-10-18T09:19:07.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:06 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:07.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:06 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:07.333 INFO:teuthology.orchestra.run.smithi061.stderr:1900000 of 3200000 tuples (59%) done (elapsed 5.12 s, remaining 3.50 s) 2023-10-18T09:19:07.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:06 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:07.560 INFO:teuthology.orchestra.run.smithi061.stderr:2000000 of 3200000 tuples (62%) done (elapsed 5.34 s, remaining 3.21 s) 2023-10-18T09:19:07.852 INFO:teuthology.orchestra.run.smithi061.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.64 s, remaining 2.95 s) 2023-10-18T09:19:08.143 INFO:teuthology.orchestra.run.smithi061.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.93 s, remaining 2.69 s) 2023-10-18T09:19:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:07 smithi110 ceph-mon[134818]: pgmap v375: 129 pgs: 129 active+clean; 123 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 17 MiB/s wr, 155 op/s 2023-10-18T09:19:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:07 smithi061 ceph-mon[98573]: pgmap v375: 129 pgs: 129 active+clean; 123 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 17 MiB/s wr, 155 op/s 2023-10-18T09:19:08.349 INFO:teuthology.orchestra.run.smithi061.stderr:2300000 of 3200000 tuples (71%) done (elapsed 6.13 s, remaining 2.40 s) 2023-10-18T09:19:08.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:07 smithi170 ceph-mon[135150]: pgmap v375: 129 pgs: 129 active+clean; 123 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 17 MiB/s wr, 155 op/s 2023-10-18T09:19:08.612 INFO:teuthology.orchestra.run.smithi061.stderr:2400000 of 3200000 tuples (75%) done (elapsed 6.40 s, remaining 2.13 s) 2023-10-18T09:19:08.964 INFO:teuthology.orchestra.run.smithi061.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.75 s, remaining 1.89 s) 2023-10-18T09:19:09.150 INFO:teuthology.orchestra.run.smithi061.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.94 s, remaining 1.60 s) 2023-10-18T09:19:09.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:08 smithi110 ceph-mon[134818]: pgmap v376: 129 pgs: 129 active+clean; 163 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 22 MiB/s wr, 144 op/s 2023-10-18T09:19:09.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:08 smithi061 ceph-mon[98573]: pgmap v376: 129 pgs: 129 active+clean; 163 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 22 MiB/s wr, 144 op/s 2023-10-18T09:19:09.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:08 smithi170 ceph-mon[135150]: pgmap v376: 129 pgs: 129 active+clean; 163 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 22 MiB/s wr, 144 op/s 2023-10-18T09:19:09.425 INFO:teuthology.orchestra.run.smithi061.stderr:2700000 of 3200000 tuples (84%) done (elapsed 7.21 s, remaining 1.34 s) 2023-10-18T09:19:09.757 INFO:teuthology.orchestra.run.smithi061.stderr:2800000 of 3200000 tuples (87%) done (elapsed 7.54 s, remaining 1.08 s) 2023-10-18T09:19:09.943 INFO:teuthology.orchestra.run.smithi061.stderr:2900000 of 3200000 tuples (90%) done (elapsed 7.73 s, remaining 0.80 s) 2023-10-18T09:19:10.234 INFO:teuthology.orchestra.run.smithi061.stderr:3000000 of 3200000 tuples (93%) done (elapsed 8.02 s, remaining 0.53 s) 2023-10-18T09:19:10.564 INFO:teuthology.orchestra.run.smithi061.stderr:3100000 of 3200000 tuples (96%) done (elapsed 8.35 s, remaining 0.27 s) 2023-10-18T09:19:10.674 INFO:teuthology.orchestra.run.smithi061.stderr:3200000 of 3200000 tuples (100%) done (elapsed 8.46 s, remaining 0.00 s) 2023-10-18T09:19:10.745 INFO:teuthology.orchestra.run.smithi061.stderr:vacuum... 2023-10-18T09:19:11.764 INFO:teuthology.orchestra.run.smithi061.stderr:set primary keys... 2023-10-18T09:19:12.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:11 smithi110 ceph-mon[134818]: pgmap v377: 129 pgs: 129 active+clean; 279 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 42 MiB/s wr, 158 op/s 2023-10-18T09:19:12.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:11 smithi061 ceph-mon[98573]: pgmap v377: 129 pgs: 129 active+clean; 279 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 42 MiB/s wr, 158 op/s 2023-10-18T09:19:12.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:11 smithi170 ceph-mon[135150]: pgmap v377: 129 pgs: 129 active+clean; 279 MiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 42 MiB/s wr, 158 op/s 2023-10-18T09:19:13.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:12 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:13.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:12 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:12 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:14.185 INFO:teuthology.orchestra.run.smithi061.stderr:done. 2023-10-18T09:19:14.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-10-18T09:19:14.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:13 smithi110 ceph-mon[134818]: pgmap v378: 129 pgs: 129 active+clean; 327 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 50 MiB/s wr, 162 op/s 2023-10-18T09:19:14.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:13 smithi061 ceph-mon[98573]: pgmap v378: 129 pgs: 129 active+clean; 327 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 50 MiB/s wr, 162 op/s 2023-10-18T09:19:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:starting vacuum...end. 2023-10-18T09:19:14.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:13 smithi170 ceph-mon[135150]: pgmap v378: 129 pgs: 129 active+clean; 327 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 50 MiB/s wr, 162 op/s 2023-10-18T09:19:15.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:14 smithi110 ceph-mon[134818]: pgmap v379: 129 pgs: 129 active+clean; 416 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 63 MiB/s wr, 148 op/s 2023-10-18T09:19:15.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:14 smithi061 ceph-mon[98573]: pgmap v379: 129 pgs: 129 active+clean; 416 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 63 MiB/s wr, 148 op/s 2023-10-18T09:19:15.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:14 smithi170 ceph-mon[135150]: pgmap v379: 129 pgs: 129 active+clean; 416 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 63 MiB/s wr, 148 op/s 2023-10-18T09:19:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:17 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:17 smithi170 ceph-mon[135150]: pgmap v380: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 64 MiB/s wr, 135 op/s 2023-10-18T09:19:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:17 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:17 smithi110 ceph-mon[134818]: pgmap v380: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 64 MiB/s wr, 135 op/s 2023-10-18T09:19:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:17 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:19:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:17 smithi061 ceph-mon[98573]: pgmap v380: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 64 MiB/s wr, 135 op/s 2023-10-18T09:19:18.795 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:19:19.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.142+0000 7fe002dff700 1 -- 172.21.15.61:0/662767406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc104560 msgr2=0x7fdffc10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.142+0000 7fe002dff700 1 --2- 172.21.15.61:0/662767406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc104560 0x7fdffc10ae10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdff400b0a0 tx=0x7fdff400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 -- 172.21.15.61:0/662767406 shutdown_connections 2023-10-18T09:19:19.146 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 --2- 172.21.15.61:0/662767406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc104560 0x7fdffc10ae10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 --2- 172.21.15.61:0/662767406 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc1029d0 0x7fdffc102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 --2- 172.21.15.61:0/662767406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc103bc0 0x7fdffc104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 -- 172.21.15.61:0/662767406 >> 172.21.15.61:0/662767406 conn(0x7fdffc0fdf40 msgr2=0x7fdffc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 -- 172.21.15.61:0/662767406 shutdown_connections 2023-10-18T09:19:19.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.144+0000 7fe002dff700 1 -- 172.21.15.61:0/662767406 wait complete. 2023-10-18T09:19:19.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 Processor -- start 2023-10-18T09:19:19.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 -- start start 2023-10-18T09:19:19.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 0x7fdffc078550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc103bc0 0x7fdffc078a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc075880 con 0x7fdffc103bc0 2023-10-18T09:19:19.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc0759f0 con 0x7fdffc104560 2023-10-18T09:19:19.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe002dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc075bb0 con 0x7fdffc1029d0 2023-10-18T09:19:19.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe00139c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe00139c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36426/0 (socket says 172.21.15.61:36426) 2023-10-18T09:19:19.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe00139c700 1 -- 172.21.15.61:0/2180716948 learned_addr learned my addr 172.21.15.61:0/2180716948 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fe000b9b700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 0x7fdffc078550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.146+0000 7fdffbfff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc103bc0 0x7fdffc078a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.147+0000 7fe00139c700 1 -- 172.21.15.61:0/2180716948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 msgr2=0x7fdffc078550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.147+0000 7fe00139c700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 0x7fdffc078550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.147+0000 7fe00139c700 1 -- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc103bc0 msgr2=0x7fdffc078a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.147+0000 7fe00139c700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc103bc0 0x7fdffc078a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.147+0000 7fe00139c700 1 -- 172.21.15.61:0/2180716948 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4009d00 con 0x7fdffc104560 2023-10-18T09:19:19.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.149+0000 7fe000b9b700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 0x7fdffc078550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:19.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.150+0000 7fe00139c700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdff4007ae0 tx=0x7fdff4004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.150+0000 7fdff9ffb700 1 -- 172.21.15.61:0/2180716948 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400e040 con 0x7fdffc104560 2023-10-18T09:19:19.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.151+0000 7fdff9ffb700 1 -- 172.21.15.61:0/2180716948 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff400ceb0 con 0x7fdffc104560 2023-10-18T09:19:19.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.151+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc1a4ed0 con 0x7fdffc104560 2023-10-18T09:19:19.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.151+0000 7fdff9ffb700 1 -- 172.21.15.61:0/2180716948 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff401d990 con 0x7fdffc104560 2023-10-18T09:19:19.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.151+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc1a52f0 con 0x7fdffc104560 2023-10-18T09:19:19.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.154+0000 7fdff9ffb700 1 -- 172.21.15.61:0/2180716948 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdff401dbf0 con 0x7fdffc104560 2023-10-18T09:19:19.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.158+0000 7fe016711b80 1 Processor -- start 2023-10-18T09:19:19.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.158+0000 7fe016711b80 1 -- start start 2023-10-18T09:19:19.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.159+0000 7fdff9ffb700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 0x7fdfec0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.159+0000 7fdff9ffb700 1 -- 172.21.15.61:0/2180716948 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdff40ce720 con 0x7fdffc104560 2023-10-18T09:19:19.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f7f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f7ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07ef16e0 con 0x563d07f7fc30 2023-10-18T09:19:19.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07ed7c60 con 0x563d07f7e5d0 2023-10-18T09:19:19.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.160+0000 7fe016711b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07ed5580 con 0x563d07f7f2d0 2023-10-18T09:19:19.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.161+0000 7fdffbfff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 0x7fdfec0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.161+0000 7fdffbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.161+0000 7fdffbfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:58840/0 (socket says 172.21.15.61:58840) 2023-10-18T09:19:19.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.161+0000 7fdffbfff700 1 -- 172.21.15.61:0/1969689492 learned_addr learned my addr 172.21.15.61:0/1969689492 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.162+0000 7fdffbfff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 0x7fdfec0a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80125e0 tx=0x7fdfe80128f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.162+0000 7fdffbfff700 1 -- 172.21.15.61:0/1969689492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 msgr2=0x563d07f7f6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:19:19.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.162+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f7f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.162+0000 7fdffbfff700 1 -- 172.21.15.61:0/1969689492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 msgr2=0x563d07f7ea50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:19.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.163+0000 7fe00139c700 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f7ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.163+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f7ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.163+0000 7fdffbfff700 1 -- 172.21.15.61:0/1969689492 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d07e8e3f0 con 0x563d07f7fc30 2023-10-18T09:19:19.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.163+0000 7fe00139c700 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f7ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:19.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.163+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdfe801d640 tx=0x7fdfe801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d2c66f9e7bc830 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.164+0000 7fdfe6ffd700 1 -- 172.21.15.61:0/1969689492 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8004220 con 0x563d07f7fc30 2023-10-18T09:19:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.164+0000 7fdfe6ffd700 1 -- 172.21.15.61:0/1969689492 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe8004380 con 0x563d07f7fc30 2023-10-18T09:19:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.164+0000 7fdfe6ffd700 1 -- 172.21.15.61:0/1969689492 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80108a0 con 0x563d07f7fc30 2023-10-18T09:19:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 -- 172.21.15.61:0/1969689492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 msgr2=0x563d07f7e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdfe801d640 tx=0x7fdfe801c000 comp rx=0 tx=0).stop 2023-10-18T09:19:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 -- 172.21.15.61:0/1969689492 shutdown_connections 2023-10-18T09:19:19.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f7ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f7f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 --2- 172.21.15.61:0/1969689492 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f7e090 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 -- 172.21.15.61:0/1969689492 >> 172.21.15.61:0/1969689492 conn(0x563d07ef2720 msgr2=0x563d07f78b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 -- 172.21.15.61:0/1969689492 shutdown_connections 2023-10-18T09:19:19.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.165+0000 7fe016711b80 1 -- 172.21.15.61:0/1969689492 wait complete. 2023-10-18T09:19:19.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.166+0000 7fe016711b80 1 -- start start 2023-10-18T09:19:19.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 Processor -- start 2023-10-18T09:19:19.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 0x563d07f87aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f8f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07ed0e70 con 0x563d07f7fc30 2023-10-18T09:19:19.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07e6d710 con 0x563d07f7f2d0 2023-10-18T09:19:19.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07e6d3b0 con 0x563d07f7e5d0 2023-10-18T09:19:19.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.167+0000 7fe016711b80 10 client.0 _ll_register_callbacks cb 0x563d07db7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:19.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fe000b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 0x563d07f87aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fe000b9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 0x563d07f87aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57670/0 (socket says 172.21.15.61:57670) 2023-10-18T09:19:19.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fe000b9b700 1 -- 172.21.15.61:0/4160951576 learned_addr learned my addr 172.21.15.61:0/4160951576 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fdffbfff700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fdffbfff700 1 -- 172.21.15.61:0/4160951576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 msgr2=0x563d07f87aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:19.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fdffbfff700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 0x563d07f87aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.168+0000 7fdffbfff700 1 -- 172.21.15.61:0/4160951576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 msgr2=0x563d07f8f020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:19.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.169+0000 7fe00139c700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f8f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.169+0000 7fdffbfff700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f8f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.169+0000 7fdffbfff700 1 -- 172.21.15.61:0/4160951576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe801d400 con 0x563d07f7f2d0 2023-10-18T09:19:19.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.170+0000 7fdffbfff700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f87fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b6c0 tx=0x7fdfe8004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf8ac179ff57a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.170+0000 7fdfdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:19.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8003cd0 con 0x563d07f7f2d0 2023-10-18T09:19:19.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d07ed5ef0 con 0x563d07f7f2d0 2023-10-18T09:19:19.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdfe8005040 con 0x563d07f7f2d0 2023-10-18T09:19:19.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe8005200 con 0x563d07f7f2d0 2023-10-18T09:19:19.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fe016711b80 10 client.25721 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:19.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.171+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563d07cfbde0 con 0x563d07f7f2d0 2023-10-18T09:19:19.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d07ed5ef0 con 0x563d07f7f2d0 2023-10-18T09:19:19.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdfe80105a0 con 0x563d07f7f2d0 2023-10-18T09:19:19.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 10 client.25721 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:19.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:19.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 20 client.25721 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:19.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07f93770 0x563d07f95b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 4 client.25721 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:19.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d07f4d8b0 con 0x563d07f93770 2023-10-18T09:19:19.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.172+0000 7fe000b9b700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07f93770 0x563d07f95b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.173+0000 7fe000b9b700 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07f93770 0x563d07f95b90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af88beabb737950 server_cookie=1b6e4987dd82371a in_seq=0 out_seq=0 2023-10-18T09:19:19.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.173+0000 7fdfdbfff700 10 client.25721 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.174+0000 7fdfdbfff700 1 -- 172.21.15.61:0/4160951576 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d07f4d8b0 con 0x563d07f93770 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.174+0000 7fdfdbfff700 10 client.25721 handle_command_reply: tid=0 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.175+0000 7fe016711b80 1 client.25721 shutdown 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.175+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 msgr2=0x563d07f87fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.175+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f87fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b6c0 tx=0x7fdfe8004560 comp rx=0 tx=0).stop 2023-10-18T09:19:19.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 shutdown_connections 2023-10-18T09:19:19.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07f93770 0x563d07f95b90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7e5d0 0x563d07f87aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 --2- 172.21.15.61:0/4160951576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f87fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 >> 172.21.15.61:0/4160951576 conn(0x563d07ef2720 msgr2=0x563d07f7d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.176+0000 7fdfdbfff700 0 client.25721 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:19.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.177+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 shutdown_connections 2023-10-18T09:19:19.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.177+0000 7fe016711b80 1 -- 172.21.15.61:0/4160951576 wait complete. 2023-10-18T09:19:19.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.177+0000 7fe016711b80 20 client.25721 trim_cache size 0 max 16384 2023-10-18T09:19:19.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 Processor -- start 2023-10-18T09:19:19.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 -- start start 2023-10-18T09:19:19.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07f73af0 con 0x563d07f7fc30 2023-10-18T09:19:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07d51920 con 0x563d07f7e5d0 2023-10-18T09:19:19.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.208+0000 7fe016711b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07f790c0 con 0x563d07f7f2d0 2023-10-18T09:19:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36448/0 (socket says 172.21.15.61:36448) 2023-10-18T09:19:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 -- 172.21.15.61:0/857056026 learned_addr learned my addr 172.21.15.61:0/857056026 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe00139c700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fdffbfff700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 -- 172.21.15.61:0/857056026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 msgr2=0x563d07f8efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f8efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 -- 172.21.15.61:0/857056026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 msgr2=0x563d07f822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f822d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.209+0000 7fe000b9b700 1 -- 172.21.15.61:0/857056026 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d07ce4c60 con 0x563d07f7e5d0 2023-10-18T09:19:19.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.210+0000 7fe00139c700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.211+0000 7fe000b9b700 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x563d07f96400 tx=0x7fdff000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c774b766e199d435 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.211+0000 7fdfe77fe700 1 -- 172.21.15.61:0/857056026 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0017040 con 0x563d07f7e5d0 2023-10-18T09:19:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.212+0000 7fdfe77fe700 1 -- 172.21.15.61:0/857056026 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff001d070 con 0x563d07f7e5d0 2023-10-18T09:19:19.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.212+0000 7fdfe77fe700 1 -- 172.21.15.61:0/857056026 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0017040 con 0x563d07f7e5d0 2023-10-18T09:19:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.213+0000 7fe016711b80 1 -- 172.21.15.61:0/857056026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 msgr2=0x563d07f86770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.214+0000 7fe016711b80 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x563d07f96400 tx=0x7fdff000b410 comp rx=0 tx=0).stop 2023-10-18T09:19:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 -- 172.21.15.61:0/857056026 shutdown_connections 2023-10-18T09:19:19.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7fc30 0x563d07f822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7e5d0 0x563d07f86770 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 --2- 172.21.15.61:0/857056026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7f2d0 0x563d07f8efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 -- 172.21.15.61:0/857056026 >> 172.21.15.61:0/857056026 conn(0x563d07ef2720 msgr2=0x563d07f86fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 -- 172.21.15.61:0/857056026 shutdown_connections 2023-10-18T09:19:19.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.215+0000 7fe016711b80 1 -- 172.21.15.61:0/857056026 wait complete. 2023-10-18T09:19:19.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.216+0000 7fe016711b80 1 -- start start 2023-10-18T09:19:19.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.216+0000 7fe016711b80 1 Processor -- start 2023-10-18T09:19:19.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7e5d0 0x563d07f8dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07f8e920 con 0x563d07f7e5d0 2023-10-18T09:19:19.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07f8ea90 con 0x563d07f7f2d0 2023-10-18T09:19:19.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d07fe7b10 con 0x563d07f7fc30 2023-10-18T09:19:19.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe016711b80 10 client.0 _ll_register_callbacks cb 0x563d07db7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:19.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe00139c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe00139c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57692/0 (socket says 172.21.15.61:57692) 2023-10-18T09:19:19.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe00139c700 1 -- 172.21.15.61:0/1462802649 learned_addr learned my addr 172.21.15.61:0/1462802649 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fe000b9b700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7e5d0 0x563d07f8dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.217+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 -- 172.21.15.61:0/1462802649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 msgr2=0x563d07fe7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:19.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 -- 172.21.15.61:0/1462802649 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7e5d0 msgr2=0x563d07f8dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:19.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7e5d0 0x563d07f8dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 -- 172.21.15.61:0/1462802649 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff0004040 con 0x563d07f7f2d0 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fe00139c700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdffbfff700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f8e100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800baa0 tx=0x7fdfe8010d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305043e809f40deb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdfdbfff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8021030 con 0x563d07f7f2d0 2023-10-18T09:19:19.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.218+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdfe8005440 con 0x563d07f7f2d0 2023-10-18T09:19:19.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.219+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfe8004380 con 0x563d07f7f2d0 2023-10-18T09:19:19.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.219+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfe80109e0 con 0x563d07f7f2d0 2023-10-18T09:19:19.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.220+0000 7fe016711b80 10 client.25727 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:19.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.220+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563d07d22250 con 0x563d07f7f2d0 2023-10-18T09:19:19.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.220+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfe8004380 con 0x563d07f7f2d0 2023-10-18T09:19:19.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.220+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdfe8042070 con 0x563d07f7f2d0 2023-10-18T09:19:19.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 10 client.25727 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:19.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:19.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 20 client.25727 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:19.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07fe92d0 0x563d07feb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 4 client.25727 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:19:19.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.221+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563d07ec9e50 con 0x563d07fe92d0 2023-10-18T09:19:19.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.224+0000 7fe000b9b700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07fe92d0 0x563d07feb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.225+0000 7fe000b9b700 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07fe92d0 0x563d07feb6f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31920d7d911fd4ec server_cookie=6f319e8f887cfe48 in_seq=0 out_seq=0 2023-10-18T09:19:19.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.225+0000 7fdfdbfff700 10 client.25727 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:19.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.225+0000 7fdfdbfff700 1 -- 172.21.15.61:0/1462802649 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563d07ec9e50 con 0x563d07fe92d0 2023-10-18T09:19:19.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fdfdbfff700 10 client.25727 handle_command_reply: tid=0 2023-10-18T09:19:19.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 client.25727 shutdown 2023-10-18T09:19:19.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 msgr2=0x563d07f8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f8e100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800baa0 tx=0x7fdfe8010d80 comp rx=0 tx=0).stop 2023-10-18T09:19:19.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 shutdown_connections 2023-10-18T09:19:19.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x563d07fe92d0 0x563d07feb6f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563d07f7fc30 0x563d07fe7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x563d07f7e5d0 0x563d07f8dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 --2- 172.21.15.61:0/1462802649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d07f7f2d0 0x563d07f8e100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 >> 172.21.15.61:0/1462802649 conn(0x563d07ef2720 msgr2=0x563d07f86770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fdfdbfff700 0 client.25727 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:19.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.226+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 shutdown_connections 2023-10-18T09:19:19.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe016711b80 1 -- 172.21.15.61:0/1462802649 wait complete. 2023-10-18T09:19:19.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe016711b80 20 client.25727 trim_cache size 0 max 16384 2023-10-18T09:19:19.235 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:19.235 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "no active scrubs running", 2023-10-18T09:19:19.235 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": {} 2023-10-18T09:19:19.235 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:19.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 msgr2=0x7fdfec0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 0x7fdfec0a39c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80125e0 tx=0x7fdfe80128f0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 msgr2=0x7fdffc0750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdff4007ae0 tx=0x7fdff4004c80 comp rx=0 tx=0).stop 2023-10-18T09:19:19.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 shutdown_connections 2023-10-18T09:19:19.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdfec0a1500 0x7fdfec0a39c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdffc104560 0x7fdffc0750e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdffc1029d0 0x7fdffc078550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 --2- 172.21.15.61:0/2180716948 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdffc103bc0 0x7fdffc078a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.227+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 >> 172.21.15.61:0/2180716948 conn(0x7fdffc0fdf40 msgr2=0x7fdffc106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.228+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 shutdown_connections 2023-10-18T09:19:19.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.228+0000 7fe002dff700 1 -- 172.21.15.61:0/2180716948 wait complete. 2023-10-18T09:19:19.261 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T09:19:19.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:19:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 5.0 s, 1694.0 tps, lat 56.169 ms stddev 70.546 2023-10-18T09:19:19.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.572+0000 7f113c5ff700 1 -- 172.21.15.61:0/3252869220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134106d70 msgr2=0x7f1134107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.572+0000 7f113c5ff700 1 --2- 172.21.15.61:0/3252869220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134106d70 0x7f1134107190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f112c009990 tx=0x7f112c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 -- 172.21.15.61:0/3252869220 shutdown_connections 2023-10-18T09:19:19.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 --2- 172.21.15.61:0/3252869220 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134108900 0x7f113410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 --2- 172.21.15.61:0/3252869220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134106d70 0x7f1134107190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 --2- 172.21.15.61:0/3252869220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134107f60 0x7f11341083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 -- 172.21.15.61:0/3252869220 >> 172.21.15.61:0/3252869220 conn(0x7f1134075d20 msgr2=0x7f1134078140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:19.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 -- 172.21.15.61:0/3252869220 shutdown_connections 2023-10-18T09:19:19.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.574+0000 7f113c5ff700 1 -- 172.21.15.61:0/3252869220 wait complete. 2023-10-18T09:19:19.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.577+0000 7f113c5ff700 1 Processor -- start 2023-10-18T09:19:19.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.578+0000 7f113c5ff700 1 -- start start 2023-10-18T09:19:19.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 0x7f113419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134107f60 0x7f113419d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4820 con 0x7f1134106d70 2023-10-18T09:19:19.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4990 con 0x7f1134108900 2023-10-18T09:19:19.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.579+0000 7f113c5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11341a4b50 con 0x7f1134107f60 2023-10-18T09:19:19.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113ab9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113ab9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36478/0 (socket says 172.21.15.61:36478) 2023-10-18T09:19:19.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113ab9c700 1 -- 172.21.15.61:0/1652682928 learned_addr learned my addr 172.21.15.61:0/1652682928 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:19.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f1139b9a700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134107f60 0x7f113419d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 0x7f113419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 -- 172.21.15.61:0/1652682928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134107f60 msgr2=0x7f113419d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134107f60 0x7f113419d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 -- 172.21.15.61:0/1652682928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 msgr2=0x7f11341a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.580+0000 7f113a39b700 1 -- 172.21.15.61:0/1652682928 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0095f0 con 0x7f1134106d70 2023-10-18T09:19:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f113ab9c700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f113a39b700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 0x7f113419cd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f112c0098e0 tx=0x7f112c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c01d070 con 0x7f1134106d70 2023-10-18T09:19:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f112c003e40 con 0x7f1134106d70 2023-10-18T09:19:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11341a4dd0 con 0x7f1134106d70 2023-10-18T09:19:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c00fd60 con 0x7f1134106d70 2023-10-18T09:19:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.581+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11341a51a0 con 0x7f1134106d70 2023-10-18T09:19:19.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.583+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113410e1f0 con 0x7f1134106d70 2023-10-18T09:19:19.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.585+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f112c00f460 con 0x7f1134106d70 2023-10-18T09:19:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.587+0000 7f11337fe700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 0x7f111c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.588+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f112c0ce1e0 con 0x7f1134106d70 2023-10-18T09:19:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.588+0000 7f1139b9a700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 0x7f111c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.589+0000 7f1139b9a700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 0x7f111c0a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1128009550 tx=0x7f11280091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:19.590+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f112c097520 con 0x7f1134106d70 2023-10-18T09:19:20.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.019+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1134067530 con 0x7f1134106d70 2023-10-18T09:19:20.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.022+0000 7f11337fe700 1 -- 172.21.15.61:0/1652682928 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11395 (secure 0 0 0) 0x7f112c09ab40 con 0x7f1134106d70 2023-10-18T09:19:20.025 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:20.025 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:19:16.512081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:19:20.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.026+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 msgr2=0x7f111c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.026+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 0x7f111c0a39c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1128009550 tx=0x7f11280091b0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.026+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 msgr2=0x7f113419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.026+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 0x7f113419cd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f112c0098e0 tx=0x7f112c003b90 comp rx=0 tx=0).stop 2023-10-18T09:19:20.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 shutdown_connections 2023-10-18T09:19:20.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f111c0a1500 0x7f111c0a39c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1134108900 0x7f11341a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1134106d70 0x7f113419cd50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 --2- 172.21.15.61:0/1652682928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1134107f60 0x7f113419d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 >> 172.21.15.61:0/1652682928 conn(0x7f1134075d20 msgr2=0x7f113410b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 shutdown_connections 2023-10-18T09:19:20.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.027+0000 7f113c5ff700 1 -- 172.21.15.61:0/1652682928 wait complete. 2023-10-18T09:19:20.034 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 32 2023-10-18T09:19:20.063 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-18T09:19:20.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:19 smithi110 ceph-mon[134818]: pgmap v381: 129 pgs: 129 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 61 MiB/s wr, 103 op/s 2023-10-18T09:19:20.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:19 smithi061 ceph-mon[98573]: pgmap v381: 129 pgs: 129 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 61 MiB/s wr, 103 op/s 2023-10-18T09:19:20.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 -- 172.21.15.61:0/356968067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220103a40 msgr2=0x7fb220105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 --2- 172.21.15.61:0/356968067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220103a40 0x7fb220105e30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb214009990 tx=0x7fb214009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 -- 172.21.15.61:0/356968067 shutdown_connections 2023-10-18T09:19:20.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 --2- 172.21.15.61:0/356968067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220106370 0x7fb220108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 --2- 172.21.15.61:0/356968067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220101110 0x7fb220103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 --2- 172.21.15.61:0/356968067 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220103a40 0x7fb220105e30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 -- 172.21.15.61:0/356968067 >> 172.21.15.61:0/356968067 conn(0x7fb2200faa80 msgr2=0x7fb2200fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 -- 172.21.15.61:0/356968067 shutdown_connections 2023-10-18T09:19:20.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.378+0000 7fb21ffff700 1 -- 172.21.15.61:0/356968067 wait complete. 2023-10-18T09:19:20.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.379+0000 7fb21ffff700 1 Processor -- start 2023-10-18T09:19:20.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.379+0000 7fb21ffff700 1 -- start start 2023-10-18T09:19:20.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 0x7fb2201987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 0x7fb22019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2201a0310 con 0x7fb220101110 2023-10-18T09:19:20.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2201a0450 con 0x7fb220106370 2023-10-18T09:19:20.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2201a05e0 con 0x7fb220103a40 2023-10-18T09:19:20.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21effd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 0x7fb2201987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57728/0 (socket says 172.21.15.61:57728) 2023-10-18T09:19:20.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21effd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 0x7fb2201987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:58906/0 (socket says 172.21.15.61:58906) 2023-10-18T09:19:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1027410527 learned_addr learned my addr 172.21.15.61:0/1027410527 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.380+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 0x7fb22019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1027410527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 msgr2=0x7fb22019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 0x7fb22019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 msgr2=0x7fb2201987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 0x7fb2201987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1027410527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2140095f0 con 0x7fb220103a40 2023-10-18T09:19:20.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 0x7fb22019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:20.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb214005910 tx=0x7fb214009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb207fff700 1 -- 172.21.15.61:0/1027410527 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21401d070 con 0x7fb220103a40 2023-10-18T09:19:20.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb207fff700 1 -- 172.21.15.61:0/1027410527 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb214021410 con 0x7fb220103a40 2023-10-18T09:19:20.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2201a0860 con 0x7fb220103a40 2023-10-18T09:19:20.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.381+0000 7fb207fff700 1 -- 172.21.15.61:0/1027410527 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214005720 con 0x7fb220103a40 2023-10-18T09:19:20.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.382+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2201a0ce0 con 0x7fb220103a40 2023-10-18T09:19:20.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.386+0000 7fb207fff700 1 -- 172.21.15.61:0/1027410527 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb214021580 con 0x7fb220103a40 2023-10-18T09:19:20.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.388+0000 7fb207fff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 0x7fb2080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.388+0000 7fb207fff700 1 -- 172.21.15.61:0/1027410527 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb2140ce330 con 0x7fb220103a40 2023-10-18T09:19:20.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.388+0000 7fb21effd700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 0x7fb2080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:19 smithi170 ceph-mon[135150]: pgmap v381: 129 pgs: 129 active+clean; 608 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 61 MiB/s wr, 103 op/s 2023-10-18T09:19:20.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.389+0000 7fb238dc1b80 1 Processor -- start 2023-10-18T09:19:20.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.389+0000 7fb21effd700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 0x7fb2080a39c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb210009c60 tx=0x7fb210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 -- start start 2023-10-18T09:19:20.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bb98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bba7f0 0x560d24bbac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24b2c6e0 con 0x560d24bb94d0 2023-10-18T09:19:20.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24b12c60 con 0x560d24bb9e30 2023-10-18T09:19:20.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.390+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24b10580 con 0x560d24bba7f0 2023-10-18T09:19:20.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bba7f0 0x560d24bbac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36490/0 (socket says 172.21.15.61:36490) 2023-10-18T09:19:20.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1915695891 learned_addr learned my addr 172.21.15.61:0/1915695891 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21effd700 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bb98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1915695891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bba7f0 msgr2=0x560d24bbac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bba7f0 0x560d24bbac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1915695891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 msgr2=0x560d24bb98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bb98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 -- 172.21.15.61:0/1915695891 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d24ac93f0 con 0x560d24bb9e30 2023-10-18T09:19:20.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21effd700 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bb98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:20.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.391+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb214006010 tx=0x7fb214021c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50448d50484d370 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.392+0000 7fb2057fa700 1 -- 172.21.15.61:0/1915695891 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140ce170 con 0x560d24bb9e30 2023-10-18T09:19:20.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.392+0000 7fb2057fa700 1 -- 172.21.15.61:0/1915695891 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21409b390 con 0x560d24bb9e30 2023-10-18T09:19:20.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.392+0000 7fb2057fa700 1 -- 172.21.15.61:0/1915695891 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21409b850 con 0x560d24bb9e30 2023-10-18T09:19:20.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 -- 172.21.15.61:0/1915695891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 msgr2=0x560d24bba2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb214006010 tx=0x7fb214021c40 comp rx=0 tx=0).stop 2023-10-18T09:19:20.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 -- 172.21.15.61:0/1915695891 shutdown_connections 2023-10-18T09:19:20.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bba7f0 0x560d24bbac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bb98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/1915695891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bba2b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 -- 172.21.15.61:0/1915695891 >> 172.21.15.61:0/1915695891 conn(0x560d24b2d720 msgr2=0x560d24bb3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 -- 172.21.15.61:0/1915695891 shutdown_connections 2023-10-18T09:19:20.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.393+0000 7fb238dc1b80 1 -- 172.21.15.61:0/1915695891 wait complete. 2023-10-18T09:19:20.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.394+0000 7fb238dc1b80 1 -- start start 2023-10-18T09:19:20.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.394+0000 7fb238dc1b80 1 Processor -- start 2023-10-18T09:19:20.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24b0be70 con 0x560d24bba7f0 2023-10-18T09:19:20.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24aa8710 con 0x560d24bb94d0 2023-10-18T09:19:20.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24aa83b0 con 0x560d24bb9e30 2023-10-18T09:19:20.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb238dc1b80 10 client.0 _ll_register_callbacks cb 0x560d249f2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:20.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36500/0 (socket says 172.21.15.61:36500) 2023-10-18T09:19:20.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 -- 172.21.15.61:0/3082810728 learned_addr learned my addr 172.21.15.61:0/3082810728 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 -- 172.21.15.61:0/3082810728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 msgr2=0x560d24bc2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 -- 172.21.15.61:0/3082810728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 msgr2=0x560d24bc9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21effd700 1 -- 172.21.15.61:0/3082810728 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2201a0ce0 con 0x560d24bb94d0 2023-10-18T09:19:20.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.395+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:20.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb21effd700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb210003980 tx=0x7fb2100047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a104d8f2066d3e4f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb202ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:20.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21001e8b0 con 0x560d24bb94d0 2023-10-18T09:19:20.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb21001ea10 con 0x560d24bb94d0 2023-10-18T09:19:20.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d24b10ef0 con 0x560d24bb94d0 2023-10-18T09:19:20.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb21001ebd0 con 0x560d24bb94d0 2023-10-18T09:19:20.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb238dc1b80 10 client.25745 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:20.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.396+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560d24936de0 con 0x560d24bb94d0 2023-10-18T09:19:20.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d24b10ef0 con 0x560d24bb94d0 2023-10-18T09:19:20.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb21001c430 con 0x560d24bb94d0 2023-10-18T09:19:20.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 10 client.25745 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:20.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 10 client.25745 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:19:20.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 20 client.25745 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:20.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24bce710 0x560d24bd0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 4 client.25745 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:20.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d24b888b0 con 0x560d24bce710 2023-10-18T09:19:20.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.397+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24bce710 0x560d24bd0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.398+0000 7fb202ff5700 10 client.25745 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:20.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.398+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24bce710 0x560d24bd0b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea1a550a25247528 server_cookie=98940fe3def0abf5 in_seq=0 out_seq=0 2023-10-18T09:19:20.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.398+0000 7fb202ff5700 1 -- 172.21.15.61:0/3082810728 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560d24b888b0 con 0x560d24bce710 2023-10-18T09:19:20.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.398+0000 7fb202ff5700 10 client.25745 handle_command_reply: tid=0 2023-10-18T09:19:20.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.398+0000 7fb238dc1b80 1 client.25745 shutdown 2023-10-18T09:19:20.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 msgr2=0x560d24bc2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb210003980 tx=0x7fb2100047c0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 shutdown_connections 2023-10-18T09:19:20.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24bce710 0x560d24bd0b30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb94d0 0x560d24bc2880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/3082810728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 >> 172.21.15.61:0/3082810728 conn(0x560d24b2d720 msgr2=0x560d24bb82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb202ff5700 0 client.25745 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:20.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 shutdown_connections 2023-10-18T09:19:20.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 1 -- 172.21.15.61:0/3082810728 wait complete. 2023-10-18T09:19:20.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.399+0000 7fb238dc1b80 20 client.25745 trim_cache size 0 max 16384 2023-10-18T09:19:20.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.413+0000 7fb238dc1b80 1 Processor -- start 2023-10-18T09:19:20.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.413+0000 7fb238dc1b80 1 -- start start 2023-10-18T09:19:20.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d2498c920 con 0x560d24bb94d0 2023-10-18T09:19:20.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24bc3a00 con 0x560d24bba7f0 2023-10-18T09:19:20.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24bac2a0 con 0x560d24bb9e30 2023-10-18T09:19:20.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb21f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36502/0 (socket says 172.21.15.61:36502) 2023-10-18T09:19:20.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.414+0000 7fb21f7fe700 1 -- 172.21.15.61:0/2378354655 learned_addr learned my addr 172.21.15.61:0/2378354655 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.415+0000 7fb21f7fe700 1 -- 172.21.15.61:0/2378354655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 msgr2=0x560d24bc9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.415+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.415+0000 7fb21f7fe700 1 -- 172.21.15.61:0/2378354655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 msgr2=0x560d24bca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.415+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.415+0000 7fb21f7fe700 1 -- 172.21.15.61:0/2378354655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d2491fc60 con 0x560d24bba7f0 2023-10-18T09:19:20.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.416+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2180040c0 tx=0x7fb218009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f59f0edf66157e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.416+0000 7fb205ffb700 1 -- 172.21.15.61:0/2378354655 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218007500 con 0x560d24bba7f0 2023-10-18T09:19:20.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.416+0000 7fb205ffb700 1 -- 172.21.15.61:0/2378354655 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21800f040 con 0x560d24bba7f0 2023-10-18T09:19:20.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.416+0000 7fb205ffb700 1 -- 172.21.15.61:0/2378354655 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21800ead0 con 0x560d24bba7f0 2023-10-18T09:19:20.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 -- 172.21.15.61:0/2378354655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 msgr2=0x560d24bb82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2180040c0 tx=0x7fb218009920 comp rx=0 tx=0).stop 2023-10-18T09:19:20.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 -- 172.21.15.61:0/2378354655 shutdown_connections 2023-10-18T09:19:20.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bba7f0 0x560d24bb82d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bb94d0 0x560d24bca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/2378354655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb9e30 0x560d24bc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 -- 172.21.15.61:0/2378354655 >> 172.21.15.61:0/2378354655 conn(0x560d24b2d720 msgr2=0x560d24bc1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 -- 172.21.15.61:0/2378354655 shutdown_connections 2023-10-18T09:19:20.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.417+0000 7fb238dc1b80 1 -- 172.21.15.61:0/2378354655 wait complete. 2023-10-18T09:19:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.418+0000 7fb238dc1b80 1 -- start start 2023-10-18T09:19:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.418+0000 7fb238dc1b80 1 Processor -- start 2023-10-18T09:19:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bc8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24bc9710 con 0x560d24bba7f0 2023-10-18T09:19:20.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24bc9880 con 0x560d24bb9e30 2023-10-18T09:19:20.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d24bc99f0 con 0x560d24bb94d0 2023-10-18T09:19:20.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb238dc1b80 10 client.0 _ll_register_callbacks cb 0x560d249f2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:20.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb21effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb21effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57762/0 (socket says 172.21.15.61:57762) 2023-10-18T09:19:20.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb21effd700 1 -- 172.21.15.61:0/580428770 learned_addr learned my addr 172.21.15.61:0/580428770 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.419+0000 7fb21f7fe700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 -- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 msgr2=0x560d24bc8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bc8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 -- 172.21.15.61:0/580428770 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 msgr2=0x560d24bc91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 -- 172.21.15.61:0/580428770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218003d10 con 0x560d24bb94d0 2023-10-18T09:19:20.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb21effd700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb210005a80 tx=0x7fb210011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd90ab11735ecae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb202ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:20.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210005240 con 0x560d24bb94d0 2023-10-18T09:19:20.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb21409b390 con 0x560d24bb94d0 2023-10-18T09:19:20.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.420+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb2100053a0 con 0x560d24bb94d0 2023-10-18T09:19:20.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb210005560 con 0x560d24bb94d0 2023-10-18T09:19:20.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 10 client.25771 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:20.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560d2495d250 con 0x560d24bb94d0 2023-10-18T09:19:20.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb21409b390 con 0x560d24bb94d0 2023-10-18T09:19:20.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb210004780 con 0x560d24bb94d0 2023-10-18T09:19:20.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:20.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 10 client.25771 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:19:20.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 20 client.25771 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:20.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24c236c0 0x560d24c25ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 4 client.25771 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:20.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.421+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d24b04e50 con 0x560d24c236c0 2023-10-18T09:19:20.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24c236c0 0x560d24c25ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb21e7fc700 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24c236c0 0x560d24c25ae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74a0f31312eabc5e server_cookie=31f782e11985ae97 in_seq=0 out_seq=0 2023-10-18T09:19:20.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb202ff5700 10 client.25771 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:20.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb202ff5700 1 -- 172.21.15.61:0/580428770 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d24b04e50 con 0x560d24c236c0 2023-10-18T09:19:20.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb202ff5700 10 client.25771 handle_command_reply: tid=0 2023-10-18T09:19:20.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.422+0000 7fb238dc1b80 1 client.25771 shutdown 2023-10-18T09:19:20.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 msgr2=0x560d24bc82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb210005a80 tx=0x7fb210011bf0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 shutdown_connections 2023-10-18T09:19:20.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x560d24c236c0 0x560d24c25ae0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560d24bba7f0 0x560d24bc91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560d24bb94d0 0x560d24bc82e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb202ff5700 0 client.25771 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:20.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 --2- 172.21.15.61:0/580428770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560d24bb9e30 0x560d24bc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 >> 172.21.15.61:0/580428770 conn(0x560d24b2d720 msgr2=0x560d24bb82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 shutdown_connections 2023-10-18T09:19:20.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 1 -- 172.21.15.61:0/580428770 wait complete. 2023-10-18T09:19:20.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.423+0000 7fb238dc1b80 20 client.25771 trim_cache size 0 max 16384 2023-10-18T09:19:20.441 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:20.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 msgr2=0x7fb2080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 0x7fb2080a39c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb210009c60 tx=0x7fb210009260 comp rx=0 tx=0).stop 2023-10-18T09:19:20.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 msgr2=0x7fb220198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb214005910 tx=0x7fb214009580 comp rx=0 tx=0).stop 2023-10-18T09:19:20.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 shutdown_connections 2023-10-18T09:19:20.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb2080a1500 0x7fb2080a39c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb220106370 0x7fb22019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb220101110 0x7fb2201987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 --2- 172.21.15.61:0/1027410527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb220103a40 0x7fb220198d20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 >> 172.21.15.61:0/1027410527 conn(0x7fb2200faa80 msgr2=0x7fb2201009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 shutdown_connections 2023-10-18T09:19:20.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.425+0000 7fb21ffff700 1 -- 172.21.15.61:0/1027410527 wait complete. 2023-10-18T09:19:20.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-18T09:19:20.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.760+0000 7f77c03ff700 1 -- 172.21.15.61:0/4177828343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 msgr2=0x7f77b80ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.760+0000 7f77c03ff700 1 --2- 172.21.15.61:0/4177828343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b80ffe60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009990 tx=0x7f77a8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.761+0000 7f77c03ff700 1 -- 172.21.15.61:0/4177828343 shutdown_connections 2023-10-18T09:19:20.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.761+0000 7f77c03ff700 1 --2- 172.21.15.61:0/4177828343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b8100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.761+0000 7f77c03ff700 1 --2- 172.21.15.61:0/4177828343 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77b80ff080 0x7f77b80ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.761+0000 7f77c03ff700 1 --2- 172.21.15.61:0/4177828343 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b80ffe60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.761+0000 7f77c03ff700 1 -- 172.21.15.61:0/4177828343 >> 172.21.15.61:0/4177828343 conn(0x7f77b80fac00 msgr2=0x7f77b80fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.762+0000 7f77c03ff700 1 -- 172.21.15.61:0/4177828343 shutdown_connections 2023-10-18T09:19:20.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.762+0000 7f77c03ff700 1 -- 172.21.15.61:0/4177828343 wait complete. 2023-10-18T09:19:20.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.764+0000 7f77c03ff700 1 Processor -- start 2023-10-18T09:19:20.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.764+0000 7f77c03ff700 1 -- start start 2023-10-18T09:19:20.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77b80ff080 0x7f77b819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b81a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a4810 con 0x7f77b80ff080 2023-10-18T09:19:20.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a4980 con 0x7f77b80ff9e0 2023-10-18T09:19:20.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77c03ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81a4b40 con 0x7f77b81003a0 2023-10-18T09:19:20.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77be99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b81a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77be99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b81a4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57778/0 (socket says 172.21.15.61:57778) 2023-10-18T09:19:20.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77be99c700 1 -- 172.21.15.61:0/3504700982 learned_addr learned my addr 172.21.15.61:0/3504700982 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.765+0000 7f77bd99a700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.766+0000 7f77bd99a700 1 -- 172.21.15.61:0/3504700982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 msgr2=0x7f77b81a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.766+0000 7f77bd99a700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b81a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.766+0000 7f77bd99a700 1 -- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77b80ff080 msgr2=0x7f77b819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.766+0000 7f77bd99a700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77b80ff080 0x7f77b819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.771 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.766+0000 7f77bd99a700 1 -- 172.21.15.61:0/3504700982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80095f0 con 0x7f77b80ff9e0 2023-10-18T09:19:20.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.767+0000 7f77bd99a700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b819d230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77a8000c00 tx=0x7f77a8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.768+0000 7f77af7fe700 1 -- 172.21.15.61:0/3504700982 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a801d070 con 0x7f77b80ff9e0 2023-10-18T09:19:20.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.768+0000 7f77af7fe700 1 -- 172.21.15.61:0/3504700982 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a8021410 con 0x7f77b80ff9e0 2023-10-18T09:19:20.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.768+0000 7f77af7fe700 1 -- 172.21.15.61:0/3504700982 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8017650 con 0x7f77b80ff9e0 2023-10-18T09:19:20.772 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.768+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b81a4dc0 con 0x7f77b80ff9e0 2023-10-18T09:19:20.773 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.768+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b81a5190 con 0x7f77b80ff9e0 2023-10-18T09:19:20.777 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.774+0000 7f77af7fe700 1 -- 172.21.15.61:0/3504700982 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f77a800fa90 con 0x7f77b80ff9e0 2023-10-18T09:19:20.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.776+0000 7f77af7fe700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 0x7f77a40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.779 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.777+0000 7f77af7fe700 1 -- 172.21.15.61:0/3504700982 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f77a80ce2f0 con 0x7f77b80ff9e0 2023-10-18T09:19:20.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.777+0000 7f77d3d38b80 1 Processor -- start 2023-10-18T09:19:20.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.777+0000 7f77be19b700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 0x7f77a40a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.777+0000 7f77d3d38b80 1 -- start start 2023-10-18T09:19:20.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668edc30 0x55c6668ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66685f6e0 con 0x55c6668ed2d0 2023-10-18T09:19:20.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c666845c60 con 0x55c6668ec5d0 2023-10-18T09:19:20.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c666843580 con 0x55c6668edc30 2023-10-18T09:19:20.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77be99c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77be99c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36542/0 (socket says 172.21.15.61:36542) 2023-10-18T09:19:20.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77be99c700 1 -- 172.21.15.61:0/3738696426 learned_addr learned my addr 172.21.15.61:0/3738696426 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77bd99a700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668edc30 0x55c6668ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77be19b700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.778+0000 7f77be99c700 1 -- 172.21.15.61:0/3738696426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668edc30 msgr2=0x55c6668ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be99c700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668edc30 0x55c6668ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be99c700 1 -- 172.21.15.61:0/3738696426 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 msgr2=0x55c6668ed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be99c700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668ed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be99c700 1 -- 172.21.15.61:0/3738696426 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6667fc3f0 con 0x55c6668ec5d0 2023-10-18T09:19:20.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be19b700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:20.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be99c700 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77b000bde0 tx=0x7f77b00106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330e776aa94abbf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77be19b700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 0x7f77a40a3a10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f77b81022e0 tx=0x7f77b400e910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77acbfe700 1 -- 172.21.15.61:0/3738696426 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0010ec0 con 0x55c6668ec5d0 2023-10-18T09:19:20.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77acbfe700 1 -- 172.21.15.61:0/3738696426 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b0015070 con 0x55c6668ec5d0 2023-10-18T09:19:20.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.779+0000 7f77acbfe700 1 -- 172.21.15.61:0/3738696426 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b0011440 con 0x55c6668ec5d0 2023-10-18T09:19:20.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 -- 172.21.15.61:0/3738696426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 msgr2=0x55c6668eca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77b000bde0 tx=0x7f77b00106b0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 -- 172.21.15.61:0/3738696426 shutdown_connections 2023-10-18T09:19:20.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ec5d0 0x55c6668eca50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668ed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/3738696426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668edc30 0x55c6668ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.781+0000 7f77d3d38b80 1 -- 172.21.15.61:0/3738696426 >> 172.21.15.61:0/3738696426 conn(0x55c666860720 msgr2=0x55c6668e6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.782+0000 7f77d3d38b80 1 -- 172.21.15.61:0/3738696426 shutdown_connections 2023-10-18T09:19:20.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.782+0000 7f77d3d38b80 1 -- 172.21.15.61:0/3738696426 wait complete. 2023-10-18T09:19:20.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.783+0000 7f77d3d38b80 1 -- start start 2023-10-18T09:19:20.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.783+0000 7f77d3d38b80 1 Processor -- start 2023-10-18T09:19:20.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.784+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ec5d0 0x55c6668f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66683ee70 con 0x55c6668ec5d0 2023-10-18T09:19:20.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6667db710 con 0x55c6668edc30 2023-10-18T09:19:20.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6667db3b0 con 0x55c6668ed2d0 2023-10-18T09:19:20.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77d3d38b80 10 client.0 _ll_register_callbacks cb 0x55c666725a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:20.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77bd99a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.785+0000 7f77bd99a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57796/0 (socket says 172.21.15.61:57796) 2023-10-18T09:19:20.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 -- 172.21.15.61:0/575336525 learned_addr learned my addr 172.21.15.61:0/575336525 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77be99c700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77be19b700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ec5d0 0x55c6668f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 -- 172.21.15.61:0/575336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 msgr2=0x55c6668fd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668fd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 -- 172.21.15.61:0/575336525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ec5d0 msgr2=0x55c6668f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ec5d0 0x55c6668f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.786+0000 7f77bd99a700 1 -- 172.21.15.61:0/575336525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b000ba10 con 0x55c6668ed2d0 2023-10-18T09:19:20.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77bd99a700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b620 tx=0x7f77a8021b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d72e8e3b06618e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:20.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a80a3650 con 0x55c6668ed2d0 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f77a80b2060 con 0x55c6668ed2d0 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c666843ef0 con 0x55c6668ed2d0 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a80ce070 con 0x55c6668ed2d0 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.787+0000 7f77d3d38b80 10 client.25783 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c666669de0 con 0x55c6668ed2d0 2023-10-18T09:19:20.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77a80b1020 con 0x55c6668ed2d0 2023-10-18T09:19:20.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77a809f120 con 0x55c6668ed2d0 2023-10-18T09:19:20.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:20.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 10 client.25783 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:19:20.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 20 client.25783 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:20.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c666901490 0x55c6669038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 4 client.25783 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:20.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.788+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c666843ef0 con 0x55c666901490 2023-10-18T09:19:20.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.789+0000 7f77be19b700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c666901490 0x55c6669038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.789+0000 7f77a1ffb700 10 client.25783 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:20.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.789+0000 7f77be19b700 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c666901490 0x55c6669038b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52f821c3750228b7 server_cookie=f294f2fa576836e0 in_seq=0 out_seq=0 2023-10-18T09:19:20.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.790+0000 7f77a1ffb700 1 -- 172.21.15.61:0/575336525 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c666843ef0 con 0x55c666901490 2023-10-18T09:19:20.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.790+0000 7f77a1ffb700 10 client.25783 handle_command_reply: tid=0 2023-10-18T09:19:20.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.790+0000 7f77d3d38b80 1 client.25783 shutdown 2023-10-18T09:19:20.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.790+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 msgr2=0x55c6668f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.790+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b620 tx=0x7f77a8021b40 comp rx=0 tx=0).stop 2023-10-18T09:19:20.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 shutdown_connections 2023-10-18T09:19:20.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ec5d0 0x55c6668f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c666901490 0x55c6669038b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/575336525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ed2d0 0x55c6668f5ff0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 >> 172.21.15.61:0/575336525 conn(0x55c666860720 msgr2=0x55c6668e6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77a1ffb700 0 client.25783 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:20.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 shutdown_connections 2023-10-18T09:19:20.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 1 -- 172.21.15.61:0/575336525 wait complete. 2023-10-18T09:19:20.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.791+0000 7f77d3d38b80 20 client.25783 trim_cache size 0 max 16384 2023-10-18T09:19:20.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.814+0000 7f77d3d38b80 1 Processor -- start 2023-10-18T09:19:20.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.814+0000 7f77d3d38b80 1 -- start start 2023-10-18T09:19:20.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ed2d0 0x55c6668fcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 0x55c6668f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6668f4b10 con 0x55c6668edc30 2023-10-18T09:19:20.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6666bf920 con 0x55c6668ed2d0 2023-10-18T09:19:20.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6668fe1b0 con 0x55c6668ec5d0 2023-10-18T09:19:20.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77be19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77be19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57808/0 (socket says 172.21.15.61:57808) 2023-10-18T09:19:20.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77be19b700 1 -- 172.21.15.61:0/1786674507 learned_addr learned my addr 172.21.15.61:0/1786674507 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.815+0000 7f77bd99a700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ed2d0 0x55c6668fcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be99c700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 0x55c6668f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 -- 172.21.15.61:0/1786674507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ed2d0 msgr2=0x55c6668fcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ed2d0 0x55c6668fcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 -- 172.21.15.61:0/1786674507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 msgr2=0x55c6668f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 0x55c6668f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 -- 172.21.15.61:0/1786674507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c666652c60 con 0x55c6668ec5d0 2023-10-18T09:19:20.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be99c700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 0x55c6668f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:20.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.816+0000 7f77be19b700 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009690 tx=0x7f77b4019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95684a1c8e146689 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.817+0000 7f77ad3ff700 1 -- 172.21.15.61:0/1786674507 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40167c0 con 0x55c6668ec5d0 2023-10-18T09:19:20.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.817+0000 7f77ad3ff700 1 -- 172.21.15.61:0/1786674507 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b400f040 con 0x55c6668ec5d0 2023-10-18T09:19:20.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.817+0000 7f77ad3ff700 1 -- 172.21.15.61:0/1786674507 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4018b30 con 0x55c6668ec5d0 2023-10-18T09:19:20.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 -- 172.21.15.61:0/1786674507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 msgr2=0x55c6668ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009690 tx=0x7f77b4019000 comp rx=0 tx=0).stop 2023-10-18T09:19:20.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 -- 172.21.15.61:0/1786674507 shutdown_connections 2023-10-18T09:19:20.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668edc30 0x55c6668f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668ec090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/1786674507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668ed2d0 0x55c6668fcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.818+0000 7f77d3d38b80 1 -- 172.21.15.61:0/1786674507 >> 172.21.15.61:0/1786674507 conn(0x55c666860720 msgr2=0x55c666908250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.819+0000 7f77d3d38b80 1 -- 172.21.15.61:0/1786674507 shutdown_connections 2023-10-18T09:19:20.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.819+0000 7f77d3d38b80 1 -- 172.21.15.61:0/1786674507 wait complete. 2023-10-18T09:19:20.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.820+0000 7f77d3d38b80 1 -- start start 2023-10-18T09:19:20.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.820+0000 7f77d3d38b80 1 Processor -- start 2023-10-18T09:19:20.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.822+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668fc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.822+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.822+0000 7f77d3d38b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6668f98a0 con 0x55c6668ed2d0 2023-10-18T09:19:20.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6668f9a10 con 0x55c6668edc30 2023-10-18T09:19:20.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77d3d38b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6668f9bd0 con 0x55c6668ec5d0 2023-10-18T09:19:20.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77be99c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77be19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668fc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77be99c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36562/0 (socket says 172.21.15.61:36562) 2023-10-18T09:19:20.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.823+0000 7f77be99c700 1 -- 172.21.15.61:0/4168078163 learned_addr learned my addr 172.21.15.61:0/4168078163 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:20.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77d3d38b80 10 client.0 _ll_register_callbacks cb 0x55c666725a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:20.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77bd99a700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77be19b700 1 -- 172.21.15.61:0/4168078163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 msgr2=0x55c6668f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77be19b700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77be19b700 1 -- 172.21.15.61:0/4168078163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 msgr2=0x55c6668fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:20.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77be19b700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77be19b700 1 -- 172.21.15.61:0/4168078163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b4016400 con 0x55c6668ec5d0 2023-10-18T09:19:20.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.824+0000 7f77bd99a700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:20.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.825+0000 7f77be19b700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668fc7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009690 tx=0x7f77b4004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e60ff972ad4770b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:20.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.825+0000 7f77a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:20.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4004db0 con 0x55c6668ec5d0 2023-10-18T09:19:20.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f77b4022570 con 0x55c6668ec5d0 2023-10-18T09:19:20.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77b0015070 con 0x55c6668ec5d0 2023-10-18T09:19:20.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77b4018ac0 con 0x55c6668ec5d0 2023-10-18T09:19:20.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77d3d38b80 10 client.25789 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:20.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.826+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c666690250 con 0x55c6668ec5d0 2023-10-18T09:19:20.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77b0015070 con 0x55c6668ec5d0 2023-10-18T09:19:20.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f77b4036020 con 0x55c6668ec5d0 2023-10-18T09:19:20.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:20.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 10 client.25789 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:19:20.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 20 client.25789 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:20.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c6669565f0 0x55c666958a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:20.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 4 client.25789 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:20.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.827+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6668bb8b0 con 0x55c6669565f0 2023-10-18T09:19:20.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.828+0000 7f77bd99a700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c6669565f0 0x55c666958a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:20.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.828+0000 7f77bd99a700 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c6669565f0 0x55c666958a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2e4c7c3285d3ba5 server_cookie=9fd1c9c30ce5cc7e in_seq=0 out_seq=0 2023-10-18T09:19:20.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.828+0000 7f77a1ffb700 10 client.25789 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:20.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.829+0000 7f77a1ffb700 1 -- 172.21.15.61:0/4168078163 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6668bb8b0 con 0x55c6669565f0 2023-10-18T09:19:20.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.829+0000 7f77a1ffb700 10 client.25789 handle_command_reply: tid=0 2023-10-18T09:19:20.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.829+0000 7f77d3d38b80 1 client.25789 shutdown 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 msgr2=0x55c6668fc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668fc7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009690 tx=0x7f77b4004a70 comp rx=0 tx=0).stop 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 shutdown_connections 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6668edc30 0x55c6668f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x55c6669565f0 0x55c666958a10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6668ec5d0 0x55c6668fc7d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77a1ffb700 0 client.25789 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:20.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 --2- 172.21.15.61:0/4168078163 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6668ed2d0 0x55c6668fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 >> 172.21.15.61:0/4168078163 conn(0x55c666860720 msgr2=0x55c6668f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 shutdown_connections 2023-10-18T09:19:20.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.830+0000 7f77d3d38b80 1 -- 172.21.15.61:0/4168078163 wait complete. 2023-10-18T09:19:20.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.831+0000 7f77d3d38b80 20 client.25789 trim_cache size 0 max 16384 2023-10-18T09:19:20.839 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:20.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 msgr2=0x7f77a40a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 0x7f77a40a3a10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f77b81022e0 tx=0x7f77b400e910 comp rx=0 tx=0).stop 2023-10-18T09:19:20.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 msgr2=0x7f77b819d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:20.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b819d230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f77a8000c00 tx=0x7f77a8005740 comp rx=0 tx=0).stop 2023-10-18T09:19:20.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 shutdown_connections 2023-10-18T09:19:20.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77a40a1550 0x7f77a40a3a10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77b81003a0 0x7f77b81a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77b80ff080 0x7f77b819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 --2- 172.21.15.61:0/3504700982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77b80ff9e0 0x7f77b819d230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:20.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 >> 172.21.15.61:0/3504700982 conn(0x7f77b80fac00 msgr2=0x7f77b80fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:20.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 shutdown_connections 2023-10-18T09:19:20.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:20.832+0000 7f77c03ff700 1 -- 172.21.15.61:0/3504700982 wait complete. 2023-10-18T09:19:20.864 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-18T09:19:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:20 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1652682928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:21.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:20 smithi110 ceph-mon[134818]: pgmap v382: 129 pgs: 129 active+clean; 911 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 66 MiB/s wr, 198 op/s 2023-10-18T09:19:21.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 -- 172.21.15.61:0/2494040521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 msgr2=0x7f61a0109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 --2- 172.21.15.61:0/2494040521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a0109c20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61940098f0 tx=0x7f6194009c00 comp rx=0 tx=0).stop 2023-10-18T09:19:21.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 -- 172.21.15.61:0/2494040521 shutdown_connections 2023-10-18T09:19:21.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 --2- 172.21.15.61:0/2494040521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010cb30 0x7f61a010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 --2- 172.21.15.61:0/2494040521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a0109c20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 --2- 172.21.15.61:0/2494040521 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010a160 0x7f61a010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.235+0000 7f61a6fff700 1 -- 172.21.15.61:0/2494040521 >> 172.21.15.61:0/2494040521 conn(0x7f61a006d7d0 msgr2=0x7f61a006fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.236+0000 7f61a6fff700 1 -- 172.21.15.61:0/2494040521 shutdown_connections 2023-10-18T09:19:21.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.236+0000 7f61a6fff700 1 -- 172.21.15.61:0/2494040521 wait complete. 2023-10-18T09:19:21.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.237+0000 7f61a6fff700 1 Processor -- start 2023-10-18T09:19:21.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.237+0000 7f61a6fff700 1 -- start start 2023-10-18T09:19:21.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.237+0000 7f61a6fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.237+0000 7f61a6fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010a160 0x7f61a01a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a6fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010cb30 0x7f61a01acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a6fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01ad070 con 0x7f61a010cb30 2023-10-18T09:19:21.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a6fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01ad1b0 con 0x7f61a01077f0 2023-10-18T09:19:21.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a6fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01ad340 con 0x7f61a010a160 2023-10-18T09:19:21.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36578/0 (socket says 172.21.15.61:36578) 2023-10-18T09:19:21.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.238+0000 7f61a4d9b700 1 -- 172.21.15.61:0/270615135 learned_addr learned my addr 172.21.15.61:0/270615135 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a559c700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010cb30 0x7f61a01acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f619ffff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010a160 0x7f61a01a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a4d9b700 1 -- 172.21.15.61:0/270615135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010a160 msgr2=0x7f61a01a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010a160 0x7f61a01a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a4d9b700 1 -- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010cb30 msgr2=0x7f61a01acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010cb30 0x7f61a01acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.239+0000 7f61a4d9b700 1 -- 172.21.15.61:0/270615135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009520 con 0x7f61a01077f0 2023-10-18T09:19:21.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.240+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6194009810 tx=0x7f619400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.240+0000 7f619dffb700 1 -- 172.21.15.61:0/270615135 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619401c070 con 0x7f61a01077f0 2023-10-18T09:19:21.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.240+0000 7f619dffb700 1 -- 172.21.15.61:0/270615135 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619400f840 con 0x7f61a01077f0 2023-10-18T09:19:21.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.240+0000 7f619dffb700 1 -- 172.21.15.61:0/270615135 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619401c070 con 0x7f61a01077f0 2023-10-18T09:19:21.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.241+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a01ad5c0 con 0x7f61a01077f0 2023-10-18T09:19:21.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.243+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01ad990 con 0x7f61a01077f0 2023-10-18T09:19:21.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.248+0000 7f619dffb700 1 -- 172.21.15.61:0/270615135 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6194020030 con 0x7f61a01077f0 2023-10-18T09:19:21.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:20 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1652682928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:21.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:20 smithi061 ceph-mon[98573]: pgmap v382: 129 pgs: 129 active+clean; 911 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 66 MiB/s wr, 198 op/s 2023-10-18T09:19:21.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.255+0000 7f619dffb700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 0x7f61900a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.255+0000 7f619dffb700 1 -- 172.21.15.61:0/270615135 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f61940cd8b0 con 0x7f61a01077f0 2023-10-18T09:19:21.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.256+0000 7f619ffff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 0x7f61900a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 Processor -- start 2023-10-18T09:19:21.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 -- start start 2023-10-18T09:19:21.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde75a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bdde86e0 con 0x55a4bde76c30 2023-10-18T09:19:21.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bddcec60 con 0x55a4bde755d0 2023-10-18T09:19:21.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.258+0000 7f61baa16b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bddcc580 con 0x55a4bde762d0 2023-10-18T09:19:21.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.259+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.259+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57840/0 (socket says 172.21.15.61:57840) 2023-10-18T09:19:21.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.259+0000 7f61a4d9b700 1 -- 172.21.15.61:0/3049448715 learned_addr learned my addr 172.21.15.61:0/3049448715 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.259+0000 7f61a559c700 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde75a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 -- 172.21.15.61:0/3049448715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 msgr2=0x55a4bde75a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde75a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 -- 172.21.15.61:0/3049448715 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 msgr2=0x55a4bde75090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:21.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde75090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 -- 172.21.15.61:0/3049448715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4bdd853f0 con 0x55a4bde762d0 2023-10-18T09:19:21.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61940a2680 tx=0x7f619409a4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bb2fa9cf82e1e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f618affd700 1 -- 172.21.15.61:0/3049448715 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409ea00 con 0x55a4bde762d0 2023-10-18T09:19:21.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f618affd700 1 -- 172.21.15.61:0/3049448715 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61940a2bb0 con 0x55a4bde762d0 2023-10-18T09:19:21.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.260+0000 7f618affd700 1 -- 172.21.15.61:0/3049448715 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194096fc0 con 0x55a4bde762d0 2023-10-18T09:19:21.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 -- 172.21.15.61:0/3049448715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 msgr2=0x55a4bde766f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61940a2680 tx=0x7f619409a4e0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f619ffff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 0x7f61900a3aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bf0 tx=0x7f618c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 -- 172.21.15.61:0/3049448715 shutdown_connections 2023-10-18T09:19:21.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde75a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde766f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3049448715 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.261+0000 7f61baa16b80 1 -- 172.21.15.61:0/3049448715 >> 172.21.15.61:0/3049448715 conn(0x55a4bdde9720 msgr2=0x55a4bde6fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.262+0000 7f61baa16b80 1 -- 172.21.15.61:0/3049448715 shutdown_connections 2023-10-18T09:19:21.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.262+0000 7f61baa16b80 1 -- 172.21.15.61:0/3049448715 wait complete. 2023-10-18T09:19:21.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.262+0000 7f61baa16b80 1 -- start start 2023-10-18T09:19:21.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.263+0000 7f61baa16b80 1 Processor -- start 2023-10-18T09:19:21.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde762d0 0x55a4bde7ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 0x55a4bde85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bddc7e70 con 0x55a4bde755d0 2023-10-18T09:19:21.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bdd64710 con 0x55a4bde762d0 2023-10-18T09:19:21.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bdd643b0 con 0x55a4bde76c30 2023-10-18T09:19:21.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.264+0000 7f61baa16b80 10 client.0 _ll_register_callbacks cb 0x55a4bdcaea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:21.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.265+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.265+0000 7f619ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde762d0 0x55a4bde7ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.265+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:59026/0 (socket says 172.21.15.61:59026) 2023-10-18T09:19:21.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.265+0000 7f61a4d9b700 1 -- 172.21.15.61:0/2799447692 learned_addr learned my addr 172.21.15.61:0/2799447692 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 -- 172.21.15.61:0/2799447692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 msgr2=0x55a4bde85d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a559c700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 0x55a4bde85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 0x55a4bde85d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 -- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde762d0 msgr2=0x55a4bde7ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde762d0 0x55a4bde7ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 -- 172.21.15.61:0/2799447692 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01ad990 con 0x55a4bde755d0 2023-10-18T09:19:21.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a559c700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 0x55a4bde85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:21.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6194096db0 tx=0x7f619409ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3e43b6e6aba72 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f617ffff700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:19:21.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409bbe0 con 0x55a4bde755d0 2023-10-18T09:19:21.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.266+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4bddccef0 con 0x55a4bde755d0 2023-10-18T09:19:21.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.267+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f619409bd40 con 0x55a4bde755d0 2023-10-18T09:19:21.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.267+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f619409ab10 con 0x55a4bde755d0 2023-10-18T09:19:21.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.267+0000 7f61baa16b80 10 client.15777 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:21.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.267+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a4bdbf2de0 con 0x55a4bde755d0 2023-10-18T09:19:21.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.267+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4bddccef0 con 0x55a4bde755d0 2023-10-18T09:19:21.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f61940b8d20 con 0x55a4bde755d0 2023-10-18T09:19:21.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:21.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 10 client.15777 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:19:21.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 20 client.15777 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:21.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bde89ff0 0x55a4bde8c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 4 client.15777 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:21.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.268+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4bde448b0 con 0x55a4bde89ff0 2023-10-18T09:19:21.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.269+0000 7f619ffff700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bde89ff0 0x55a4bde8c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.270+0000 7f619ffff700 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bde89ff0 0x55a4bde8c410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2e15c56e768af5c server_cookie=15f816184d0544c5 in_seq=0 out_seq=0 2023-10-18T09:19:21.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.270+0000 7f617ffff700 10 client.15777 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:21.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.271+0000 7f617ffff700 1 -- 172.21.15.61:0/2799447692 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4bde448b0 con 0x55a4bde89ff0 2023-10-18T09:19:21.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.271+0000 7f617ffff700 10 client.15777 handle_command_reply: tid=0 2023-10-18T09:19:21.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.271+0000 7f61baa16b80 1 client.15777 shutdown 2023-10-18T09:19:21.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 msgr2=0x55a4bde7e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6194096db0 tx=0x7f619409ea70 comp rx=0 tx=0).stop 2023-10-18T09:19:21.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 shutdown_connections 2023-10-18T09:19:21.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bde89ff0 0x55a4bde8c410 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde76c30 0x55a4bde85d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde7e780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 --2- 172.21.15.61:0/2799447692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde762d0 0x55a4bde7ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 >> 172.21.15.61:0/2799447692 conn(0x55a4bdde9720 msgr2=0x55a4bde74080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.272+0000 7f617ffff700 0 client.15777 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:21.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.274+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 shutdown_connections 2023-10-18T09:19:21.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.274+0000 7f61baa16b80 1 -- 172.21.15.61:0/2799447692 wait complete. 2023-10-18T09:19:21.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.274+0000 7f61baa16b80 20 client.15777 trim_cache size 0 max 16384 2023-10-18T09:19:21.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 Processor -- start 2023-10-18T09:19:21.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 -- start start 2023-10-18T09:19:21.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde85cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde79810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bdc48920 con 0x55a4bde76c30 2023-10-18T09:19:21.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bde448b0 con 0x55a4bde755d0 2023-10-18T09:19:21.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.299+0000 7f61baa16b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bde7e310 con 0x55a4bde762d0 2023-10-18T09:19:21.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.300+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.300+0000 7f61a4d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36616/0 (socket says 172.21.15.61:36616) 2023-10-18T09:19:21.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.300+0000 7f61a4d9b700 1 -- 172.21.15.61:0/4182701818 learned_addr learned my addr 172.21.15.61:0/4182701818 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.300+0000 7f619ffff700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde85cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.300+0000 7f61a559c700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde79810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 -- 172.21.15.61:0/4182701818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 msgr2=0x55a4bde85cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde85cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 -- 172.21.15.61:0/4182701818 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 msgr2=0x55a4bde79810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde79810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 -- 172.21.15.61:0/4182701818 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4bdbdbc60 con 0x55a4bde755d0 2023-10-18T09:19:21.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f619ffff700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:21.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.301+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f619400fbd0 tx=0x7f6194009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f598efc5c0eb6f4b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.302+0000 7f618b7fe700 1 -- 172.21.15.61:0/4182701818 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619409ab10 con 0x55a4bde755d0 2023-10-18T09:19:21.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.302+0000 7f618b7fe700 1 -- 172.21.15.61:0/4182701818 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f619409ad10 con 0x55a4bde755d0 2023-10-18T09:19:21.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.302+0000 7f618b7fe700 1 -- 172.21.15.61:0/4182701818 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194099fe0 con 0x55a4bde755d0 2023-10-18T09:19:21.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 -- 172.21.15.61:0/4182701818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 msgr2=0x55a4bde792d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f619400fbd0 tx=0x7f6194009c00 comp rx=0 tx=0).stop 2023-10-18T09:19:21.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 -- 172.21.15.61:0/4182701818 shutdown_connections 2023-10-18T09:19:21.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde76c30 0x55a4bde79810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde755d0 0x55a4bde792d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 --2- 172.21.15.61:0/4182701818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde85cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.303+0000 7f61baa16b80 1 -- 172.21.15.61:0/4182701818 >> 172.21.15.61:0/4182701818 conn(0x55a4bdde9720 msgr2=0x55a4bde6f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.304+0000 7f61baa16b80 1 -- 172.21.15.61:0/4182701818 shutdown_connections 2023-10-18T09:19:21.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.304+0000 7f61baa16b80 1 -- 172.21.15.61:0/4182701818 wait complete. 2023-10-18T09:19:21.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.305+0000 7f61baa16b80 1 -- start start 2023-10-18T09:19:21.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.306+0000 7f61baa16b80 1 Processor -- start 2023-10-18T09:19:21.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde83e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bde857b0 con 0x55a4bde755d0 2023-10-18T09:19:21.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bde85920 con 0x55a4bde76c30 2023-10-18T09:19:21.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4bde85ae0 con 0x55a4bde762d0 2023-10-18T09:19:21.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.307+0000 7f61baa16b80 10 client.0 _ll_register_callbacks cb 0x55a4bdcaea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:21.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f61a559c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f61a559c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36622/0 (socket says 172.21.15.61:36622) 2023-10-18T09:19:21.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f61a559c700 1 -- 172.21.15.61:0/3272727903 learned_addr learned my addr 172.21.15.61:0/3272727903 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 -- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 msgr2=0x55a4bde85200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 -- 172.21.15.61:0/3272727903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 msgr2=0x55a4bde83e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde83e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f619ffff700 1 -- 172.21.15.61:0/3272727903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c0119e0 con 0x55a4bde762d0 2023-10-18T09:19:21.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.308+0000 7f61a559c700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:21.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.309+0000 7f619ffff700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde843b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f618c011770 tx=0x7f618c014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278fc92f8384221c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.309+0000 7f617ffff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:21.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618c0269a0 con 0x55a4bde762d0 2023-10-18T09:19:21.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f618c018070 con 0x55a4bde762d0 2023-10-18T09:19:21.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61940a2bb0 con 0x55a4bde762d0 2023-10-18T09:19:21.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f618c026b00 con 0x55a4bde762d0 2023-10-18T09:19:21.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f61baa16b80 10 client.25804 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:21.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.310+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a4bdbf2de0 con 0x55a4bde762d0 2023-10-18T09:19:21.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.311+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61940a2bb0 con 0x55a4bde762d0 2023-10-18T09:19:21.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.311+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f618c01f420 con 0x55a4bde762d0 2023-10-18T09:19:21.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 10 client.25804 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:21.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 10 client.25804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:19:21.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 20 client.25804 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:21.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bdedf1a0 0x55a4bdee15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 4 client.25804 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:21.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a4bddc0e50 con 0x55a4bdedf1a0 2023-10-18T09:19:21.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.313+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bdedf1a0 0x55a4bdee15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.314+0000 7f61a4d9b700 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bdedf1a0 0x55a4bdee15c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3cb9d9f0a87a8c2 server_cookie=ca9c1e17748a199c in_seq=0 out_seq=0 2023-10-18T09:19:21.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.314+0000 7f617ffff700 10 client.25804 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:21.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.314+0000 7f617ffff700 1 -- 172.21.15.61:0/3272727903 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a4bddc0e50 con 0x55a4bdedf1a0 2023-10-18T09:19:21.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.314+0000 7f617ffff700 10 client.25804 handle_command_reply: tid=0 2023-10-18T09:19:21.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.314+0000 7f61baa16b80 1 client.25804 shutdown 2023-10-18T09:19:21.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 msgr2=0x55a4bde843b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde843b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f618c011770 tx=0x7f618c014070 comp rx=0 tx=0).stop 2023-10-18T09:19:21.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 shutdown_connections 2023-10-18T09:19:21.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55a4bdedf1a0 0x55a4bdee15c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a4bde76c30 0x55a4bde85200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f617ffff700 0 client.25804 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:21.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a4bde755d0 0x55a4bde83e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 --2- 172.21.15.61:0/3272727903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a4bde762d0 0x55a4bde843b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.315+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 >> 172.21.15.61:0/3272727903 conn(0x55a4bdde9720 msgr2=0x55a4bde7c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.316+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 shutdown_connections 2023-10-18T09:19:21.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.316+0000 7f61baa16b80 1 -- 172.21.15.61:0/3272727903 wait complete. 2023-10-18T09:19:21.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.317+0000 7f61baa16b80 20 client.25804 trim_cache size 0 max 16384 2023-10-18T09:19:21.326 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:21.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 msgr2=0x7f61900a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 0x7f61900a3aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bf0 tx=0x7f618c009260 comp rx=0 tx=0).stop 2023-10-18T09:19:21.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 msgr2=0x7f61a01a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6194009810 tx=0x7f619400b840 comp rx=0 tx=0).stop 2023-10-18T09:19:21.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 shutdown_connections 2023-10-18T09:19:21.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f61900a15e0 0x7f61900a3aa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f61a010cb30 0x7f61a01acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f61a01077f0 0x7f61a01a5540 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 --2- 172.21.15.61:0/270615135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61a010a160 0x7f61a01a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.319+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 >> 172.21.15.61:0/270615135 conn(0x7f61a006d7d0 msgr2=0x7f61a010a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.320+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 shutdown_connections 2023-10-18T09:19:21.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.320+0000 7f61a6fff700 1 -- 172.21.15.61:0/270615135 wait complete. 2023-10-18T09:19:21.351 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-18T09:19:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:20 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1652682928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:20 smithi170 ceph-mon[135150]: pgmap v382: 129 pgs: 129 active+clean; 911 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 66 MiB/s wr, 198 op/s 2023-10-18T09:19:21.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 -- 172.21.15.61:0/4203028616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a40ff690 msgr2=0x7f32a40ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 --2- 172.21.15.61:0/4203028616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a40ff690 0x7f32a40ffab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3294009990 tx=0x7f3294009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 -- 172.21.15.61:0/4203028616 shutdown_connections 2023-10-18T09:19:21.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 --2- 172.21.15.61:0/4203028616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4101220 0x7f32a4067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 --2- 172.21.15.61:0/4203028616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a40ff690 0x7f32a40ffab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 --2- 172.21.15.61:0/4203028616 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a4100880 0x7f32a4100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 -- 172.21.15.61:0/4203028616 >> 172.21.15.61:0/4203028616 conn(0x7f32a40fac00 msgr2=0x7f32a40fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 -- 172.21.15.61:0/4203028616 shutdown_connections 2023-10-18T09:19:21.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.658+0000 7f32a9fff700 1 -- 172.21.15.61:0/4203028616 wait complete. 2023-10-18T09:19:21.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.659+0000 7f32a9fff700 1 Processor -- start 2023-10-18T09:19:21.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 -- start start 2023-10-18T09:19:21.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a40ff690 0x7f32a41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4100880 0x7f32a4198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a41a04f0 con 0x7f32a40ff690 2023-10-18T09:19:21.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a41a0630 con 0x7f32a4101220 2023-10-18T09:19:21.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a9fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a41a07c0 con 0x7f32a4100880 2023-10-18T09:19:21.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4100880 0x7f32a4198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a40ff690 0x7f32a41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36646/0 (socket says 172.21.15.61:36646) 2023-10-18T09:19:21.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.660+0000 7f32a3fff700 1 -- 172.21.15.61:0/1396236766 learned_addr learned my addr 172.21.15.61:0/1396236766 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 -- 172.21.15.61:0/1396236766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4100880 msgr2=0x7f32a4198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4100880 0x7f32a4198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 -- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a40ff690 msgr2=0x7f32a41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a40ff690 0x7f32a41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 -- 172.21.15.61:0/1396236766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940095f0 con 0x7f32a4101220 2023-10-18T09:19:21.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a3fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f329c00ef30 tx=0x7f329c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a0ff9700 1 -- 172.21.15.61:0/1396236766 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c018070 con 0x7f32a4101220 2023-10-18T09:19:21.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a0ff9700 1 -- 172.21.15.61:0/1396236766 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329c00f040 con 0x7f32a4101220 2023-10-18T09:19:21.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a41a0a40 con 0x7f32a4101220 2023-10-18T09:19:21.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a0ff9700 1 -- 172.21.15.61:0/1396236766 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00cd00 con 0x7f32a4101220 2023-10-18T09:19:21.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.661+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41a0f50 con 0x7f32a4101220 2023-10-18T09:19:21.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.665+0000 7f32a0ff9700 1 -- 172.21.15.61:0/1396236766 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f329c003a00 con 0x7f32a4101220 2023-10-18T09:19:21.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.666+0000 7f32a0ff9700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 0x7f328c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.666+0000 7f32a0ff9700 1 -- 172.21.15.61:0/1396236766 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f329c0ccb80 con 0x7f32a4101220 2023-10-18T09:19:21.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.667+0000 7f32a37fe700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 0x7f328c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 Processor -- start 2023-10-18T09:19:21.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 -- start start 2023-10-18T09:19:21.671 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32a37fe700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 0x7f328c0a39c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f329400b680 tx=0x7f3294005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x56509898d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509898da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650989006e0 con 0x56509898ec30 2023-10-18T09:19:21.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650988e6c60 con 0x56509898e2d0 2023-10-18T09:19:21.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.668+0000 7f32bda12b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650988e4580 con 0x56509898d5d0 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509898da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36648/0 (socket says 172.21.15.61:36648) 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 -- 172.21.15.61:0/2436085072 learned_addr learned my addr 172.21.15.61:0/2436085072 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x56509898d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 -- 172.21.15.61:0/2436085072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 msgr2=0x56509898da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509898da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 -- 172.21.15.61:0/2436085072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 msgr2=0x56509898d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x56509898d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 -- 172.21.15.61:0/2436085072 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56509889d3f0 con 0x56509898e2d0 2023-10-18T09:19:21.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32a41006e0 tx=0x7f3294020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d90872480f0249f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.669+0000 7f3289ffb700 1 -- 172.21.15.61:0/2436085072 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329402b070 con 0x56509898e2d0 2023-10-18T09:19:21.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f3289ffb700 1 -- 172.21.15.61:0/2436085072 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329401e040 con 0x56509898e2d0 2023-10-18T09:19:21.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f3289ffb700 1 -- 172.21.15.61:0/2436085072 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400f880 con 0x56509898e2d0 2023-10-18T09:19:21.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f32bda12b80 1 -- 172.21.15.61:0/2436085072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 msgr2=0x56509898e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f32a41006e0 tx=0x7f3294020000 comp rx=0 tx=0).stop 2023-10-18T09:19:21.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f32bda12b80 1 -- 172.21.15.61:0/2436085072 shutdown_connections 2023-10-18T09:19:21.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.670+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509898da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898e2d0 0x56509898e6f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2436085072 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x56509898d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 -- 172.21.15.61:0/2436085072 >> 172.21.15.61:0/2436085072 conn(0x565098901720 msgr2=0x565098987b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 -- 172.21.15.61:0/2436085072 shutdown_connections 2023-10-18T09:19:21.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 -- 172.21.15.61:0/2436085072 wait complete. 2023-10-18T09:19:21.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.671+0000 7f32bda12b80 1 -- start start 2023-10-18T09:19:21.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 Processor -- start 2023-10-18T09:19:21.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x565098996fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898ec30 0x56509899e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650988dfe70 con 0x56509898e2d0 2023-10-18T09:19:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509887c710 con 0x56509898d5d0 2023-10-18T09:19:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509887c3b0 con 0x56509898ec30 2023-10-18T09:19:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.672+0000 7f32bda12b80 10 client.0 _ll_register_callbacks cb 0x5650987c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36650/0 (socket says 172.21.15.61:36650) 2023-10-18T09:19:21.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 -- 172.21.15.61:0/2817849968 learned_addr learned my addr 172.21.15.61:0/2817849968 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a3fff700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898ec30 0x56509899e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x565098996fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 -- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898ec30 msgr2=0x56509899e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898ec30 0x56509899e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 -- 172.21.15.61:0/2817849968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 msgr2=0x565098996fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x565098996fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a37fe700 1 -- 172.21.15.61:0/2817849968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a4068260 con 0x56509898d5d0 2023-10-18T09:19:21.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.673+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x565098996fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:21.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f32a37fe700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a163ff5252917670 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f3282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:21.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329402b070 con 0x56509898d5d0 2023-10-18T09:19:21.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f329400f810 con 0x56509898d5d0 2023-10-18T09:19:21.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650988e4ef0 con 0x56509898d5d0 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.674+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3294034070 con 0x56509898d5d0 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.675+0000 7f32bda12b80 10 client.25799 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.675+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56509870ade0 con 0x56509898d5d0 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.675+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3294040030 con 0x56509898d5d0 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.675+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f3294039920 con 0x56509898d5d0 2023-10-18T09:19:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.676+0000 7f32bda12b80 10 client.25799 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:21.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.676+0000 7f32bda12b80 10 client.25799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:19:21.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.676+0000 7f32bda12b80 20 client.25799 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:21.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.676+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5650989a24a0 0x5650989a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.677+0000 7f32bda12b80 4 client.25799 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:21.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.677+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56509895c8b0 con 0x5650989a24a0 2023-10-18T09:19:21.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.677+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5650989a24a0 0x5650989a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.678+0000 7f3282ffd700 10 client.25799 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:21.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.678+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5650989a24a0 0x5650989a48c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3fcc9617a1f26fae server_cookie=8da31635658428cb in_seq=0 out_seq=0 2023-10-18T09:19:21.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.679+0000 7f3282ffd700 1 -- 172.21.15.61:0/2817849968 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56509895c8b0 con 0x5650989a24a0 2023-10-18T09:19:21.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.679+0000 7f3282ffd700 10 client.25799 handle_command_reply: tid=0 2023-10-18T09:19:21.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.679+0000 7f32bda12b80 1 client.25799 shutdown 2023-10-18T09:19:21.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 msgr2=0x565098996aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3294000c00 tx=0x7f3294004ba0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 shutdown_connections 2023-10-18T09:19:21.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x5650989a24a0 0x5650989a48c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898ec30 0x56509899e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x565098996aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 --2- 172.21.15.61:0/2817849968 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x565098996fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 >> 172.21.15.61:0/2817849968 conn(0x565098901720 msgr2=0x56509898c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.681+0000 7f3282ffd700 0 client.25799 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:21.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.682+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 shutdown_connections 2023-10-18T09:19:21.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.682+0000 7f32bda12b80 1 -- 172.21.15.61:0/2817849968 wait complete. 2023-10-18T09:19:21.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.682+0000 7f32bda12b80 20 client.25799 trim_cache size 0 max 16384 2023-10-18T09:19:21.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.707+0000 7f32bda12b80 1 Processor -- start 2023-10-18T09:19:21.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.707+0000 7f32bda12b80 1 -- start start 2023-10-18T09:19:21.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x5650989825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 0x56509899dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x565098760920 con 0x56509898ec30 2023-10-18T09:19:21.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x565098984300 con 0x56509898d5d0 2023-10-18T09:19:21.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.709+0000 7f32bda12b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509899f1a0 con 0x56509898e2d0 2023-10-18T09:19:21.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x5650989825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 0x56509899dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:59090/0 (socket says 172.21.15.61:59090) 2023-10-18T09:19:21.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a3fff700 1 -- 172.21.15.61:0/1936645603 learned_addr learned my addr 172.21.15.61:0/1936645603 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a2ffd700 1 -- 172.21.15.61:0/1936645603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 msgr2=0x5650989825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x5650989825d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.710+0000 7f32a2ffd700 1 -- 172.21.15.61:0/1936645603 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 msgr2=0x565098995fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.711+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.711+0000 7f32a2ffd700 1 -- 172.21.15.61:0/1936645603 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650986f3c60 con 0x56509898e2d0 2023-10-18T09:19:21.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.711+0000 7f32a3fff700 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:21.716 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.711+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 0x56509899dfe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5650989a4ea0 tx=0x7f3298008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65877d52ae4d21bc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.717 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.712+0000 7f328a7fc700 1 -- 172.21.15.61:0/1936645603 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329801c070 con 0x56509898e2d0 2023-10-18T09:19:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.712+0000 7f328a7fc700 1 -- 172.21.15.61:0/1936645603 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3298016430 con 0x56509898e2d0 2023-10-18T09:19:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.712+0000 7f328a7fc700 1 -- 172.21.15.61:0/1936645603 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298015620 con 0x56509898e2d0 2023-10-18T09:19:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 -- 172.21.15.61:0/1936645603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 msgr2=0x56509899dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.718 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 0x56509899dfe0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5650989a4ea0 tx=0x7f3298008040 comp rx=0 tx=0).stop 2023-10-18T09:19:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 -- 172.21.15.61:0/1936645603 shutdown_connections 2023-10-18T09:19:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898ec30 0x565098995fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898d5d0 0x5650989825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 --2- 172.21.15.61:0/1936645603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898e2d0 0x56509899dfe0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.719 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.713+0000 7f32bda12b80 1 -- 172.21.15.61:0/1936645603 >> 172.21.15.61:0/1936645603 conn(0x565098901720 msgr2=0x56509898d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.714+0000 7f32bda12b80 1 -- 172.21.15.61:0/1936645603 shutdown_connections 2023-10-18T09:19:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.714+0000 7f32bda12b80 1 -- 172.21.15.61:0/1936645603 wait complete. 2023-10-18T09:19:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.714+0000 7f32bda12b80 1 -- start start 2023-10-18T09:19:21.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.715+0000 7f32bda12b80 1 Processor -- start 2023-10-18T09:19:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509899db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x56509899c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509899e350 con 0x56509898e2d0 2023-10-18T09:19:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509899ccb0 con 0x56509898ec30 2023-10-18T09:19:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56509899ce70 con 0x56509898d5d0 2023-10-18T09:19:21.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32bda12b80 10 client.0 _ll_register_callbacks cb 0x5650987c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36682/0 (socket says 172.21.15.61:36682) 2023-10-18T09:19:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 -- 172.21.15.61:0/4072262531 learned_addr learned my addr 172.21.15.61:0/4072262531 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.717+0000 7f32a37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509899db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a2ffd700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x56509899c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 -- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 msgr2=0x56509899db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509899db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 -- 172.21.15.61:0/4072262531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 msgr2=0x56509899c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:21.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x56509899c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.718+0000 7f32a3fff700 1 -- 172.21.15.61:0/4072262531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a41a0f50 con 0x56509898ec30 2023-10-18T09:19:21.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.719+0000 7f32a37fe700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509899db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:21.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.719+0000 7f32a3fff700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f329c099880 tx=0x7f329c01b3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d720d3dbfbfa250 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:21.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.719+0000 7f3282ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:21.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.720+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c099b80 con 0x56509898ec30 2023-10-18T09:19:21.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.720+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f329c09dcd0 con 0x56509898ec30 2023-10-18T09:19:21.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565098995ab0 con 0x56509898ec30 2023-10-18T09:19:21.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f329c0a1e80 con 0x56509898ec30 2023-10-18T09:19:21.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f32bda12b80 10 client.25805 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:21.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f3298009840 con 0x56509898ec30 2023-10-18T09:19:21.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565098995ab0 con 0x56509898ec30 2023-10-18T09:19:21.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.721+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f329c0a19f0 con 0x56509898ec30 2023-10-18T09:19:21.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:21.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 10 client.25805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:19:21.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 20 client.25805 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:21.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x565098a00230 0x565098a02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:21.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 4 client.25805 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:21.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f329401e040 con 0x565098a00230 2023-10-18T09:19:21.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.722+0000 7f32a37fe700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x565098a00230 0x565098a02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:21.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.723+0000 7f32a37fe700 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x565098a00230 0x565098a02650 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f02e9cab47b399d server_cookie=3212af3d0913954b in_seq=0 out_seq=0 2023-10-18T09:19:21.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.723+0000 7f3282ffd700 10 client.25805 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:21.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.724+0000 7f3282ffd700 1 -- 172.21.15.61:0/4072262531 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f329401e040 con 0x565098a00230 2023-10-18T09:19:21.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.724+0000 7f3282ffd700 10 client.25805 handle_command_reply: tid=0 2023-10-18T09:19:21.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.724+0000 7f32bda12b80 1 client.25805 shutdown 2023-10-18T09:19:21.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 msgr2=0x56509899c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f329c099880 tx=0x7f329c01b3d0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 shutdown_connections 2023-10-18T09:19:21.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x565098a00230 0x565098a02650 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56509898ec30 0x56509899c770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56509898d5d0 0x56509899db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 --2- 172.21.15.61:0/4072262531 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56509898e2d0 0x56509899c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 >> 172.21.15.61:0/4072262531 conn(0x565098901720 msgr2=0x565098994940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f3282ffd700 0 client.25805 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:21.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.726+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 shutdown_connections 2023-10-18T09:19:21.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.727+0000 7f32bda12b80 1 -- 172.21.15.61:0/4072262531 wait complete. 2023-10-18T09:19:21.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.727+0000 7f32bda12b80 20 client.25805 trim_cache size 0 max 16384 2023-10-18T09:19:21.734 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:21.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 msgr2=0x7f328c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 0x7f328c0a39c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f329400b680 tx=0x7f3294005fd0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 msgr2=0x7f32a419ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:21.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f329c00ef30 tx=0x7f329c00c3a0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 shutdown_connections 2023-10-18T09:19:21.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f328c0a1500 0x7f328c0a39c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f32a4101220 0x7f32a419ff20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f32a40ff690 0x7f32a41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 --2- 172.21.15.61:0/1396236766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a4100880 0x7f32a4198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:21.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 >> 172.21.15.61:0/1396236766 conn(0x7f32a40fac00 msgr2=0x7f32a4101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:21.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.728+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 shutdown_connections 2023-10-18T09:19:21.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:21.729+0000 7f32a9fff700 1 -- 172.21.15.61:0/1396236766 wait complete. 2023-10-18T09:19:21.763 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T09:19:22.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 -- 172.21.15.61:0/183945392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 msgr2=0x7fcb10105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 --2- 172.21.15.61:0/183945392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb10105f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcb000098f0 tx=0x7fcb00009c00 comp rx=0 tx=0).stop 2023-10-18T09:19:22.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 -- 172.21.15.61:0/183945392 shutdown_connections 2023-10-18T09:19:22.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 --2- 172.21.15.61:0/183945392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10106470 0x7fcb101088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 --2- 172.21.15.61:0/183945392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10101210 0x7fcb10103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 --2- 172.21.15.61:0/183945392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb10105f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.151+0000 7fcb177ff700 1 -- 172.21.15.61:0/183945392 >> 172.21.15.61:0/183945392 conn(0x7fcb100fac00 msgr2=0x7fcb100fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.152+0000 7fcb177ff700 1 -- 172.21.15.61:0/183945392 shutdown_connections 2023-10-18T09:19:22.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.152+0000 7fcb177ff700 1 -- 172.21.15.61:0/183945392 wait complete. 2023-10-18T09:19:22.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.153+0000 7fcb177ff700 1 Processor -- start 2023-10-18T09:19:22.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 -- start start 2023-10-18T09:19:22.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 0x7fcb1019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb101a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 0x7fcb101a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb101a4860 con 0x7fcb10101210 2023-10-18T09:19:22.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10194ce0 con 0x7fcb10106470 2023-10-18T09:19:22.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.154+0000 7fcb177ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb10194ea0 con 0x7fcb10103b40 2023-10-18T09:19:22.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.155+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb101a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.155+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb101a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57912/0 (socket says 172.21.15.61:57912) 2023-10-18T09:19:22.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.155+0000 7fcb14d9a700 1 -- 172.21.15.61:0/2932566092 learned_addr learned my addr 172.21.15.61:0/2932566092 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:22.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.155+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 0x7fcb101a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.155+0000 7fcb1559b700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 0x7fcb1019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 -- 172.21.15.61:0/2932566092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 msgr2=0x7fcb101a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb101a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 -- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 msgr2=0x7fcb1019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 0x7fcb1019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 -- 172.21.15.61:0/2932566092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00009520 con 0x7fcb10106470 2023-10-18T09:19:22.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb1559b700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 0x7fcb1019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:22.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.156+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 0x7fcb101a4090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800efc0 tx=0x7fcb0800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.157+0000 7fcb067fc700 1 -- 172.21.15.61:0/2932566092 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08018070 con 0x7fcb10106470 2023-10-18T09:19:22.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.157+0000 7fcb067fc700 1 -- 172.21.15.61:0/2932566092 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0800f040 con 0x7fcb10106470 2023-10-18T09:19:22.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.157+0000 7fcb067fc700 1 -- 172.21.15.61:0/2932566092 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08018070 con 0x7fcb10106470 2023-10-18T09:19:22.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.158+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb10195120 con 0x7fcb10106470 2023-10-18T09:19:22.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.158+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb101954f0 con 0x7fcb10106470 2023-10-18T09:19:22.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.163+0000 7fcb067fc700 1 -- 172.21.15.61:0/2932566092 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcb08003a00 con 0x7fcb10106470 2023-10-18T09:19:22.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.165+0000 7fcb2b0aeb80 1 Processor -- start 2023-10-18T09:19:22.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.165+0000 7fcb067fc700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 0x7fcafc0a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.165+0000 7fcb2b0aeb80 1 -- start start 2023-10-18T09:19:22.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 0x5559fb085090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559faff86e0 con 0x5559fb086c30 2023-10-18T09:19:22.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fafdec60 con 0x5559fb0855d0 2023-10-18T09:19:22.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fafdc580 con 0x5559fb0862d0 2023-10-18T09:19:22.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb067fc700 1 -- 172.21.15.61:0/2932566092 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcb080cc090 con 0x7fcb10106470 2023-10-18T09:19:22.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 0x7fcafc0a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57916/0 (socket says 172.21.15.61:57916) 2023-10-18T09:19:22.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb1559b700 1 -- 172.21.15.61:0/3693686771 learned_addr learned my addr 172.21.15.61:0/3693686771 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:22.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 0x5559fb085090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb1559b700 1 -- 172.21.15.61:0/3693686771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 msgr2=0x5559fb085a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:22.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.166+0000 7fcb1559b700 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcb1559b700 1 -- 172.21.15.61:0/3693686771 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 msgr2=0x5559fb085090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:22.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcb1559b700 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 0x5559fb085090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcb1559b700 1 -- 172.21.15.61:0/3693686771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559faf953f0 con 0x5559fb0862d0 2023-10-18T09:19:22.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 0x5559fb085090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:22.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcb1559b700 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009db0 tx=0x7fcb0c0116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c88568563b1aec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcafb7fe700 1 -- 172.21.15.61:0/3693686771 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c011ed0 con 0x5559fb0862d0 2023-10-18T09:19:22.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcafb7fe700 1 -- 172.21.15.61:0/3693686771 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c012910 con 0x5559fb0862d0 2023-10-18T09:19:22.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.167+0000 7fcafb7fe700 1 -- 172.21.15.61:0/3693686771 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00f940 con 0x5559fb0862d0 2023-10-18T09:19:22.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.168+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 0x7fcafc0a3a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcb1010fcf0 tx=0x7fcb00019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.169+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/3693686771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 msgr2=0x5559fb0866f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.169+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c009db0 tx=0x7fcb0c0116c0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/3693686771 shutdown_connections 2023-10-18T09:19:22.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb0866f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/3693686771 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb086c30 0x5559fb085090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/3693686771 >> 172.21.15.61:0/3693686771 conn(0x5559faff9720 msgr2=0x5559fb07fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/3693686771 shutdown_connections 2023-10-18T09:19:22.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.171+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/3693686771 wait complete. 2023-10-18T09:19:22.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 -- start start 2023-10-18T09:19:22.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 Processor -- start 2023-10-18T09:19:22.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb08eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb08f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 0x5559fb096100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fafd7e70 con 0x5559fb0855d0 2023-10-18T09:19:22.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559faf74710 con 0x5559fb086c30 2023-10-18T09:19:22.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559faf743b0 con 0x5559fb0862d0 2023-10-18T09:19:22.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.172+0000 7fcb2b0aeb80 10 client.0 _ll_register_callbacks cb 0x5559faebea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:22.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb08f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb08f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57928/0 (socket says 172.21.15.61:57928) 2023-10-18T09:19:22.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb14d9a700 1 -- 172.21.15.61:0/1383289433 learned_addr learned my addr 172.21.15.61:0/1383289433 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:22.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 0x5559fb096100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb1559b700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb08eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 -- 172.21.15.61:0/1383289433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 msgr2=0x5559fb08f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:22.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb08f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 -- 172.21.15.61:0/1383289433 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 msgr2=0x5559fb08eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:22.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb08eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb15d9c700 1 -- 172.21.15.61:0/1383289433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c0099e0 con 0x5559fb086c30 2023-10-18T09:19:22.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.173+0000 7fcb1559b700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb08eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:22.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.174+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 0x5559fb096100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb080c9bb0 tx=0x7fcb08098ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102c42e253058f37 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.174+0000 7fcaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:22.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.174+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08099760 con 0x5559fb086c30 2023-10-18T09:19:22.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.174+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcb080ace40 con 0x5559fb086c30 2023-10-18T09:19:22.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.175+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559fafdcef0 con 0x5559fb086c30 2023-10-18T09:19:22.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.175+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb08099060 con 0x5559fb086c30 2023-10-18T09:19:22.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.175+0000 7fcb2b0aeb80 10 client.25814 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:22.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.175+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5559fae02de0 con 0x5559fb086c30 2023-10-18T09:19:22.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.175+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5559fafdcef0 con 0x5559fb086c30 2023-10-18T09:19:22.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcb080a0f60 con 0x5559fb086c30 2023-10-18T09:19:22.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcb2b0aeb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:22.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcb2b0aeb80 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:22.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcb2b0aeb80 20 client.25814 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:22.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb09a8d0 0x5559fb09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.176+0000 7fcb2b0aeb80 4 client.25814 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:22.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.177+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559fb0548b0 con 0x5559fb09a8d0 2023-10-18T09:19:22.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.177+0000 7fcb1559b700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb09a8d0 0x5559fb09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.178+0000 7fcaf8ff9700 10 client.25814 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:22.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.178+0000 7fcb1559b700 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb09a8d0 0x5559fb09ccf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462f9e8ba7e6856a server_cookie=c5fc47c1d3a81e01 in_seq=0 out_seq=0 2023-10-18T09:19:22.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.178+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/1383289433 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559fb0548b0 con 0x5559fb09a8d0 2023-10-18T09:19:22.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.178+0000 7fcaf8ff9700 10 client.25814 handle_command_reply: tid=0 2023-10-18T09:19:22.191 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 client.25814 shutdown 2023-10-18T09:19:22.191 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 msgr2=0x5559fb096100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.191 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 0x5559fb096100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb080c9bb0 tx=0x7fcb08098ba0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.191 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 shutdown_connections 2023-10-18T09:19:22.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb09a8d0 0x5559fb09ccf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb086c30 0x5559fb096100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcaf8ff9700 0 client.25814 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:22.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb08eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/1383289433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb0862d0 0x5559fb08f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.179+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 >> 172.21.15.61:0/1383289433 conn(0x5559faff9720 msgr2=0x5559fb084080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.180+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 shutdown_connections 2023-10-18T09:19:22.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.180+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/1383289433 wait complete. 2023-10-18T09:19:22.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.180+0000 7fcb2b0aeb80 20 client.25814 trim_cache size 0 max 16384 2023-10-18T09:19:22.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.215+0000 7fcb2b0aeb80 1 Processor -- start 2023-10-18T09:19:22.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.215+0000 7fcb2b0aeb80 1 -- start start 2023-10-18T09:19:22.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 0x5559fb07a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb096020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fae58920 con 0x5559fb0862d0 2023-10-18T09:19:22.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fb097db0 con 0x5559fb0855d0 2023-10-18T09:19:22.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.216+0000 7fcb2b0aeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fb0972d0 con 0x5559fb086c30 2023-10-18T09:19:22.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36732/0 (socket says 172.21.15.61:36732) 2023-10-18T09:19:22.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 -- 172.21.15.61:0/380787285 learned_addr learned my addr 172.21.15.61:0/380787285 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:22.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 -- 172.21.15.61:0/380787285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 msgr2=0x5559fb096020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:19:22.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb096020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb1559b700 1 -- 172.21.15.61:0/380787285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 msgr2=0x5559fb07a5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:19:22.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.217+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 0x5559fb07a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.218+0000 7fcb1559b700 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 0x5559fb07a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.218+0000 7fcb1559b700 1 -- 172.21.15.61:0/380787285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559fadebc60 con 0x5559fb0855d0 2023-10-18T09:19:22.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.218+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 0x5559fb07a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:22.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.218+0000 7fcb1559b700 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0059d0 tx=0x7fcb0c005960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e78a6c12c3ee52 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.219+0000 7fcafbfff700 1 -- 172.21.15.61:0/380787285 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c012450 con 0x5559fb0855d0 2023-10-18T09:19:22.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.219+0000 7fcafbfff700 1 -- 172.21.15.61:0/380787285 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c0125b0 con 0x5559fb0855d0 2023-10-18T09:19:22.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.219+0000 7fcafbfff700 1 -- 172.21.15.61:0/380787285 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003bc0 con 0x5559fb0855d0 2023-10-18T09:19:22.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/380787285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 msgr2=0x5559fb085090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0059d0 tx=0x7fcb0c005960 comp rx=0 tx=0).stop 2023-10-18T09:19:22.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/380787285 shutdown_connections 2023-10-18T09:19:22.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb096020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0855d0 0x5559fb085090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/380787285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0862d0 0x5559fb07a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/380787285 >> 172.21.15.61:0/380787285 conn(0x5559faff9720 msgr2=0x5559fb0859b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/380787285 shutdown_connections 2023-10-18T09:19:22.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.220+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/380787285 wait complete. 2023-10-18T09:19:22.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.221+0000 7fcb2b0aeb80 1 -- start start 2023-10-18T09:19:22.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.221+0000 7fcb2b0aeb80 1 Processor -- start 2023-10-18T09:19:22.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 0x5559fb094620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb2b0aeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb095590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb1559b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:59138/0 (socket says 172.21.15.61:59138) 2023-10-18T09:19:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.223+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 0x5559fb094620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb14d9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 0x5559fb094620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36746/0 (socket says 172.21.15.61:36746) 2023-10-18T09:19:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb14d9a700 1 -- 172.21.15.61:0/842179390 learned_addr learned my addr 172.21.15.61:0/842179390 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:22.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fb095ad0 con 0x5559fb0855d0 2023-10-18T09:19:22.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fb095c40 con 0x5559fb0862d0 2023-10-18T09:19:22.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559fb095e00 con 0x5559fb086c30 2023-10-18T09:19:22.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb2b0aeb80 10 client.0 _ll_register_callbacks cb 0x5559faebea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:22.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.224+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb095590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 -- 172.21.15.61:0/842179390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 msgr2=0x5559fb095590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:22.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb095590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 -- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 msgr2=0x5559fb094620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:22.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 0x5559fb094620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559faf953f0 con 0x5559fb0855d0 2023-10-18T09:19:22.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb15d9c700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb095590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:22.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcb1559b700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5559fb0971f0 tx=0x7fcb0c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f77bf23ff7511e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:22.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.225+0000 7fcaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:19:22.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003ac0 con 0x5559fb0855d0 2023-10-18T09:19:22.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcb0c003c20 con 0x5559fb0855d0 2023-10-18T09:19:22.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb0c012910 con 0x5559fb0855d0 2023-10-18T09:19:22.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb0c003e20 con 0x5559fb0855d0 2023-10-18T09:19:22.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcb2b0aeb80 10 client.15807 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:22.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.226+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5559fae29250 con 0x5559fb0855d0 2023-10-18T09:19:22.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.227+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb0c005580 con 0x5559fb0855d0 2023-10-18T09:19:22.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcb0c0113e0 con 0x5559fb0855d0 2023-10-18T09:19:22.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 10 client.15807 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:22.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:22.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 20 client.15807 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:22.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb0ef340 0x5559fb0f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:22.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 4 client.15807 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:22.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.230+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fcb0c012910 con 0x5559fb0ef340 2023-10-18T09:19:22.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.231+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb0ef340 0x5559fb0f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:22.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.231+0000 7fcb14d9a700 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb0ef340 0x5559fb0f1760 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14727f008040dff server_cookie=27910520a58bc8c7 in_seq=0 out_seq=0 2023-10-18T09:19:22.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.231+0000 7fcaf8ff9700 10 client.15807 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:22.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.231+0000 7fcaf8ff9700 1 -- 172.21.15.61:0/842179390 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fcb0c012910 con 0x5559fb0ef340 2023-10-18T09:19:22.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcaf8ff9700 10 client.15807 handle_command_reply: tid=0 2023-10-18T09:19:22.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 client.15807 shutdown 2023-10-18T09:19:22.241 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 msgr2=0x5559fb0940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x5559fb0971f0 tx=0x7fcb0c005a90 comp rx=0 tx=0).stop 2023-10-18T09:19:22.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 shutdown_connections 2023-10-18T09:19:22.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5559fb0ef340 0x5559fb0f1760 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.242 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559fb086c30 0x5559fb095590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559fb0855d0 0x5559fb0940e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 --2- 172.21.15.61:0/842179390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559fb0862d0 0x5559fb094620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 >> 172.21.15.61:0/842179390 conn(0x5559faff9720 msgr2=0x5559fb07fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcaf8ff9700 0 client.15807 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:22.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 shutdown_connections 2023-10-18T09:19:22.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.232+0000 7fcb2b0aeb80 1 -- 172.21.15.61:0/842179390 wait complete. 2023-10-18T09:19:22.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.233+0000 7fcb2b0aeb80 20 client.15807 trim_cache size 0 max 16384 2023-10-18T09:19:22.245 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:22.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 msgr2=0x7fcafc0a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 0x7fcafc0a3a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcb1010fcf0 tx=0x7fcb00019040 comp rx=0 tx=0).stop 2023-10-18T09:19:22.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 msgr2=0x7fcb101a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:22.246 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 0x7fcb101a4090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800efc0 tx=0x7fcb0800c3a0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 shutdown_connections 2023-10-18T09:19:22.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcafc0a1560 0x7fcafc0a3a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb10106470 0x7fcb101a4090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcb10101210 0x7fcb1019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 --2- 172.21.15.61:0/2932566092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10103b40 0x7fcb101a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:22.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 >> 172.21.15.61:0/2932566092 conn(0x7fcb100fac00 msgr2=0x7fcb101050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:22.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.234+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 shutdown_connections 2023-10-18T09:19:22.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:22.235+0000 7fcb177ff700 1 -- 172.21.15.61:0/2932566092 wait complete. 2023-10-18T09:19:23.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:22 smithi061 ceph-mon[98573]: pgmap v383: 129 pgs: 129 active+clean; 975 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 50 MiB/s wr, 219 op/s 2023-10-18T09:19:23.273 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T09:19:23.274 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T09:19:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:22 smithi170 ceph-mon[135150]: pgmap v383: 129 pgs: 129 active+clean; 975 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 50 MiB/s wr, 219 op/s 2023-10-18T09:19:23.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:22 smithi110 ceph-mon[134818]: pgmap v383: 129 pgs: 129 active+clean; 975 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 50 MiB/s wr, 219 op/s 2023-10-18T09:19:23.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 -- 172.21.15.61:0/79241681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 msgr2=0x7fef14108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/79241681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef14108c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00b0a0 tx=0x7fef0c00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 -- 172.21.15.61:0/79241681 shutdown_connections 2023-10-18T09:19:23.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/79241681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef14108c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/79241681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef140fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/79241681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 0x7fef14101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.658+0000 7fef1c3ff700 1 -- 172.21.15.61:0/79241681 >> 172.21.15.61:0/79241681 conn(0x7fef140fa380 msgr2=0x7fef140fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.659+0000 7fef1c3ff700 1 -- 172.21.15.61:0/79241681 shutdown_connections 2023-10-18T09:19:23.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.659+0000 7fef1c3ff700 1 -- 172.21.15.61:0/79241681 wait complete. 2023-10-18T09:19:23.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.660+0000 7fef1c3ff700 1 Processor -- start 2023-10-18T09:19:23.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1c3ff700 1 -- start start 2023-10-18T09:19:23.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1c3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1c3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 0x7fef14199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1c3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef141a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1a19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1a19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57960/0 (socket says 172.21.15.61:57960) 2023-10-18T09:19:23.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1a19b700 1 -- 172.21.15.61:0/2368316142 learned_addr learned my addr 172.21.15.61:0/2368316142 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:23.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.661+0000 7fef1c3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141a05f0 con 0x7fef14101a20 2023-10-18T09:19:23.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1999a700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 0x7fef14199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141a0730 con 0x7fef141023c0 2023-10-18T09:19:23.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef141a08c0 con 0x7fef140fea50 2023-10-18T09:19:23.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a99c700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef141a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a19b700 1 -- 172.21.15.61:0/2368316142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 msgr2=0x7fef141a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a19b700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef141a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a19b700 1 -- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 msgr2=0x7fef14199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a19b700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 0x7fef14199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.662+0000 7fef1a19b700 1 -- 172.21.15.61:0/2368316142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c009d00 con 0x7fef140fea50 2023-10-18T09:19:23.668 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.663+0000 7fef1a19b700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef1000e970 tx=0x7fef1000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.663+0000 7fef0b7fe700 1 -- 172.21.15.61:0/2368316142 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10009790 con 0x7fef140fea50 2023-10-18T09:19:23.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.663+0000 7fef0b7fe700 1 -- 172.21.15.61:0/2368316142 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1000c7e0 con 0x7fef140fea50 2023-10-18T09:19:23.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.663+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef141a0ba0 con 0x7fef140fea50 2023-10-18T09:19:23.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.664+0000 7fef0b7fe700 1 -- 172.21.15.61:0/2368316142 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10003cb0 con 0x7fef140fea50 2023-10-18T09:19:23.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.664+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef141a1100 con 0x7fef140fea50 2023-10-18T09:19:23.672 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.669+0000 7fef0b7fe700 1 -- 172.21.15.61:0/2368316142 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fef1000c950 con 0x7fef140fea50 2023-10-18T09:19:23.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.671+0000 7fef2fd6cb80 1 Processor -- start 2023-10-18T09:19:23.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.671+0000 7fef2fd6cb80 1 -- start start 2023-10-18T09:19:23.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.671+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 0x560ca363c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca363cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca35af760 con 0x560ca363c650 2023-10-18T09:19:23.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca3595ce0 con 0x560ca363dcb0 2023-10-18T09:19:23.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca3593600 con 0x560ca363d350 2023-10-18T09:19:23.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef1a99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef1a99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57964/0 (socket says 172.21.15.61:57964) 2023-10-18T09:19:23.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef1a99c700 1 -- 172.21.15.61:0/1492239013 learned_addr learned my addr 172.21.15.61:0/1492239013 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:23.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef1a19b700 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 0x560ca363c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.672+0000 7fef0b7fe700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 0x7fef000a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 -- 172.21.15.61:0/1492239013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 msgr2=0x560ca363c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef0b7fe700 1 -- 172.21.15.61:0/2368316142 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fef10014070 con 0x7fef140fea50 2023-10-18T09:19:23.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 0x560ca363c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 -- 172.21.15.61:0/1492239013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 msgr2=0x560ca363cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca363cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 -- 172.21.15.61:0/1492239013 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca354c470 con 0x560ca363d350 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 0x7fef000a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a19b700 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 0x560ca363c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.673+0000 7fef1a99c700 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef0c015000 tx=0x7fef0c009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e6ae6107ea6742 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.674+0000 7fef08bfe700 1 -- 172.21.15.61:0/1492239013 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00e040 con 0x560ca363d350 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.674+0000 7fef08bfe700 1 -- 172.21.15.61:0/1492239013 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef0c003c00 con 0x560ca363d350 2023-10-18T09:19:23.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.674+0000 7fef08bfe700 1 -- 172.21.15.61:0/1492239013 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c028440 con 0x560ca363d350 2023-10-18T09:19:23.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/1492239013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 msgr2=0x560ca363d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef0c015000 tx=0x7fef0c009460 comp rx=0 tx=0).stop 2023-10-18T09:19:23.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef1a99c700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 0x7fef000a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fef0c02a000 tx=0x7fef0c027000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/1492239013 shutdown_connections 2023-10-18T09:19:23.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca363cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca363d770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.675+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/1492239013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363dcb0 0x560ca363c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.676+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/1492239013 >> 172.21.15.61:0/1492239013 conn(0x560ca35b07a0 msgr2=0x560ca3636bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.676+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/1492239013 shutdown_connections 2023-10-18T09:19:23.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.676+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/1492239013 wait complete. 2023-10-18T09:19:23.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.677+0000 7fef2fd6cb80 1 -- start start 2023-10-18T09:19:23.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.677+0000 7fef2fd6cb80 1 Processor -- start 2023-10-18T09:19:23.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.677+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca3645b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.677+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363d350 0x560ca3646090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.678+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.678+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca358eef0 con 0x560ca363c650 2023-10-18T09:19:23.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.678+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca352b790 con 0x560ca363d350 2023-10-18T09:19:23.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.679+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca352b430 con 0x560ca363dcb0 2023-10-18T09:19:23.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.679+0000 7fef2fd6cb80 10 client.0 _ll_register_callbacks cb 0x560ca3475b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:23.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.680+0000 7fef1999a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363d350 0x560ca3646090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.680+0000 7fef1a99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.680+0000 7fef1a99c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57978/0 (socket says 172.21.15.61:57978) 2023-10-18T09:19:23.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.680+0000 7fef1a99c700 1 -- 172.21.15.61:0/3599679619 learned_addr learned my addr 172.21.15.61:0/3599679619 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:23.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a19b700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca3645b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a99c700 1 -- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363d350 msgr2=0x560ca3646090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a99c700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363d350 0x560ca3646090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a99c700 1 -- 172.21.15.61:0/3599679619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 msgr2=0x560ca3645b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a99c700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca3645b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a99c700 1 -- 172.21.15.61:0/3599679619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef141a1100 con 0x560ca363dcb0 2023-10-18T09:19:23.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.681+0000 7fef1a19b700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca3645b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:23.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.682+0000 7fef1a99c700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009a20 tx=0x7fef0c020fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ac1c613848dd26 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.682+0000 7feefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:23.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.682+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c028440 con 0x560ca363dcb0 2023-10-18T09:19:23.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.682+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca3593f70 con 0x560ca363dcb0 2023-10-18T09:19:23.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.682+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fef0c003a00 con 0x560ca363dcb0 2023-10-18T09:19:23.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef0c01e670 con 0x560ca363dcb0 2023-10-18T09:19:23.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 10 client.25846 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:23.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fef14109570 con 0x560ca363dcb0 2023-10-18T09:19:23.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560ca3593f70 con 0x560ca363dcb0 2023-10-18T09:19:23.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fef0c005170 con 0x560ca363dcb0 2023-10-18T09:19:23.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 10 client.25846 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:23.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 10 client.25846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:23.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 20 client.25846 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:23.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca3662560 0x560ca3664980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 4 client.25846 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:23.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.683+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ca360b930 con 0x560ca3662560 2023-10-18T09:19:23.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.686+0000 7fef1a19b700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca3662560 0x560ca3664980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.686+0000 7feefdffb700 10 client.25846 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:23.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.686+0000 7fef1a19b700 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca3662560 0x560ca3664980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec0961ad07b58742 server_cookie=1237e6015edb9546 in_seq=0 out_seq=0 2023-10-18T09:19:23.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.687+0000 7feefdffb700 1 -- 172.21.15.61:0/3599679619 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ca360b930 con 0x560ca3662560 2023-10-18T09:19:23.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.687+0000 7feefdffb700 10 client.25846 handle_command_reply: tid=0 2023-10-18T09:19:23.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.688+0000 7fef2fd6cb80 1 client.25846 shutdown 2023-10-18T09:19:23.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.688+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 msgr2=0x560ca364d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.688+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009a20 tx=0x7fef0c020fc0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 shutdown_connections 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca3662560 0x560ca3664980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca364d0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363c650 0x560ca3645b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3599679619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363d350 0x560ca3646090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 >> 172.21.15.61:0/3599679619 conn(0x560ca35b07a0 msgr2=0x560ca363b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7feefdffb700 0 client.25846 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:23.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 shutdown_connections 2023-10-18T09:19:23.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.689+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3599679619 wait complete. 2023-10-18T09:19:23.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.690+0000 7fef2fd6cb80 20 client.25846 trim_cache size 0 max 16384 2023-10-18T09:19:23.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.729+0000 7fef2fd6cb80 1 Processor -- start 2023-10-18T09:19:23.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.729+0000 7fef2fd6cb80 1 -- start start 2023-10-18T09:19:23.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca3640350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363dcb0 0x560ca3640890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca340f9a0 con 0x560ca363dcb0 2023-10-18T09:19:23.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca3646a50 con 0x560ca363c650 2023-10-18T09:19:23.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca3593f70 con 0x560ca363d350 2023-10-18T09:19:23.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:57982/0 (socket says 172.21.15.61:57982) 2023-10-18T09:19:23.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 -- 172.21.15.61:0/3822000171 learned_addr learned my addr 172.21.15.61:0/3822000171 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:23.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1a19b700 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca3640350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 -- 172.21.15.61:0/3822000171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 msgr2=0x560ca3640350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca3640350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 -- 172.21.15.61:0/3822000171 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363dcb0 msgr2=0x560ca3640890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363dcb0 0x560ca3640890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.730+0000 7fef1999a700 1 -- 172.21.15.61:0/3822000171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca33b9e60 con 0x560ca363d350 2023-10-18T09:19:23.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.731+0000 7fef1999a700 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ba0 tx=0x7fef04009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e583e19a5494c213 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.731+0000 7fef093ff700 1 -- 172.21.15.61:0/3822000171 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04011970 con 0x560ca363d350 2023-10-18T09:19:23.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.731+0000 7fef093ff700 1 -- 172.21.15.61:0/3822000171 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef04014440 con 0x560ca363d350 2023-10-18T09:19:23.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.731+0000 7fef093ff700 1 -- 172.21.15.61:0/3822000171 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0401d640 con 0x560ca363d350 2023-10-18T09:19:23.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3822000171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 msgr2=0x560ca364d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fef04009ba0 tx=0x7fef04009eb0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3822000171 shutdown_connections 2023-10-18T09:19:23.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363dcb0 0x560ca3640890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca3640350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/3822000171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363d350 0x560ca364d090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3822000171 >> 172.21.15.61:0/3822000171 conn(0x560ca35b07a0 msgr2=0x560ca3643b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3822000171 shutdown_connections 2023-10-18T09:19:23.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.732+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/3822000171 wait complete. 2023-10-18T09:19:23.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.733+0000 7fef2fd6cb80 1 -- start start 2023-10-18T09:19:23.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.733+0000 7fef2fd6cb80 1 Processor -- start 2023-10-18T09:19:23.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca364bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca36beb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca364cc60 con 0x560ca363d350 2023-10-18T09:19:23.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca36bf100 con 0x560ca363c650 2023-10-18T09:19:23.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca36bf2c0 con 0x560ca363dcb0 2023-10-18T09:19:23.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef2fd6cb80 10 client.0 _ll_register_callbacks cb 0x560ca3475b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:23.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.734+0000 7fef1999a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:59168/0 (socket says 172.21.15.61:59168) 2023-10-18T09:19:23.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 -- 172.21.15.61:0/635451010 learned_addr learned my addr 172.21.15.61:0/635451010 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:23.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1a19b700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca364bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1a99c700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca36beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 -- 172.21.15.61:0/635451010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 msgr2=0x560ca36beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca36beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 -- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 msgr2=0x560ca364bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:23.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca364bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1999a700 1 -- 172.21.15.61:0/635451010 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef04009850 con 0x560ca363d350 2023-10-18T09:19:23.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.735+0000 7fef1a99c700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca36beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:23.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.736+0000 7fef1a19b700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca364bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:23.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.736+0000 7fef1999a700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef04014d70 tx=0x7fef04011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b283b6cd00c0d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:23.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.736+0000 7feefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:19:23.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.737+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04021040 con 0x560ca363d350 2023-10-18T09:19:23.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.737+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef0c003c00 con 0x560ca363d350 2023-10-18T09:19:23.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.737+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fef0401d810 con 0x560ca363d350 2023-10-18T09:19:23.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.738+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef04014b90 con 0x560ca363d350 2023-10-18T09:19:23.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.738+0000 7fef2fd6cb80 10 client.15813 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:23.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.738+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x560ca33a2ce0 con 0x560ca363d350 2023-10-18T09:19:23.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.738+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef0c003c00 con 0x560ca363d350 2023-10-18T09:19:23.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fef0401b020 con 0x560ca363d350 2023-10-18T09:19:23.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:23.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:23.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 20 client.15813 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:23.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca36c0d20 0x560ca36c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:23.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 4 client.15813 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T09:19:23.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.739+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560ca3587ed0 con 0x560ca36c0d20 2023-10-18T09:19:23.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.740+0000 7fef1a19b700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca36c0d20 0x560ca36c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:23.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.740+0000 7feefdffb700 10 client.15813 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:23.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.740+0000 7fef1a19b700 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca36c0d20 0x560ca36c3140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86405addbd52d29 server_cookie=1421bbe794e81db8 in_seq=0 out_seq=0 2023-10-18T09:19:23.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7feefdffb700 1 -- 172.21.15.61:0/635451010 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560ca3587ed0 con 0x560ca36c0d20 2023-10-18T09:19:23.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7feefdffb700 10 client.15813 handle_command_reply: tid=0 2023-10-18T09:19:23.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 client.15813 shutdown 2023-10-18T09:19:23.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 msgr2=0x560ca364c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef04014d70 tx=0x7fef04011c50 comp rx=0 tx=0).stop 2023-10-18T09:19:23.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 shutdown_connections 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560ca36c0d20 0x560ca36c3140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ca363dcb0 0x560ca36beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560ca363c650 0x560ca364bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 --2- 172.21.15.61:0/635451010 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560ca363d350 0x560ca364c530 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 >> 172.21.15.61:0/635451010 conn(0x560ca35b07a0 msgr2=0x560ca3640350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.742+0000 7feefdffb700 0 client.15813 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:23.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.743+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 shutdown_connections 2023-10-18T09:19:23.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.743+0000 7fef2fd6cb80 1 -- 172.21.15.61:0/635451010 wait complete. 2023-10-18T09:19:23.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.743+0000 7fef2fd6cb80 20 client.15813 trim_cache size 0 max 16384 2023-10-18T09:19:23.753 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:23.754 INFO:teuthology.orchestra.run.smithi061.stdout: "return_code": 0, 2023-10-18T09:19:23.754 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_tag": "596a2eac-e7e7-46e2-9d34-349f7fe8246b", 2023-10-18T09:19:23.754 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": "asynchronous" 2023-10-18T09:19:23.754 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:23.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.744+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 msgr2=0x7fef000a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.744+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 0x7fef000a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fef0c02a000 tx=0x7fef0c027000 comp rx=0 tx=0).stop 2023-10-18T09:19:23.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.744+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 msgr2=0x7fef14198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:23.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.744+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef1000e970 tx=0x7fef1000ec80 comp rx=0 tx=0).stop 2023-10-18T09:19:23.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 shutdown_connections 2023-10-18T09:19:23.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fef000a1500 0x7fef000a39c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef141023c0 0x7fef141a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef140fea50 0x7fef14198ac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 --2- 172.21.15.61:0/2368316142 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fef14101a20 0x7fef14199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:23.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 >> 172.21.15.61:0/2368316142 conn(0x7fef140fa380 msgr2=0x7fef140ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:23.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 shutdown_connections 2023-10-18T09:19:23.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:23.745+0000 7fef1c3ff700 1 -- 172.21.15.61:0/2368316142 wait complete. 2023-10-18T09:19:23.783 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:19:24.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 -- 172.21.15.61:0/2857927597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff960 msgr2=0x7fc1f00ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/2857927597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff960 0x7fc1f00ffde0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009990 tx=0x7fc1e4009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 -- 172.21.15.61:0/2857927597 shutdown_connections 2023-10-18T09:19:24.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/2857927597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 0x7fc1f01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/2857927597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff000 0x7fc1f00ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/2857927597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff960 0x7fc1f00ffde0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 -- 172.21.15.61:0/2857927597 >> 172.21.15.61:0/2857927597 conn(0x7fc1f00fac00 msgr2=0x7fc1f00fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.098+0000 7fc1f55ff700 1 -- 172.21.15.61:0/2857927597 shutdown_connections 2023-10-18T09:19:24.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.099+0000 7fc1f55ff700 1 -- 172.21.15.61:0/2857927597 wait complete. 2023-10-18T09:19:24.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.100+0000 7fc1f55ff700 1 Processor -- start 2023-10-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.100+0000 7fc1f55ff700 1 -- start start 2023-10-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1eeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1eeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:58016/0 (socket says 172.21.15.61:58016) 2023-10-18T09:19:24.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 0x7fc1f019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a0450 con 0x7fc1f0100320 2023-10-18T09:19:24.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36824/0 (socket says 172.21.15.61:36824) 2023-10-18T09:19:24.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a0590 con 0x7fc1f00ff960 2023-10-18T09:19:24.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01a0720 con 0x7fc1f00ff000 2023-10-18T09:19:24.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.101+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/3166348924 learned_addr learned my addr 172.21.15.61:0/3166348924 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:24.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 -- 172.21.15.61:0/3166348924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 msgr2=0x7fc1f0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 0x7fc1f019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 -- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 msgr2=0x7fc1f019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 0x7fc1f019fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 -- 172.21.15.61:0/3166348924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40095f0 con 0x7fc1f00ff000 2023-10-18T09:19:24.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:24.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1eeffd700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009fa0 tx=0x7fc1e000ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1d7fff700 1 -- 172.21.15.61:0/3166348924 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000ee60 con 0x7fc1f00ff000 2023-10-18T09:19:24.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1d7fff700 1 -- 172.21.15.61:0/3166348924 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e000cc50 con 0x7fc1f00ff000 2023-10-18T09:19:24.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1d7fff700 1 -- 172.21.15.61:0/3166348924 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e001fd60 con 0x7fc1f00ff000 2023-10-18T09:19:24.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.102+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f0101270 con 0x7fc1f00ff000 2023-10-18T09:19:24.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.103+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f0101730 con 0x7fc1f00ff000 2023-10-18T09:19:24.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.107+0000 7fc1d7fff700 1 -- 172.21.15.61:0/3166348924 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc1e000c790 con 0x7fc1f00ff000 2023-10-18T09:19:24.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.109+0000 7fc208fa9b80 1 Processor -- start 2023-10-18T09:19:24.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.109+0000 7fc208fa9b80 1 -- start start 2023-10-18T09:19:24.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 0x562e00dbc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 0x562e00dbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbb5d0 0x562e00dbba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d2e6e0 con 0x562e00dbc2d0 2023-10-18T09:19:24.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d14c60 con 0x562e00dbb5d0 2023-10-18T09:19:24.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.110+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d12580 con 0x562e00dbcc30 2023-10-18T09:19:24.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 0x562e00dbc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1d7fff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 0x7fc1d80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 0x562e00dbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 0x562e00dbc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:59212/0 (socket says 172.21.15.61:59212) 2023-10-18T09:19:24.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/401367022 learned_addr learned my addr 172.21.15.61:0/401367022 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:24.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbb5d0 0x562e00dbba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1d7fff700 1 -- 172.21.15.61:0/3166348924 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1e00d5750 con 0x7fc1f00ff000 2023-10-18T09:19:24.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/401367022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbb5d0 msgr2=0x562e00dbba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbb5d0 0x562e00dbba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/401367022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 msgr2=0x562e00dbc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 0x562e00dbc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.111+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/401367022 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e00ccb3f0 con 0x562e00dbcc30 2023-10-18T09:19:24.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.112+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 0x562e00dbb090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400b680 tx=0x7fc1e4005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5049eb9b092dd3b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.112+0000 7fc1eeffd700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 0x7fc1d80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.112+0000 7fc1d57fa700 1 -- 172.21.15.61:0/401367022 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e401d030 con 0x562e00dbcc30 2023-10-18T09:19:24.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.112+0000 7fc1d57fa700 1 -- 172.21.15.61:0/401367022 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e401e040 con 0x562e00dbcc30 2023-10-18T09:19:24.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.112+0000 7fc1d57fa700 1 -- 172.21.15.61:0/401367022 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4004da0 con 0x562e00dbcc30 2023-10-18T09:19:24.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.113+0000 7fc208fa9b80 1 -- 172.21.15.61:0/401367022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 msgr2=0x562e00dbb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.113+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 0x562e00dbb090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400b680 tx=0x7fc1e4005bc0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 -- 172.21.15.61:0/401367022 shutdown_connections 2023-10-18T09:19:24.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbb5d0 0x562e00dbba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbc2d0 0x562e00dbc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/401367022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbcc30 0x562e00dbb090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 -- 172.21.15.61:0/401367022 >> 172.21.15.61:0/401367022 conn(0x562e00d2f720 msgr2=0x562e00db5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc1eeffd700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 0x7fc1d80a3a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00aa520 tx=0x7fc1e001f6a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 -- 172.21.15.61:0/401367022 shutdown_connections 2023-10-18T09:19:24.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.114+0000 7fc208fa9b80 1 -- 172.21.15.61:0/401367022 wait complete. 2023-10-18T09:19:24.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.115+0000 7fc208fa9b80 1 -- start start 2023-10-18T09:19:24.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.115+0000 7fc208fa9b80 1 Processor -- start 2023-10-18T09:19:24.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dc4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dca1d0 0x562e00dcc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d0de70 con 0x562e00dbb5d0 2023-10-18T09:19:24.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00caa710 con 0x562e00dbc2d0 2023-10-18T09:19:24.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00caa3b0 con 0x562e00dca1d0 2023-10-18T09:19:24.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.116+0000 7fc208fa9b80 10 client.0 _ll_register_callbacks cb 0x562e00bf4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:24.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36830/0 (socket says 172.21.15.61:36830) 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/3147266890 learned_addr learned my addr 172.21.15.61:0/3147266890 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/3147266890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dca1d0 msgr2=0x562e00dcc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dca1d0 0x562e00dcc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/3147266890 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 msgr2=0x562e00dc4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dc4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.117+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/3147266890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f010b790 con 0x562e00dbc2d0 2023-10-18T09:19:24.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.118+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009ce0 tx=0x7fc1e40246a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cd458945ee8adb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.118+0000 7fc1d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:24.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.118+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4024d90 con 0x562e00dbc2d0 2023-10-18T09:19:24.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.118+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1e400f460 con 0x562e00dbc2d0 2023-10-18T09:19:24.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.118+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e00d12ef0 con 0x562e00dbc2d0 2023-10-18T09:19:24.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e400f620 con 0x562e00dbc2d0 2023-10-18T09:19:24.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 10 client.25850 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:24.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e00b38de0 con 0x562e00dbc2d0 2023-10-18T09:19:24.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e400fc40 con 0x562e00dbc2d0 2023-10-18T09:19:24.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc1e402d990 con 0x562e00dbc2d0 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 10 client.25850 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 20 client.25850 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00dd0930 0x562e00dd2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 4 client.25850 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:24.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.119+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e00d12ef0 con 0x562e00dd0930 2023-10-18T09:19:24.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.120+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00dd0930 0x562e00dd2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.121+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00dd0930 0x562e00dd2d90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67b9d6ba14cdfa8 server_cookie=dce657f9399d8db3 in_seq=0 out_seq=0 2023-10-18T09:19:24.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.121+0000 7fc1d2ff5700 10 client.25850 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:24.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/3147266890 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562e00d12ef0 con 0x562e00dd0930 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc1d2ff5700 10 client.25850 handle_command_reply: tid=0 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 client.25850 shutdown 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 msgr2=0x562e00dc51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009ce0 tx=0x7fc1e40246a0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 shutdown_connections 2023-10-18T09:19:24.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00dd0930 0x562e00dd2d90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dca1d0 0x562e00dcc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dc4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/3147266890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00dc51a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 >> 172.21.15.61:0/3147266890 conn(0x562e00d2f720 msgr2=0x562e00dba080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.122+0000 7fc1d2ff5700 0 client.25850 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:24.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.123+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 shutdown_connections 2023-10-18T09:19:24.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.123+0000 7fc208fa9b80 1 -- 172.21.15.61:0/3147266890 wait complete. 2023-10-18T09:19:24.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.123+0000 7fc208fa9b80 20 client.25850 trim_cache size 0 max 16384 2023-10-18T09:19:24.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 Processor -- start 2023-10-18T09:19:24.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 -- start start 2023-10-18T09:19:24.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbc2d0 0x562e00db05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00b8e920 con 0x562e00dbb5d0 2023-10-18T09:19:24.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00dadfd0 con 0x562e00dca1d0 2023-10-18T09:19:24.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.154+0000 7fc208fa9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00db7bb0 con 0x562e00dbc2d0 2023-10-18T09:19:24.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.155+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.155+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36842/0 (socket says 172.21.15.61:36842) 2023-10-18T09:19:24.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.155+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/2401166690 learned_addr learned my addr 172.21.15.61:0/2401166690 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:24.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.155+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbc2d0 0x562e00db05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/2401166690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbc2d0 msgr2=0x562e00db05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbc2d0 0x562e00db05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/2401166690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 msgr2=0x562e00dbb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/2401166690 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e00b21c60 con 0x562e00dca1d0 2023-10-18T09:19:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009be0 tx=0x7fc1e801e5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ebd785d175e6dc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1d5ffb700 1 -- 172.21.15.61:0/2401166690 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80095e0 con 0x562e00dca1d0 2023-10-18T09:19:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1d5ffb700 1 -- 172.21.15.61:0/2401166690 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e800f040 con 0x562e00dca1d0 2023-10-18T09:19:24.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.156+0000 7fc1d5ffb700 1 -- 172.21.15.61:0/2401166690 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e8003d80 con 0x562e00dca1d0 2023-10-18T09:19:24.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.157+0000 7fc208fa9b80 1 -- 172.21.15.61:0/2401166690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 msgr2=0x562e00de05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.157+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8009be0 tx=0x7fc1e801e5b0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 -- 172.21.15.61:0/2401166690 shutdown_connections 2023-10-18T09:19:24.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dca1d0 0x562e00de05b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dbb5d0 0x562e00dbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/2401166690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbc2d0 0x562e00db05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 -- 172.21.15.61:0/2401166690 >> 172.21.15.61:0/2401166690 conn(0x562e00d2f720 msgr2=0x562e00dbf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 -- 172.21.15.61:0/2401166690 shutdown_connections 2023-10-18T09:19:24.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.158+0000 7fc208fa9b80 1 -- 172.21.15.61:0/2401166690 wait complete. 2023-10-18T09:19:24.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.159+0000 7fc208fa9b80 1 -- start start 2023-10-18T09:19:24.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.159+0000 7fc208fa9b80 1 Processor -- start 2023-10-18T09:19:24.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.160+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.160+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00e21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc1ef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:58064/0 (socket says 172.21.15.61:58064) 2023-10-18T09:19:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc1ee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00e21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc208fa9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dca1d0 0x562e00e23a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00e24140 con 0x562e00dca1d0 2023-10-18T09:19:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00e242b0 con 0x562e00dbc2d0 2023-10-18T09:19:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00e24470 con 0x562e00dbb5d0 2023-10-18T09:19:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc208fa9b80 10 client.0 _ll_register_callbacks cb 0x562e00bf4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.161+0000 7fc1ef7fe700 1 -- 172.21.15.61:0/12700167 learned_addr learned my addr 172.21.15.61:0/12700167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/12700167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 msgr2=0x562e00e1eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/12700167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dca1d0 msgr2=0x562e00e23a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:24.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dca1d0 0x562e00e23a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ee7fc700 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e801e040 con 0x562e00dbc2d0 2023-10-18T09:19:24.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.162+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.163+0000 7fc1d2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.163+0000 7fc1ee7fc700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00e21430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400b7b0 tx=0x7fc1e4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2407dc1e70218a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.163+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e401d070 con 0x562e00dbc2d0 2023-10-18T09:19:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.164+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1e4000e40 con 0x562e00dbc2d0 2023-10-18T09:19:24.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.164+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1e401e040 con 0x562e00dbc2d0 2023-10-18T09:19:24.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.164+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1e4017770 con 0x562e00dbc2d0 2023-10-18T09:19:24.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.165+0000 7fc208fa9b80 10 client.25856 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:24.172 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.165+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562e00b5f250 con 0x562e00dbc2d0 2023-10-18T09:19:24.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.165+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1e401e040 con 0x562e00dbc2d0 2023-10-18T09:19:24.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.165+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fc1e4048070 con 0x562e00dbc2d0 2023-10-18T09:19:24.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:24.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 10 client.25856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:24.173 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 20 client.25856 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:24.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00e25a70 0x562e00e27e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:24.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 4 client.25856 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:19:24.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e00d8a8b0 con 0x562e00e25a70 2023-10-18T09:19:24.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.166+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00e25a70 0x562e00e27e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:24.174 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.167+0000 7fc1d2ff5700 10 client.25856 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.167+0000 7fc1ef7fe700 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00e25a70 0x562e00e27e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7df81a8e5f8b5d5 server_cookie=7d464858ad583d79 in_seq=0 out_seq=0 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.167+0000 7fc1d2ff5700 1 -- 172.21.15.61:0/12700167 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562e00d8a8b0 con 0x562e00e25a70 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.167+0000 7fc1d2ff5700 10 client.25856 handle_command_reply: tid=0 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.167+0000 7fc208fa9b80 1 client.25856 shutdown 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 msgr2=0x562e00e21430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.175 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00e21430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400b7b0 tx=0x7fc1e4009580 comp rx=0 tx=0).stop 2023-10-18T09:19:24.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 shutdown_connections 2023-10-18T09:19:24.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562e00e25a70 0x562e00e27e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562e00dca1d0 0x562e00e23a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562e00dbb5d0 0x562e00e1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.176 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 --2- 172.21.15.61:0/12700167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562e00dbc2d0 0x562e00e21430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 >> 172.21.15.61:0/12700167 conn(0x562e00d2f720 msgr2=0x562e00db5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc1d2ff5700 0 client.25856 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 shutdown_connections 2023-10-18T09:19:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.168+0000 7fc208fa9b80 1 -- 172.21.15.61:0/12700167 wait complete. 2023-10-18T09:19:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.169+0000 7fc208fa9b80 20 client.25856 trim_cache size 0 max 16384 2023-10-18T09:19:24.178 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:24.178 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T09:19:24.178 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:19:24.178 INFO:teuthology.orchestra.run.smithi061.stdout: "596a2eac-e7e7-46e2-9d34-349f7fe8246b": { 2023-10-18T09:19:24.179 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:19:24.179 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "596a2eac-e7e7-46e2-9d34-349f7fe8246b", 2023-10-18T09:19:24.179 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:19:24.179 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:19:24.179 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:19:24.180 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:24.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 msgr2=0x7fc1d80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 0x7fc1d80a3a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00aa520 tx=0x7fc1e001f6a0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 msgr2=0x7fc1f0198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:24.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009fa0 tx=0x7fc1e000ec20 comp rx=0 tx=0).stop 2023-10-18T09:19:24.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 shutdown_connections 2023-10-18T09:19:24.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc1d80a15d0 0x7fc1d80a3a90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc1f0100320 0x7fc1f019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1f00ff000 0x7fc1f0198920 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 --2- 172.21.15.61:0/3166348924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc1f00ff960 0x7fc1f0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:24.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.170+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 >> 172.21.15.61:0/3166348924 conn(0x7fc1f00fac00 msgr2=0x7fc1f00fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:24.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.171+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 shutdown_connections 2023-10-18T09:19:24.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:24.171+0000 7fc1f55ff700 1 -- 172.21.15.61:0/3166348924 wait complete. 2023-10-18T09:19:24.203 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:23 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:24.203 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:23 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:24.204 INFO:tasks.cephfs.filesystem:scrub status for tag:596a2eac-e7e7-46e2-9d34-349f7fe8246b - {'path': '/', 'tag': '596a2eac-e7e7-46e2-9d34-349f7fe8246b', 'options': 'recursive,force'} 2023-10-18T09:19:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 10.0 s, 1639.0 tps, lat 60.605 ms stddev 80.805 2023-10-18T09:19:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:23 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:24.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:23 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:24.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:23 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:24.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:23 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub queued for path: / 2023-10-18T09:19:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:24 smithi061 ceph-mon[98573]: pgmap v384: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 316 op/s 2023-10-18T09:19:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub queued for path: / 2023-10-18T09:19:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:24 smithi170 ceph-mon[135150]: pgmap v384: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 316 op/s 2023-10-18T09:19:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub queued for path: / 2023-10-18T09:19:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:19:25.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:19:25.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:24 smithi110 ceph-mon[134818]: pgmap v384: 129 pgs: 129 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 50 MiB/s wr, 316 op/s 2023-10-18T09:19:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:27 smithi170 ceph-mon[135150]: pgmap v385: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 408 op/s 2023-10-18T09:19:27.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:27 smithi110 ceph-mon[134818]: pgmap v385: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 408 op/s 2023-10-18T09:19:28.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:27 smithi061 ceph-mon[98573]: pgmap v385: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 41 MiB/s wr, 408 op/s 2023-10-18T09:19:29.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:28 smithi061 ceph-mon[98573]: pgmap v386: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 483 op/s 2023-10-18T09:19:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 15.0 s, 1316.6 tps, lat 75.698 ms stddev 106.145 2023-10-18T09:19:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:28 smithi170 ceph-mon[135150]: pgmap v386: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 483 op/s 2023-10-18T09:19:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:28 smithi110 ceph-mon[134818]: pgmap v386: 129 pgs: 129 active+clean; 1.0 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 40 MiB/s wr, 483 op/s 2023-10-18T09:19:31.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:30 smithi061 ceph-mon[98573]: pgmap v387: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 859 op/s 2023-10-18T09:19:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:30 smithi170 ceph-mon[135150]: pgmap v387: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 859 op/s 2023-10-18T09:19:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:30 smithi110 ceph-mon[134818]: pgmap v387: 129 pgs: 129 active+clean; 1.1 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 MiB/s wr, 859 op/s 2023-10-18T09:19:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:32 smithi061 ceph-mon[98573]: pgmap v388: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 57 MiB/s wr, 906 op/s 2023-10-18T09:19:33.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:32 smithi170 ceph-mon[135150]: pgmap v388: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 57 MiB/s wr, 906 op/s 2023-10-18T09:19:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:32 smithi110 ceph-mon[134818]: pgmap v388: 129 pgs: 129 active+clean; 1.1 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 57 MiB/s wr, 906 op/s 2023-10-18T09:19:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 20.0 s, 2013.0 tps, lat 49.770 ms stddev 62.380 2023-10-18T09:19:35.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:34 smithi061 ceph-mon[98573]: pgmap v389: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 60 MiB/s wr, 1.01k op/s 2023-10-18T09:19:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:34 smithi170 ceph-mon[135150]: pgmap v389: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 60 MiB/s wr, 1.01k op/s 2023-10-18T09:19:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:34 smithi110 ceph-mon[134818]: pgmap v389: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 60 MiB/s wr, 1.01k op/s 2023-10-18T09:19:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:35 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:35 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:35 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:35 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:35 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:19:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:35 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:35 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:35 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:35 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:35 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:19:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:35 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:35 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:35 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:35 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:35 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:19:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:37 smithi170 ceph-mon[135150]: pgmap v390: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 1.11k op/s 2023-10-18T09:19:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:37 smithi110 ceph-mon[134818]: pgmap v390: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 1.11k op/s 2023-10-18T09:19:38.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:37 smithi061 ceph-mon[98573]: pgmap v390: 129 pgs: 129 active+clean; 1.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 58 MiB/s wr, 1.11k op/s 2023-10-18T09:19:39.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:38 smithi061 ceph-mon[98573]: pgmap v391: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 1.11k op/s 2023-10-18T09:19:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 25.0 s, 1934.2 tps, lat 51.865 ms stddev 87.668 2023-10-18T09:19:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:38 smithi170 ceph-mon[135150]: pgmap v391: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 1.11k op/s 2023-10-18T09:19:39.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:38 smithi110 ceph-mon[134818]: pgmap v391: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 57 MiB/s wr, 1.11k op/s 2023-10-18T09:19:41.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:41 smithi061 ceph-mon[98573]: pgmap v392: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 1.29k op/s 2023-10-18T09:19:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:41 smithi170 ceph-mon[135150]: pgmap v392: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 1.29k op/s 2023-10-18T09:19:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:40 smithi110 ceph-mon[134818]: pgmap v392: 129 pgs: 129 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 59 MiB/s wr, 1.29k op/s 2023-10-18T09:19:43.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:42 smithi061 ceph-mon[98573]: pgmap v393: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 951 op/s 2023-10-18T09:19:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:42 smithi170 ceph-mon[135150]: pgmap v393: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 951 op/s 2023-10-18T09:19:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:42 smithi110 ceph-mon[134818]: pgmap v393: 129 pgs: 129 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 35 MiB/s wr, 951 op/s 2023-10-18T09:19:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 30.0 s, 2208.7 tps, lat 44.864 ms stddev 54.336 2023-10-18T09:19:45.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:44 smithi061 ceph-mon[98573]: pgmap v394: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 963 op/s 2023-10-18T09:19:45.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:44 smithi170 ceph-mon[135150]: pgmap v394: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 963 op/s 2023-10-18T09:19:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:44 smithi110 ceph-mon[134818]: pgmap v394: 129 pgs: 129 active+clean; 1.2 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 31 MiB/s wr, 963 op/s 2023-10-18T09:19:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:47 smithi170 ceph-mon[135150]: pgmap v395: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 917 op/s 2023-10-18T09:19:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:47 smithi110 ceph-mon[134818]: pgmap v395: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 917 op/s 2023-10-18T09:19:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:47 smithi061 ceph-mon[98573]: pgmap v395: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 27 MiB/s wr, 917 op/s 2023-10-18T09:19:49.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:48 smithi061 ceph-mon[98573]: pgmap v396: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 786 op/s 2023-10-18T09:19:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 35.0 s, 1936.5 tps, lat 52.218 ms stddev 66.132 2023-10-18T09:19:49.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:49 smithi170 ceph-mon[135150]: pgmap v396: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 786 op/s 2023-10-18T09:19:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:48 smithi110 ceph-mon[134818]: pgmap v396: 129 pgs: 129 active+clean; 1.3 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 MiB/s wr, 786 op/s 2023-10-18T09:19:51.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:50 smithi061 ceph-mon[98573]: pgmap v397: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 914 op/s 2023-10-18T09:19:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:50 smithi170 ceph-mon[135150]: pgmap v397: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 914 op/s 2023-10-18T09:19:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:50 smithi110 ceph-mon[134818]: pgmap v397: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 MiB/s wr, 914 op/s 2023-10-18T09:19:52.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:19:52.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:52.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:52 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:52.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:19:52.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:52.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:52 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:52.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:19:52.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:52.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:52 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:53 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:53 smithi170 ceph-mon[135150]: pgmap v398: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 694 op/s 2023-10-18T09:19:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:53 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:53 smithi110 ceph-mon[134818]: pgmap v398: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 694 op/s 2023-10-18T09:19:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:53 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:53.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:53 smithi061 ceph-mon[98573]: pgmap v398: 129 pgs: 129 active+clean; 1.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 694 op/s 2023-10-18T09:19:54.205 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:19:54.329 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:19:54.330 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:19:54.330 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 40.0 s, 1762.8 tps, lat 54.555 ms stddev 66.693 2023-10-18T09:19:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:54 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:19:54.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:54 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:19:54.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:54 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:54.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:54 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:19:54.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:54 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:19:54.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:54 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:19:54.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 -- 172.21.15.61:0/1996107786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c0ff000 msgr2=0x7f592c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 --2- 172.21.15.61:0/1996107786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c0ff000 0x7f592c101430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f592400b0a0 tx=0x7f592400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 -- 172.21.15.61:0/1996107786 shutdown_connections 2023-10-18T09:19:54.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 --2- 172.21.15.61:0/1996107786 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c104340 0x7f592c1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 --2- 172.21.15.61:0/1996107786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c0ff000 0x7f592c101430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 --2- 172.21.15.61:0/1996107786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c101970 0x7f592c103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.590+0000 7f59313ff700 1 -- 172.21.15.61:0/1996107786 >> 172.21.15.61:0/1996107786 conn(0x7f592c0fac00 msgr2=0x7f592c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.591+0000 7f59313ff700 1 -- 172.21.15.61:0/1996107786 shutdown_connections 2023-10-18T09:19:54.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.592+0000 7f59313ff700 1 -- 172.21.15.61:0/1996107786 wait complete. 2023-10-18T09:19:54.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.593+0000 7f59313ff700 1 Processor -- start 2023-10-18T09:19:54.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 -- start start 2023-10-18T09:19:54.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c0ff000 0x7f592c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c104340 0x7f592c1a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1a4aa0 con 0x7f592c0ff000 2023-10-18T09:19:54.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c105620 con 0x7f592c101970 2023-10-18T09:19:54.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.594+0000 7f59313ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c1057b0 con 0x7f592c104340 2023-10-18T09:19:54.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.596+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c104340 0x7f592c1a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.596+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46002/0 (socket says 172.21.15.61:46002) 2023-10-18T09:19:54.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 -- 172.21.15.61:0/473501285 learned_addr learned my addr 172.21.15.61:0/473501285 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:54.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 -- 172.21.15.61:0/473501285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c104340 msgr2=0x7f592c1a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c104340 0x7f592c1a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 -- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c0ff000 msgr2=0x7f592c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c0ff000 0x7f592c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.597+0000 7f592a7fc700 1 -- 172.21.15.61:0/473501285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924009d00 con 0x7f592c101970 2023-10-18T09:19:54.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.598+0000 7f592a7fc700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b530 tx=0x7f591c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.599+0000 7f5913fff700 1 -- 172.21.15.61:0/473501285 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c011830 con 0x7f592c101970 2023-10-18T09:19:54.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.599+0000 7f5913fff700 1 -- 172.21.15.61:0/473501285 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c011e50 con 0x7f592c101970 2023-10-18T09:19:54.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.599+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c105a90 con 0x7f592c101970 2023-10-18T09:19:54.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.600+0000 7f5913fff700 1 -- 172.21.15.61:0/473501285 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00f2e0 con 0x7f592c101970 2023-10-18T09:19:54.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.600+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c105f90 con 0x7f592c101970 2023-10-18T09:19:54.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.608+0000 7f5913fff700 1 -- 172.21.15.61:0/473501285 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f591c0105f0 con 0x7f592c101970 2023-10-18T09:19:54.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.610+0000 7f5913fff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 0x7f59140a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.610+0000 7f5913fff700 1 -- 172.21.15.61:0/473501285 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f591c0ccb50 con 0x7f592c101970 2023-10-18T09:19:54.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.611+0000 7f592affd700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 0x7f59140a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.612+0000 7f592affd700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 0x7f59140a3a40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f592400b070 tx=0x7f5924006000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.613+0000 7f5944dbcb80 1 Processor -- start 2023-10-18T09:19:54.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.613+0000 7f5944dbcb80 1 -- start start 2023-10-18T09:19:54.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.613+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f7c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.613+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.614+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.618 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.614+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3eee6e0 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.614+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3ed4c60 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.614+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3ed2580 con 0x55c9c3f7cc30 2023-10-18T09:19:54.619 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.614+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.615+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41570/0 (socket says 172.21.15.61:41570) 2023-10-18T09:19:54.620 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.615+0000 7f592a7fc700 1 -- 172.21.15.61:0/3053413124 learned_addr learned my addr 172.21.15.61:0/3053413124 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592affd700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f7c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592a7fc700 1 -- 172.21.15.61:0/3053413124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 msgr2=0x55c9c3f7c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592b7fe700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592a7fc700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f7c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592a7fc700 1 -- 172.21.15.61:0/3053413124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 msgr2=0x55c9c3f7ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.621 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592a7fc700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592a7fc700 1 -- 172.21.15.61:0/3053413124 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9c3e8b3f0 con 0x55c9c3f7cc30 2023-10-18T09:19:54.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.616+0000 7f592affd700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:54.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.617+0000 7f592b7fe700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:54.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.617+0000 7f592a7fc700 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b500 tx=0x7f591c006ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb79cb9384652739 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.619+0000 7f59117fa700 1 -- 172.21.15.61:0/3053413124 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c099ac0 con 0x55c9c3f7cc30 2023-10-18T09:19:54.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.619+0000 7f59117fa700 1 -- 172.21.15.61:0/3053413124 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c09dc10 con 0x55c9c3f7cc30 2023-10-18T09:19:54.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.619+0000 7f59117fa700 1 -- 172.21.15.61:0/3053413124 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0a1c20 con 0x55c9c3f7cc30 2023-10-18T09:19:54.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.620+0000 7f5944dbcb80 1 -- 172.21.15.61:0/3053413124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 msgr2=0x55c9c3f7b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.620+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b500 tx=0x7f591c006ed0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 -- 172.21.15.61:0/3053413124 shutdown_connections 2023-10-18T09:19:54.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/3053413124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7b090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 -- 172.21.15.61:0/3053413124 >> 172.21.15.61:0/3053413124 conn(0x55c9c3eef720 msgr2=0x55c9c3f75b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 -- 172.21.15.61:0/3053413124 shutdown_connections 2023-10-18T09:19:54.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.621+0000 7f5944dbcb80 1 -- 172.21.15.61:0/3053413124 wait complete. 2023-10-18T09:19:54.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.622+0000 7f5944dbcb80 1 -- start start 2023-10-18T09:19:54.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.623+0000 7f5944dbcb80 1 Processor -- start 2023-10-18T09:19:54.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f84b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3ecde70 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3e6a710 con 0x55c9c3f7cc30 2023-10-18T09:19:54.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3e6a3b0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f5944dbcb80 10 client.0 _ll_register_callbacks cb 0x55c9c3db4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:54.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.624+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.625+0000 7f592a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41580/0 (socket says 172.21.15.61:41580) 2023-10-18T09:19:54.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.625+0000 7f592a7fc700 1 -- 172.21.15.61:0/1666637232 learned_addr learned my addr 172.21.15.61:0/1666637232 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:54.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.625+0000 7f592a7fc700 1 -- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7cc30 msgr2=0x55c9c3f8c0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:54.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592a7fc700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8c0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592affd700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f84b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592a7fc700 1 -- 172.21.15.61:0/1666637232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 msgr2=0x55c9c3f84b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592a7fc700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f84b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592a7fc700 1 -- 172.21.15.61:0/1666637232 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c105f90 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.626+0000 7f592affd700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f84b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:54.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f592a7fc700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f591c096530 tx=0x7f591c09a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b95331e124b2954 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f590eff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:54.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0a1bd0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f591c0b20d0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9c3ed2ef0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f591c0b2290 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f5944dbcb80 10 client.25870 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:54.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.627+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c9c3cf8de0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.629+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9c3ed2ef0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.629+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f591c098bc0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.629+0000 7f5944dbcb80 10 client.25870 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:54.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.629+0000 7f5944dbcb80 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:54.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.630+0000 7f5944dbcb80 20 client.25870 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:54.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.630+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3f92950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.630+0000 7f5944dbcb80 4 client.25870 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:54.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.630+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9c3f4a8b0 con 0x55c9c3f90510 2023-10-18T09:19:54.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.630+0000 7f592affd700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3f92950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.631+0000 7f590eff5700 10 client.25870 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:54.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.631+0000 7f592affd700 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3f92950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9adf5add901206ee server_cookie=9141c976a71e404b in_seq=0 out_seq=0 2023-10-18T09:19:54.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.632+0000 7f590eff5700 1 -- 172.21.15.61:0/1666637232 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9c3f4a8b0 con 0x55c9c3f90510 2023-10-18T09:19:54.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.632+0000 7f590eff5700 10 client.25870 handle_command_reply: tid=0 2023-10-18T09:19:54.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.632+0000 7f5944dbcb80 1 client.25870 shutdown 2023-10-18T09:19:54.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 msgr2=0x55c9c3f85060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f591c096530 tx=0x7f591c09a300 comp rx=0 tx=0).stop 2023-10-18T09:19:54.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 shutdown_connections 2023-10-18T09:19:54.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3f92950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f84b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f590eff5700 0 client.25870 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:54.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/1666637232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f85060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 >> 172.21.15.61:0/1666637232 conn(0x55c9c3eef720 msgr2=0x55c9c3f7a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 shutdown_connections 2023-10-18T09:19:54.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.634+0000 7f5944dbcb80 1 -- 172.21.15.61:0/1666637232 wait complete. 2023-10-18T09:19:54.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.635+0000 7f5944dbcb80 20 client.25870 trim_cache size 0 max 16384 2023-10-18T09:19:54.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.670+0000 7f5944dbcb80 1 Processor -- start 2023-10-18T09:19:54.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.670+0000 7f5944dbcb80 1 -- start start 2023-10-18T09:19:54.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3d4e920 con 0x55c9c3f7cc30 2023-10-18T09:19:54.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3f8d290 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.671+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3f77c00 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41586/0 (socket says 172.21.15.61:41586) 2023-10-18T09:19:54.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 -- 172.21.15.61:0/191391984 learned_addr learned my addr 172.21.15.61:0/191391984 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:54.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592b7fe700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592a7fc700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 -- 172.21.15.61:0/191391984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 msgr2=0x55c9c3f8c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 -- 172.21.15.61:0/191391984 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 msgr2=0x55c9c3f7f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.672+0000 7f592affd700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.673+0000 7f592affd700 1 -- 172.21.15.61:0/191391984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9c3ce1c60 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.673+0000 7f592a7fc700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:54.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.673+0000 7f592affd700 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59240062a0 tx=0x7f5924019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1111c0d784d057f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.673+0000 7f5911ffb700 1 -- 172.21.15.61:0/191391984 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592400e040 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.673+0000 7f5911ffb700 1 -- 172.21.15.61:0/191391984 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f592400cea0 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.674+0000 7f5911ffb700 1 -- 172.21.15.61:0/191391984 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592401b6b0 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.674+0000 7f5944dbcb80 1 -- 172.21.15.61:0/191391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 msgr2=0x55c9c3f7f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.674+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59240062a0 tx=0x7f5924019000 comp rx=0 tx=0).stop 2023-10-18T09:19:54.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 -- 172.21.15.61:0/191391984 shutdown_connections 2023-10-18T09:19:54.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f7f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f7f2d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/191391984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 -- 172.21.15.61:0/191391984 >> 172.21.15.61:0/191391984 conn(0x55c9c3eef720 msgr2=0x55c9c3f84b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 -- 172.21.15.61:0/191391984 shutdown_connections 2023-10-18T09:19:54.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.675+0000 7f5944dbcb80 1 -- 172.21.15.61:0/191391984 wait complete. 2023-10-18T09:19:54.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.676+0000 7f5944dbcb80 1 -- start start 2023-10-18T09:19:54.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.676+0000 7f5944dbcb80 1 Processor -- start 2023-10-18T09:19:54.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3f8ae40 con 0x55c9c3f7cc30 2023-10-18T09:19:54.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3f8afb0 con 0x55c9c3f7c2d0 2023-10-18T09:19:54.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9c3f8b0f0 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.677+0000 7f5944dbcb80 10 client.0 _ll_register_callbacks cb 0x55c9c3db4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:54.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.678+0000 7f592b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.678+0000 7f592affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.678+0000 7f592affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41602/0 (socket says 172.21.15.61:41602) 2023-10-18T09:19:54.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.678+0000 7f592affd700 1 -- 172.21.15.61:0/337997042 learned_addr learned my addr 172.21.15.61:0/337997042 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:54.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592a7fc700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592affd700 1 -- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 msgr2=0x55c9c3f8a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592affd700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592affd700 1 -- 172.21.15.61:0/337997042 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 msgr2=0x55c9c3f8a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:54.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592affd700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592affd700 1 -- 172.21.15.61:0/337997042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5924021040 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f592b7fe700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.679+0000 7f590eff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f592affd700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5924009fd0 tx=0x7f5924003b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dce364ea36629e5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592400e040 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f592402ce40 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f591c09dc10 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592402c350 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f5944dbcb80 10 client.25876 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:54.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.680+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c9c3d1f250 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f591c09dc10 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f592401b450 con 0x55c9c3f7b5d0 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 10 client.25876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 20 client.25876 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:54.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3ff8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:54.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 4 client.25876 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:19:54.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c9c3ec6e50 con 0x55c9c3f90510 2023-10-18T09:19:54.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.681+0000 7f592a7fc700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3ff8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:54.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f592a7fc700 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3ff8810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cabf83149bb090 server_cookie=81987ba67d5f55f in_seq=0 out_seq=0 2023-10-18T09:19:54.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f590eff5700 10 client.25876 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:54.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f590eff5700 1 -- 172.21.15.61:0/337997042 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c9c3ec6e50 con 0x55c9c3f90510 2023-10-18T09:19:54.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f590eff5700 10 client.25876 handle_command_reply: tid=0 2023-10-18T09:19:54.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 client.25876 shutdown 2023-10-18T09:19:54.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 msgr2=0x55c9c3f8bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5924009fd0 tx=0x7f5924003b30 comp rx=0 tx=0).stop 2023-10-18T09:19:54.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 shutdown_connections 2023-10-18T09:19:54.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9c3f90510 0x55c9c3ff8810 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9c3f7cc30 0x55c9c3f8a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9c3f7b5d0 0x55c9c3f8bcc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 --2- 172.21.15.61:0/337997042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9c3f7c2d0 0x55c9c3f8a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.682+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 >> 172.21.15.61:0/337997042 conn(0x55c9c3eef720 msgr2=0x55c9c3f7f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.683+0000 7f590eff5700 0 client.25876 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:54.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.683+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 shutdown_connections 2023-10-18T09:19:54.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.683+0000 7f5944dbcb80 1 -- 172.21.15.61:0/337997042 wait complete. 2023-10-18T09:19:54.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.683+0000 7f5944dbcb80 20 client.25876 trim_cache size 0 max 16384 2023-10-18T09:19:54.696 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:54.697 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "no active scrubs running", 2023-10-18T09:19:54.697 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": {} 2023-10-18T09:19:54.697 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:54.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 msgr2=0x7f59140a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 0x7f59140a3a40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f592400b070 tx=0x7f5924006000 comp rx=0 tx=0).stop 2023-10-18T09:19:54.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 msgr2=0x7f592c19d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:54.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b530 tx=0x7f591c00b8f0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 shutdown_connections 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f59140a1580 0x7f59140a3a40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592c104340 0x7f592c1a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f592c0ff000 0x7f592c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 --2- 172.21.15.61:0/473501285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f592c101970 0x7f592c19d500 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:54.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.685+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 >> 172.21.15.61:0/473501285 conn(0x7f592c0fac00 msgr2=0x7f592c102350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:54.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.686+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 shutdown_connections 2023-10-18T09:19:54.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:54.686+0000 7f59313ff700 1 -- 172.21.15.61:0/473501285 wait complete. 2023-10-18T09:19:54.721 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T09:19:54.721 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:19:55.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 -- 172.21.15.61:0/2798104325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e0100f80 msgr2=0x7fc8e0103370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2798104325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e0100f80 0x7fc8e0103370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009990 tx=0x7fc8d8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 -- 172.21.15.61:0/2798104325 shutdown_connections 2023-10-18T09:19:55.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2798104325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8e01061e0 0x7fc8e0108650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2798104325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e0100f80 0x7fc8e0103370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2798104325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e01038b0 0x7fc8e0105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.105+0000 7fc8e559d700 1 -- 172.21.15.61:0/2798104325 >> 172.21.15.61:0/2798104325 conn(0x7fc8e00fa970 msgr2=0x7fc8e00fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.106+0000 7fc8e559d700 1 -- 172.21.15.61:0/2798104325 shutdown_connections 2023-10-18T09:19:55.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.107+0000 7fc8e559d700 1 -- 172.21.15.61:0/2798104325 wait complete. 2023-10-18T09:19:55.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 Processor -- start 2023-10-18T09:19:55.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 -- start start 2023-10-18T09:19:55.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 0x7fc8e0194030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8e01038b0 0x7fc8e0194570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 0x7fc8e006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0194e30 con 0x7fc8e01038b0 2023-10-18T09:19:55.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e006c760 con 0x7fc8e0100f80 2023-10-18T09:19:55.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.108+0000 7fc8e559d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e006c920 con 0x7fc8e01061e0 2023-10-18T09:19:55.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.109+0000 7fc8e4d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 0x7fc8e006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.109+0000 7fc8e4d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 0x7fc8e006c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41628/0 (socket says 172.21.15.61:41628) 2023-10-18T09:19:55.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.109+0000 7fc8e4d9c700 1 -- 172.21.15.61:0/2157189592 learned_addr learned my addr 172.21.15.61:0/2157189592 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.109+0000 7fc8dffff700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 0x7fc8e0194030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 -- 172.21.15.61:0/2157189592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 msgr2=0x7fc8e006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 0x7fc8e006c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 -- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8e01038b0 msgr2=0x7fc8e0194570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8e01038b0 0x7fc8e0194570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 -- 172.21.15.61:0/2157189592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80095f0 con 0x7fc8e0100f80 2023-10-18T09:19:55.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.110+0000 7fc8dffff700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 0x7fc8e0194030 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b710 tx=0x7fc8d800bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.111+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d801d070 con 0x7fc8e0100f80 2023-10-18T09:19:55.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.111+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8d8003dd0 con 0x7fc8e0100f80 2023-10-18T09:19:55.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.111+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8021410 con 0x7fc8e0100f80 2023-10-18T09:19:55.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.111+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e006cb40 con 0x7fc8e0100f80 2023-10-18T09:19:55.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.112+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e006cf70 con 0x7fc8e0100f80 2023-10-18T09:19:55.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.114+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0107d80 con 0x7fc8e0100f80 2023-10-18T09:19:55.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.116+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc8d8003910 con 0x7fc8e0100f80 2023-10-18T09:19:55.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.119+0000 7fc8dd7fa700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 0x7fc8cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.120+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8d80ce4e0 con 0x7fc8e0100f80 2023-10-18T09:19:55.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.121+0000 7fc8df7fe700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 0x7fc8cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.122+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc8d8097820 con 0x7fc8e0100f80 2023-10-18T09:19:55.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.124+0000 7fc8df7fe700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 0x7fc8cc0a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800e350 tx=0x7fc8c800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:55 smithi170 ceph-mon[135150]: pgmap v399: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 777 op/s 2023-10-18T09:19:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:55 smithi110 ceph-mon[134818]: pgmap v399: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 777 op/s 2023-10-18T09:19:55.511 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:55 smithi061 ceph-mon[98573]: pgmap v399: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 25 MiB/s wr, 777 op/s 2023-10-18T09:19:55.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.597+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8e0107760 con 0x7fc8e0100f80 2023-10-18T09:19:55.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.599+0000 7fc8dd7fa700 1 -- 172.21.15.61:0/2157189592 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11395 (secure 0 0 0) 0x7fc8d809ae40 con 0x7fc8e0100f80 2023-10-18T09:19:55.602 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:19:55.602 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:19:16.512081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:19:55.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.605+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 msgr2=0x7fc8cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.605+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 0x7fc8cc0a39c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800e350 tx=0x7fc8c800b040 comp rx=0 tx=0).stop 2023-10-18T09:19:55.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.605+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 msgr2=0x7fc8e0194030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.605+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 0x7fc8e0194030 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800b710 tx=0x7fc8d800bb00 comp rx=0 tx=0).stop 2023-10-18T09:19:55.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 shutdown_connections 2023-10-18T09:19:55.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc8cc0a1500 0x7fc8cc0a39c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8e01061e0 0x7fc8e006c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc8e0100f80 0x7fc8e0194030 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 --2- 172.21.15.61:0/2157189592 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc8e01038b0 0x7fc8e0194570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 >> 172.21.15.61:0/2157189592 conn(0x7fc8e00fa970 msgr2=0x7fc8e01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 shutdown_connections 2023-10-18T09:19:55.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.606+0000 7fc8e559d700 1 -- 172.21.15.61:0/2157189592 wait complete. 2023-10-18T09:19:55.612 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 32 2023-10-18T09:19:55.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-18T09:19:55.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.902+0000 7f738759e700 1 -- 172.21.15.61:0/3135118124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388074830 msgr2=0x7f7388072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.902+0000 7f738759e700 1 --2- 172.21.15.61:0/3135118124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388074830 0x7f7388072c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7378009990 tx=0x7f7378009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 -- 172.21.15.61:0/3135118124 shutdown_connections 2023-10-18T09:19:55.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 --2- 172.21.15.61:0/3135118124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 0x7f738810f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 --2- 172.21.15.61:0/3135118124 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388074830 0x7f7388072c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 --2- 172.21.15.61:0/3135118124 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388073250 0x7f73880736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 -- 172.21.15.61:0/3135118124 >> 172.21.15.61:0/3135118124 conn(0x7f73880fbbd0 msgr2=0x7f73880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 -- 172.21.15.61:0/3135118124 shutdown_connections 2023-10-18T09:19:55.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.903+0000 7f738759e700 1 -- 172.21.15.61:0/3135118124 wait complete. 2023-10-18T09:19:55.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 Processor -- start 2023-10-18T09:19:55.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 -- start start 2023-10-18T09:19:55.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 0x7f738819d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 0x7f73881a4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881a4720 con 0x7f7388073250 2023-10-18T09:19:55.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881a4860 con 0x7f7388074830 2023-10-18T09:19:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881a49f0 con 0x7f738810cc10 2023-10-18T09:19:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34950/0 (socket says 172.21.15.61:34950) 2023-10-18T09:19:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f738659c700 1 -- 172.21.15.61:0/4064863542 learned_addr learned my addr 172.21.15.61:0/4064863542 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.904+0000 7f7385d9b700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 0x7f738819d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f7386d9d700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 0x7f73881a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 -- 172.21.15.61:0/4064863542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 msgr2=0x7f73881a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 0x7f73881a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 -- 172.21.15.61:0/4064863542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 msgr2=0x7f738819d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 0x7f738819d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 -- 172.21.15.61:0/4064863542 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73780095f0 con 0x7f7388073250 2023-10-18T09:19:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f7385d9b700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 0x7f738819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738659c700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f73777fe700 1 -- 172.21.15.61:0/4064863542 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737801d070 con 0x7f7388073250 2023-10-18T09:19:55.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f73777fe700 1 -- 172.21.15.61:0/4064863542 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7378021410 con 0x7f7388073250 2023-10-18T09:19:55.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73881a4c70 con 0x7f7388073250 2023-10-18T09:19:55.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.905+0000 7f73777fe700 1 -- 172.21.15.61:0/4064863542 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7378017650 con 0x7f7388073250 2023-10-18T09:19:55.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.906+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73881a50f0 con 0x7f7388073250 2023-10-18T09:19:55.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.910+0000 7f73777fe700 1 -- 172.21.15.61:0/4064863542 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7378021580 con 0x7f7388073250 2023-10-18T09:19:55.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.911+0000 7f73a0858b80 1 Processor -- start 2023-10-18T09:19:55.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.911+0000 7f73a0858b80 1 -- start start 2023-10-18T09:19:55.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.911+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73a0858b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179fe26e0 con 0x55717a06fe30 2023-10-18T09:19:55.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73a0858b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179fc8c60 con 0x55717a06f4d0 2023-10-18T09:19:55.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73a0858b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179fc6580 con 0x55717a0707f0 2023-10-18T09:19:55.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73777fe700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 0x7f73700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f73777fe700 1 -- 172.21.15.61:0/4064863542 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f73780ce380 con 0x7f7388073250 2023-10-18T09:19:55.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f7386d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f7386d9d700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 0x7f73700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34958/0 (socket says 172.21.15.61:34958) 2023-10-18T09:19:55.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 -- 172.21.15.61:0/1335290224 learned_addr learned my addr 172.21.15.61:0/1335290224 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f7385d9b700 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 -- 172.21.15.61:0/1335290224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 msgr2=0x55717a070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 -- 172.21.15.61:0/1335290224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 msgr2=0x55717a06f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a06f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.912+0000 7f738659c700 1 -- 172.21.15.61:0/1335290224 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557179f7f3f0 con 0x55717a06fe30 2023-10-18T09:19:55.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.913+0000 7f738659c700 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f73780a3150 tx=0x7f7378095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db4dc764318ce4a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.913+0000 7f7386d9d700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 0x7f73700a39c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7380009550 tx=0x7f738000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.913+0000 7f7374ff9700 1 -- 172.21.15.61:0/1335290224 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780ce170 con 0x55717a06fe30 2023-10-18T09:19:55.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.913+0000 7f7374ff9700 1 -- 172.21.15.61:0/1335290224 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737809ba80 con 0x55717a06fe30 2023-10-18T09:19:55.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.913+0000 7f7374ff9700 1 -- 172.21.15.61:0/1335290224 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737809b5e0 con 0x55717a06fe30 2023-10-18T09:19:55.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- 172.21.15.61:0/1335290224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 msgr2=0x55717a0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f73780a3150 tx=0x7f7378095f90 comp rx=0 tx=0).stop 2023-10-18T09:19:55.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- 172.21.15.61:0/1335290224 shutdown_connections 2023-10-18T09:19:55.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a06f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1335290224 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a0702b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- 172.21.15.61:0/1335290224 >> 172.21.15.61:0/1335290224 conn(0x557179fe3720 msgr2=0x55717a069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- 172.21.15.61:0/1335290224 shutdown_connections 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- 172.21.15.61:0/1335290224 wait complete. 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.914+0000 7f73a0858b80 1 -- start start 2023-10-18T09:19:55.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 Processor -- start 2023-10-18T09:19:55.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a078dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 0x55717a07fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179fc1e70 con 0x55717a06f4d0 2023-10-18T09:19:55.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179f5e710 con 0x55717a0707f0 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179f5e3b0 con 0x55717a06fe30 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f73a0858b80 10 client.0 _ll_register_callbacks cb 0x557179ea8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f7386d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 0x55717a07fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f738659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:34974/0 (socket says 172.21.15.61:34974) 2023-10-18T09:19:55.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f7385d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a078dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.915+0000 7f738659c700 1 -- 172.21.15.61:0/1374734934 learned_addr learned my addr 172.21.15.61:0/1374734934 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 -- 172.21.15.61:0/1374734934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 msgr2=0x55717a078dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a078dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 -- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 msgr2=0x55717a07fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 0x55717a07fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 -- 172.21.15.61:0/1374734934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73881a50f0 con 0x55717a06f4d0 2023-10-18T09:19:55.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f7385d9b700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a078dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:55.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f7386d9d700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 0x55717a07fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:55.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f738659c700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7378006010 tx=0x7f737809f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d595c4b364a27f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f736dffb700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:19:55.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780ce170 con 0x55717a06f4d0 2023-10-18T09:19:55.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f737809b380 con 0x55717a06f4d0 2023-10-18T09:19:55.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.916+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557179fc6ef0 con 0x55717a06f4d0 2023-10-18T09:19:55.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73780cbcc0 con 0x55717a06f4d0 2023-10-18T09:19:55.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 10 client.15837 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:55.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557179decde0 con 0x55717a06f4d0 2023-10-18T09:19:55.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557179fc6ef0 con 0x55717a06f4d0 2023-10-18T09:19:55.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f73780a1290 con 0x55717a06f4d0 2023-10-18T09:19:55.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:55.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 10 client.15837 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:19:55.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 20 client.15837 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:55.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0841b0 0x55717a0865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 4 client.15837 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:55.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55717a03e8b0 con 0x55717a0841b0 2023-10-18T09:19:55.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.917+0000 7f7385d9b700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0841b0 0x55717a0865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.918+0000 7f736dffb700 10 client.15837 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:55.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.918+0000 7f7385d9b700 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0841b0 0x55717a0865a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed3c5c85f6cf1637 server_cookie=dc0d1f0b52b14e08 in_seq=0 out_seq=0 2023-10-18T09:19:55.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f736dffb700 1 -- 172.21.15.61:0/1374734934 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55717a03e8b0 con 0x55717a0841b0 2023-10-18T09:19:55.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f736dffb700 10 client.15837 handle_command_reply: tid=0 2023-10-18T09:19:55.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 client.15837 shutdown 2023-10-18T09:19:55.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 msgr2=0x55717a078880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7378006010 tx=0x7f737809f3a0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 shutdown_connections 2023-10-18T09:19:55.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0841b0 0x55717a0865a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a0707f0 0x55717a07fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06f4d0 0x55717a078880 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 --2- 172.21.15.61:0/1374734934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a078dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f736dffb700 0 client.15837 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:55.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 >> 172.21.15.61:0/1374734934 conn(0x557179fe3720 msgr2=0x55717a06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 shutdown_connections 2023-10-18T09:19:55.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 1 -- 172.21.15.61:0/1374734934 wait complete. 2023-10-18T09:19:55.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.919+0000 7f73a0858b80 20 client.15837 trim_cache size 0 max 16384 2023-10-18T09:19:55.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 Processor -- start 2023-10-18T09:19:55.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 -- start start 2023-10-18T09:19:55.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a073c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 0x55717a076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179e42920 con 0x55717a0707f0 2023-10-18T09:19:55.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557179fc6ef0 con 0x55717a06f4d0 2023-10-18T09:19:55.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f73a0858b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a0663f0 con 0x55717a06fe30 2023-10-18T09:19:55.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f7385d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f7385d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41664/0 (socket says 172.21.15.61:41664) 2023-10-18T09:19:55.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f7385d9b700 1 -- 172.21.15.61:0/915212857 learned_addr learned my addr 172.21.15.61:0/915212857 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f7386d9d700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 0x55717a076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.933+0000 7f738659c700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 -- 172.21.15.61:0/915212857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 msgr2=0x55717a073c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a073c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 -- 172.21.15.61:0/915212857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 msgr2=0x55717a076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 0x55717a076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 -- 172.21.15.61:0/915212857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557179dd5c60 con 0x55717a06fe30 2023-10-18T09:19:55.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7386d9d700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 0x55717a076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:55.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f7385d9b700 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f737c00a010 tx=0x7f737c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c361f4fb9ab09a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f73757fa700 1 -- 172.21.15.61:0/915212857 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c017040 con 0x55717a06fe30 2023-10-18T09:19:55.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f73757fa700 1 -- 172.21.15.61:0/915212857 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737c018040 con 0x55717a06fe30 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.934+0000 7f73757fa700 1 -- 172.21.15.61:0/915212857 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c004a90 con 0x55717a06fe30 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- 172.21.15.61:0/915212857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 msgr2=0x55717a07fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f737c00a010 tx=0x7f737c009260 comp rx=0 tx=0).stop 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- 172.21.15.61:0/915212857 shutdown_connections 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a0707f0 0x55717a076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a073c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 --2- 172.21.15.61:0/915212857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a06fe30 0x55717a07fdc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- 172.21.15.61:0/915212857 >> 172.21.15.61:0/915212857 conn(0x557179fe3720 msgr2=0x55717a073520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- 172.21.15.61:0/915212857 shutdown_connections 2023-10-18T09:19:55.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- 172.21.15.61:0/915212857 wait complete. 2023-10-18T09:19:55.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 -- start start 2023-10-18T09:19:55.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.935+0000 7f73a0858b80 1 Processor -- start 2023-10-18T09:19:55.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a07f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a0d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a07f9c0 con 0x55717a06fe30 2023-10-18T09:19:55.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a0d84c0 con 0x55717a06f4d0 2023-10-18T09:19:55.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55717a0d8630 con 0x55717a0707f0 2023-10-18T09:19:55.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f73a0858b80 10 client.0 _ll_register_callbacks cb 0x557179ea8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:55.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f738659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f738659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46118/0 (socket says 172.21.15.61:46118) 2023-10-18T09:19:55.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f738659c700 1 -- 172.21.15.61:0/2484402230 learned_addr learned my addr 172.21.15.61:0/2484402230 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:55.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f7385d9b700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a07f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f7386d9d700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a0d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.936+0000 7f738659c700 1 -- 172.21.15.61:0/2484402230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 msgr2=0x55717a0d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f738659c700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a0d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f738659c700 1 -- 172.21.15.61:0/2484402230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 msgr2=0x55717a07f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:55.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f738659c700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a07f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f738659c700 1 -- 172.21.15.61:0/2484402230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c021a00 con 0x55717a06f4d0 2023-10-18T09:19:55.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f7386d9d700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a0d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:55.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f738659c700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7378012040 tx=0x7f73780b96b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601ab5ee72d30b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:55.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:55.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780ce170 con 0x55717a06f4d0 2023-10-18T09:19:55.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f737809f4d0 con 0x55717a06f4d0 2023-10-18T09:19:55.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f737809ba80 con 0x55717a06f4d0 2023-10-18T09:19:55.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7378097170 con 0x55717a06f4d0 2023-10-18T09:19:55.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f73a0858b80 10 client.25889 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:55.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557179e13250 con 0x55717a06f4d0 2023-10-18T09:19:55.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f737809ba80 con 0x55717a06f4d0 2023-10-18T09:19:55.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.937+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f73780a1290 con 0x55717a06f4d0 2023-10-18T09:19:55.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 10 client.25889 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 20 client.25889 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0d9ac0 0x55717a0dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 4 client.25889 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557179fbae50 con 0x55717a0d9ac0 2023-10-18T09:19:55.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f7385d9b700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0d9ac0 0x55717a0dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:55.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f7385d9b700 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0d9ac0 0x55717a0dbee0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df7f14bb225b6671 server_cookie=694382df797de753 in_seq=0 out_seq=0 2023-10-18T09:19:55.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f736dffb700 10 client.25889 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:55.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.938+0000 7f736dffb700 1 -- 172.21.15.61:0/2484402230 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557179fbae50 con 0x55717a0d9ac0 2023-10-18T09:19:55.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f736dffb700 10 client.25889 handle_command_reply: tid=0 2023-10-18T09:19:55.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 client.25889 shutdown 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 msgr2=0x55717a07ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7378012040 tx=0x7f73780b96b0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 shutdown_connections 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55717a0d9ac0 0x55717a0dbee0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55717a0707f0 0x55717a0d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55717a06f4d0 0x55717a07ede0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f736dffb700 0 client.25889 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:19:55.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 --2- 172.21.15.61:0/2484402230 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55717a06fe30 0x55717a07f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 >> 172.21.15.61:0/2484402230 conn(0x557179fe3720 msgr2=0x55717a078630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 shutdown_connections 2023-10-18T09:19:55.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 1 -- 172.21.15.61:0/2484402230 wait complete. 2023-10-18T09:19:55.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.939+0000 7f73a0858b80 20 client.25889 trim_cache size 0 max 16384 2023-10-18T09:19:55.961 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:55.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 msgr2=0x7f73700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 0x7f73700a39c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7380009550 tx=0x7f738000b040 comp rx=0 tx=0).stop 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 msgr2=0x7f738819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7378000c00 tx=0x7f7378005720 comp rx=0 tx=0).stop 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 shutdown_connections 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f73700a1500 0x7f73700a39c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f738810cc10 0x7f73881a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7388073250 0x7f738819cc40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 --2- 172.21.15.61:0/4064863542 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7388074830 0x7f738819d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:55.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 >> 172.21.15.61:0/4064863542 conn(0x7f73880fbbd0 msgr2=0x7f73880fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:55.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 shutdown_connections 2023-10-18T09:19:55.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:55.940+0000 7f738759e700 1 -- 172.21.15.61:0/4064863542 wait complete. 2023-10-18T09:19:55.965 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-18T09:19:56.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:56 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2157189592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:56.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 -- 172.21.15.61:0/1543645824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308101210 msgr2=0x7f8308103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 --2- 172.21.15.61:0/1543645824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308101210 0x7f8308103600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009f90 tx=0x7f82f800b330 comp rx=0 tx=0).stop 2023-10-18T09:19:56.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 -- 172.21.15.61:0/1543645824 shutdown_connections 2023-10-18T09:19:56.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 --2- 172.21.15.61:0/1543645824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308106470 0x7f83081088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 --2- 172.21.15.61:0/1543645824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308101210 0x7f8308103600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 --2- 172.21.15.61:0/1543645824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8308103b40 0x7f8308105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.282+0000 7f830c9ff700 1 -- 172.21.15.61:0/1543645824 >> 172.21.15.61:0/1543645824 conn(0x7f83080fac00 msgr2=0x7f83080fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.283+0000 7f830c9ff700 1 -- 172.21.15.61:0/1543645824 shutdown_connections 2023-10-18T09:19:56.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.283+0000 7f830c9ff700 1 -- 172.21.15.61:0/1543645824 wait complete. 2023-10-18T09:19:56.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.285+0000 7f830c9ff700 1 Processor -- start 2023-10-18T09:19:56.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.285+0000 7f830c9ff700 1 -- start start 2023-10-18T09:19:56.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8308101210 0x7f8308198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308106470 0x7f830819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a0440 con 0x7f8308106470 2023-10-18T09:19:56.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a0580 con 0x7f8308101210 2023-10-18T09:19:56.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f830c9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a0710 con 0x7f8308103b40 2023-10-18T09:19:56.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41688/0 (socket says 172.21.15.61:41688) 2023-10-18T09:19:56.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f8305d9b700 1 -- 172.21.15.61:0/3002374674 learned_addr learned my addr 172.21.15.61:0/3002374674 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.286+0000 7f8306d9d700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308106470 0x7f830819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.287+0000 7f8305d9b700 1 -- 172.21.15.61:0/3002374674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8308101210 msgr2=0x7f8308198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.287+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8308101210 0x7f8308198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.287+0000 7f8305d9b700 1 -- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308106470 msgr2=0x7f830819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.287+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308106470 0x7f830819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.287+0000 7f8305d9b700 1 -- 172.21.15.61:0/3002374674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f8009c40 con 0x7f8308103b40 2023-10-18T09:19:56.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.293+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9c0 tx=0x7f82fc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.301+0000 7f82f77fe700 1 -- 172.21.15.61:0/3002374674 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00cbd0 con 0x7f8308103b40 2023-10-18T09:19:56.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.301+0000 7f82f77fe700 1 -- 172.21.15.61:0/3002374674 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc00eeb0 con 0x7f8308103b40 2023-10-18T09:19:56.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.301+0000 7f82f77fe700 1 -- 172.21.15.61:0/3002374674 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc005580 con 0x7f8308103b40 2023-10-18T09:19:56.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.301+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081a09f0 con 0x7f8308103b40 2023-10-18T09:19:56.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.302+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a0eb0 con 0x7f8308103b40 2023-10-18T09:19:56.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.311+0000 7f82f77fe700 1 -- 172.21.15.61:0/3002374674 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f82fc004750 con 0x7f8308103b40 2023-10-18T09:19:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.313+0000 7f82f77fe700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 0x7f82f00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.313+0000 7f82f77fe700 1 -- 172.21.15.61:0/3002374674 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f82fc014070 con 0x7f8308103b40 2023-10-18T09:19:56.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.313+0000 7f83202d1b80 1 Processor -- start 2023-10-18T09:19:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 -- start start 2023-10-18T09:19:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96fff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb96696e0 con 0x559eb96ffae0 2023-10-18T09:19:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.314+0000 7f83202d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb964fc60 con 0x559eb9700440 2023-10-18T09:19:56.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.315+0000 7f83202d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb964d580 con 0x559eb96fede0 2023-10-18T09:19:56.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.316+0000 7f830659c700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 0x7f82f00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.316+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.316+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46128/0 (socket says 172.21.15.61:46128) 2023-10-18T09:19:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.316+0000 7f8305d9b700 1 -- 172.21.15.61:0/3390240781 learned_addr learned my addr 172.21.15.61:0/3390240781 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.317+0000 7f8306d9d700 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.317+0000 7f830659c700 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 -- 172.21.15.61:0/3390240781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 msgr2=0x559eb96ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 -- 172.21.15.61:0/3390240781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 msgr2=0x559eb96fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 -- 172.21.15.61:0/3390240781 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb96063f0 con 0x559eb9700440 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f830659c700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 0x7f82f00a3a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007ba0 tx=0x7f82f8009620 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f8305d9b700 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0a2130 tx=0x7f82fc099f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=453d9a9f09ef5c95 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.318+0000 7f82f4ff9700 1 -- 172.21.15.61:0/3390240781 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0cd160 con 0x559eb9700440 2023-10-18T09:19:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.319+0000 7f82f4ff9700 1 -- 172.21.15.61:0/3390240781 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc09a2d0 con 0x559eb9700440 2023-10-18T09:19:56.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.319+0000 7f82f4ff9700 1 -- 172.21.15.61:0/3390240781 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82fc09a790 con 0x559eb9700440 2023-10-18T09:19:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 -- 172.21.15.61:0/3390240781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 msgr2=0x559eb96fe8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0a2130 tx=0x7f82fc099f40 comp rx=0 tx=0).stop 2023-10-18T09:19:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 -- 172.21.15.61:0/3390240781 shutdown_connections 2023-10-18T09:19:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 --2- 172.21.15.61:0/3390240781 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fe8a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 -- 172.21.15.61:0/3390240781 >> 172.21.15.61:0/3390240781 conn(0x559eb966a720 msgr2=0x559eb96f0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.320+0000 7f83202d1b80 1 -- 172.21.15.61:0/3390240781 shutdown_connections 2023-10-18T09:19:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.321+0000 7f83202d1b80 1 -- 172.21.15.61:0/3390240781 wait complete. 2023-10-18T09:19:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.321+0000 7f83202d1b80 1 -- start start 2023-10-18T09:19:56.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.321+0000 7f83202d1b80 1 Processor -- start 2023-10-18T09:19:56.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.323+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96fede0 0x559eb96f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.323+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 0x559eb96f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.323+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.323+0000 7f83202d1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb9648e70 con 0x559eb96fede0 2023-10-18T09:19:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f8306d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f8306d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46132/0 (socket says 172.21.15.61:46132) 2023-10-18T09:19:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f8306d9d700 1 -- 172.21.15.61:0/2914154276 learned_addr learned my addr 172.21.15.61:0/2914154276 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f8305d9b700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 0x559eb96f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f830659c700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96fede0 0x559eb96f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb95e5710 con 0x559eb9700440 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb95e53b0 con 0x559eb96ffae0 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.324+0000 7f8306d9d700 1 -- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 msgr2=0x559eb96f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f8306d9d700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 0x559eb96f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f8306d9d700 1 -- 172.21.15.61:0/2914154276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96fede0 msgr2=0x559eb96f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f8306d9d700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96fede0 0x559eb96f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f8306d9d700 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081a0eb0 con 0x559eb9700440 2023-10-18T09:19:56.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f8305d9b700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 0x559eb96f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:56.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.325+0000 7f83202d1b80 10 client.0 _ll_register_callbacks cb 0x559eb952fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:56.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.326+0000 7f8306d9d700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8300003930 tx=0x7f8300003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c51e88b59d6222 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.326+0000 7f82edffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:56.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.326+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eb964def0 con 0x559eb9700440 2023-10-18T09:19:56.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.326+0000 7f82edffb700 1 -- 172.21.15.61:0/2914154276 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300004290 con 0x559eb9700440 2023-10-18T09:19:56.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.326+0000 7f82edffb700 1 -- 172.21.15.61:0/2914154276 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f830001b440 con 0x559eb9700440 2023-10-18T09:19:56.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.328+0000 7f82edffb700 1 -- 172.21.15.61:0/2914154276 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830001b600 con 0x559eb9700440 2023-10-18T09:19:56.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.328+0000 7f83202d1b80 10 client.25898 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:56.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.328+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=32}) v3 -- 0x559eb9473de0 con 0x559eb9700440 2023-10-18T09:19:56.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f82edffb700 1 -- 172.21.15.61:0/2914154276 <== mon.2 v2:172.21.15.110:3300/0 4 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f830001b760 con 0x559eb9700440 2023-10-18T09:19:56.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:56.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 10 client.25898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:19:56.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 20 client.25898 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:56.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb971c570 0x559eb971e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 4 client.25898 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:56.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.329+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eb96c58b0 con 0x559eb971c570 2023-10-18T09:19:56.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.331+0000 7f830659c700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb971c570 0x559eb971e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.332+0000 7f830659c700 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb971c570 0x559eb971e9b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1317a4009dc1fee1 server_cookie=163978475529b39e in_seq=0 out_seq=0 2023-10-18T09:19:56.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.332+0000 7f82edffb700 10 client.25898 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:56.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.333+0000 7f82edffb700 1 -- 172.21.15.61:0/2914154276 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559eb96c58b0 con 0x559eb971c570 2023-10-18T09:19:56.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.333+0000 7f82edffb700 10 client.25898 handle_command_reply: tid=0 2023-10-18T09:19:56.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.333+0000 7f83202d1b80 1 client.25898 shutdown 2023-10-18T09:19:56.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 msgr2=0x559eb96fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8300003930 tx=0x7f8300003cf0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 shutdown_connections 2023-10-18T09:19:56.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96fd1a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96fede0 0x559eb96f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb971c570 0x559eb971e9b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 --2- 172.21.15.61:0/2914154276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96ffae0 0x559eb96f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 >> 172.21.15.61:0/2914154276 conn(0x559eb966a720 msgr2=0x559eb9702ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f82edffb700 0 client.25898 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:56.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 shutdown_connections 2023-10-18T09:19:56.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.334+0000 7f83202d1b80 1 -- 172.21.15.61:0/2914154276 wait complete. 2023-10-18T09:19:56.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.341+0000 7f83202d1b80 20 client.25898 trim_cache size 0 max 16384 2023-10-18T09:19:56.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.374+0000 7f83202d1b80 1 Processor -- start 2023-10-18T09:19:56.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.375+0000 7f83202d1b80 1 -- start start 2023-10-18T09:19:56.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb94c9920 con 0x559eb96ffae0 2023-10-18T09:19:56.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb96f10c0 con 0x559eb9700440 2023-10-18T09:19:56.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.376+0000 7f83202d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb96f2b80 con 0x559eb96fede0 2023-10-18T09:19:56.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.378+0000 7f830659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.378+0000 7f830659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41706/0 (socket says 172.21.15.61:41706) 2023-10-18T09:19:56.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.378+0000 7f830659c700 1 -- 172.21.15.61:0/711579971 learned_addr learned my addr 172.21.15.61:0/711579971 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.378+0000 7f8306d9d700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.378+0000 7f8305d9b700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.379+0000 7f8306d9d700 1 -- 172.21.15.61:0/711579971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 msgr2=0x559eb96ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.379+0000 7f8306d9d700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.379+0000 7f8306d9d700 1 -- 172.21.15.61:0/711579971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 msgr2=0x559eb96f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.379+0000 7f8306d9d700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.379+0000 7f8306d9d700 1 -- 172.21.15.61:0/711579971 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eb945cc60 con 0x559eb9700440 2023-10-18T09:19:56.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.380+0000 7f830659c700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:56.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.380+0000 7f8306d9d700 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96f7c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8300005950 tx=0x7f8300015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1833bd8d792cf89 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.380+0000 7f82f57fa700 1 -- 172.21.15.61:0/711579971 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300004290 con 0x559eb9700440 2023-10-18T09:19:56.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.381+0000 7f82f57fa700 1 -- 172.21.15.61:0/711579971 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559eb964def0 con 0x559eb9700440 2023-10-18T09:19:56.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.381+0000 7f82f57fa700 1 -- 172.21.15.61:0/711579971 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300004c90 con 0x559eb9700440 2023-10-18T09:19:56.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 -- 172.21.15.61:0/711579971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 msgr2=0x559eb96f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96f7c30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8300005950 tx=0x7f8300015000 comp rx=0 tx=0).stop 2023-10-18T09:19:56.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 -- 172.21.15.61:0/711579971 shutdown_connections 2023-10-18T09:19:56.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb9700440 0x559eb96f7c30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb96ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 --2- 172.21.15.61:0/711579971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb96ffae0 0x559eb96f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 -- 172.21.15.61:0/711579971 >> 172.21.15.61:0/711579971 conn(0x559eb966a720 msgr2=0x559eb9723350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.382+0000 7f83202d1b80 1 -- 172.21.15.61:0/711579971 shutdown_connections 2023-10-18T09:19:56.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.383+0000 7f83202d1b80 1 -- 172.21.15.61:0/711579971 wait complete. 2023-10-18T09:19:56.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.383+0000 7f83202d1b80 1 -- start start 2023-10-18T09:19:56.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.384+0000 7f83202d1b80 1 Processor -- start 2023-10-18T09:19:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:56 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2157189592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:56.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.389+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 0x559eb9768b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 0x559eb976f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb976fa60 con 0x559eb9700440 2023-10-18T09:19:56.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb976fba0 con 0x559eb96ffae0 2023-10-18T09:19:56.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eb976fd60 con 0x559eb96fede0 2023-10-18T09:19:56.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.390+0000 7f83202d1b80 10 client.0 _ll_register_callbacks cb 0x559eb952fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:56.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.391+0000 7f8305d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 0x559eb9768b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.392+0000 7f830659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.392+0000 7f8306d9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 0x559eb976f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.392+0000 7f830659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41718/0 (socket says 172.21.15.61:41718) 2023-10-18T09:19:56.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.392+0000 7f830659c700 1 -- 172.21.15.61:0/1500949725 learned_addr learned my addr 172.21.15.61:0/1500949725 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 -- 172.21.15.61:0/1500949725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 msgr2=0x559eb9768b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 0x559eb9768b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 -- 172.21.15.61:0/1500949725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 msgr2=0x559eb976f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 0x559eb976f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 -- 172.21.15.61:0/1500949725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081094d0 con 0x559eb96fede0 2023-10-18T09:19:56.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f8305d9b700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 0x559eb9768b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:56.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f8306d9d700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 0x559eb976f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:56.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.393+0000 7f830659c700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82f8004bd0 tx=0x7f82f800b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b03848efc05a53 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.394+0000 7f82edffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:56.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.394+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800e040 con 0x559eb96fede0 2023-10-18T09:19:56.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.394+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f82f80075e0 con 0x559eb96fede0 2023-10-18T09:19:56.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.394+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82fc09a2d0 con 0x559eb96fede0 2023-10-18T09:19:56.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.395+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82f80077a0 con 0x559eb96fede0 2023-10-18T09:19:56.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.395+0000 7f83202d1b80 10 client.25918 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:56.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.395+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559eb9473de0 con 0x559eb96fede0 2023-10-18T09:19:56.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82fc09a2d0 con 0x559eb96fede0 2023-10-18T09:19:56.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f82f8029930 con 0x559eb96fede0 2023-10-18T09:19:56.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 10 client.25918 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:56.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 10 client.25918 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:19:56.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 20 client.25918 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:56.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb9771810 0x559eb9773c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 4 client.25918 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:56.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.397+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559eb9641e50 con 0x559eb9771810 2023-10-18T09:19:56.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.398+0000 7f8305d9b700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb9771810 0x559eb9773c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.398+0000 7f8305d9b700 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb9771810 0x559eb9773c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4863bb9df613eb9 server_cookie=3fca5369792bddba in_seq=0 out_seq=0 2023-10-18T09:19:56.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.398+0000 7f82edffb700 10 client.25918 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:56.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.399+0000 7f82edffb700 1 -- 172.21.15.61:0/1500949725 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559eb9641e50 con 0x559eb9771810 2023-10-18T09:19:56.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.399+0000 7f82edffb700 10 client.25918 handle_command_reply: tid=0 2023-10-18T09:19:56.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.399+0000 7f83202d1b80 1 client.25918 shutdown 2023-10-18T09:19:56.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 msgr2=0x559eb9768620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82f8004bd0 tx=0x7f82f800b680 comp rx=0 tx=0).stop 2023-10-18T09:19:56.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 shutdown_connections 2023-10-18T09:19:56.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559eb9700440 0x559eb976f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x559eb9771810 0x559eb9773c30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559eb96fede0 0x559eb9768620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f82edffb700 0 client.25918 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:19:56.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 --2- 172.21.15.61:0/1500949725 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559eb96ffae0 0x559eb9768b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.400+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 >> 172.21.15.61:0/1500949725 conn(0x559eb966a720 msgr2=0x559eb9706070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.401+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 shutdown_connections 2023-10-18T09:19:56.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.401+0000 7f83202d1b80 1 -- 172.21.15.61:0/1500949725 wait complete. 2023-10-18T09:19:56.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.401+0000 7f83202d1b80 20 client.25918 trim_cache size 0 max 16384 2023-10-18T09:19:56.408 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:56.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 msgr2=0x7f82f00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 0x7f82f00a3a90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007ba0 tx=0x7f82f8009620 comp rx=0 tx=0).stop 2023-10-18T09:19:56.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 msgr2=0x7f8308198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00e9c0 tx=0x7f82fc00ecd0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 shutdown_connections 2023-10-18T09:19:56.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f82f00a15d0 0x7f82f00a3a90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8308106470 0x7f830819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8308101210 0x7f8308198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 --2- 172.21.15.61:0/3002374674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8308103b40 0x7f8308198e50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 >> 172.21.15.61:0/3002374674 conn(0x7f83080fac00 msgr2=0x7f8308100930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.402+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 shutdown_connections 2023-10-18T09:19:56.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.403+0000 7f830c9ff700 1 -- 172.21.15.61:0/3002374674 wait complete. 2023-10-18T09:19:56.437 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-18T09:19:56.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:56 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2157189592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:19:56.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.808+0000 7f4689d9b700 1 -- 172.21.15.61:0/2089382392 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674003be0 con 0x7f468c103a20 2023-10-18T09:19:56.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 -- 172.21.15.61:0/2089382392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c103a20 msgr2=0x7f468c103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 --2- 172.21.15.61:0/2089382392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c103a20 0x7f468c103e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4674009990 tx=0x7f4674009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 -- 172.21.15.61:0/2089382392 shutdown_connections 2023-10-18T09:19:56.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 --2- 172.21.15.61:0/2089382392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c1043c0 0x7f468c10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 --2- 172.21.15.61:0/2089382392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f468c102830 0x7f468c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 --2- 172.21.15.61:0/2089382392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c103a20 0x7f468c103e80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 -- 172.21.15.61:0/2089382392 >> 172.21.15.61:0/2089382392 conn(0x7f468c0fddc0 msgr2=0x7f468c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 -- 172.21.15.61:0/2089382392 shutdown_connections 2023-10-18T09:19:56.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.809+0000 7f468b59e700 1 -- 172.21.15.61:0/2089382392 wait complete. 2023-10-18T09:19:56.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.811+0000 7f468b59e700 1 Processor -- start 2023-10-18T09:19:56.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.811+0000 7f468b59e700 1 -- start start 2023-10-18T09:19:56.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 0x7f468c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 0x7f468c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f468c1043c0 0x7f468c1a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c1a46c0 con 0x7f468c1043c0 2023-10-18T09:19:56.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c1a4800 con 0x7f468c103a20 2023-10-18T09:19:56.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468b59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c1a4990 con 0x7f468c102830 2023-10-18T09:19:56.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 0x7f468c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 0x7f468c19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41744/0 (socket says 172.21.15.61:41744) 2023-10-18T09:19:56.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 0x7f468c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.812+0000 7f468a59c700 1 -- 172.21.15.61:0/491716307 learned_addr learned my addr 172.21.15.61:0/491716307 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 -- 172.21.15.61:0/491716307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 msgr2=0x7f468c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 0x7f468c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 -- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f468c1043c0 msgr2=0x7f468c1a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f468c1043c0 0x7f468c1a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.813+0000 7f4683fff700 1 -- 172.21.15.61:0/491716307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740095f0 con 0x7f468c103a20 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.814+0000 7f4683fff700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 0x7f468c19d0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46740098e0 tx=0x7f4674003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.814+0000 7f46837fe700 1 -- 172.21.15.61:0/491716307 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467401d070 con 0x7f468c103a20 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.814+0000 7f46837fe700 1 -- 172.21.15.61:0/491716307 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4674003e30 con 0x7f468c103a20 2023-10-18T09:19:56.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.814+0000 7f46837fe700 1 -- 172.21.15.61:0/491716307 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467400fe10 con 0x7f468c103a20 2023-10-18T09:19:56.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.815+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c1a4c10 con 0x7f468c103a20 2023-10-18T09:19:56.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.816+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1a4fe0 con 0x7f468c103a20 2023-10-18T09:19:56.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.822+0000 7f46837fe700 1 -- 172.21.15.61:0/491716307 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f467400f560 con 0x7f468c103a20 2023-10-18T09:19:56.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.824+0000 7f46a4456b80 1 Processor -- start 2023-10-18T09:19:56.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.825+0000 7f46837fe700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 0x7f46780a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.825+0000 7f46837fe700 1 -- 172.21.15.61:0/491716307 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f46740cdfc0 con 0x7f468c103a20 2023-10-18T09:19:56.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.826+0000 7f468ad9d700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 0x7f46780a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.826+0000 7f46a4456b80 1 -- start start 2023-10-18T09:19:56.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b29e30 0x55e748b2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b2ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a9c6e0 con 0x55e748b294d0 2023-10-18T09:19:56.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a82c60 con 0x55e748b29e30 2023-10-18T09:19:56.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.827+0000 7f46a4456b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a80580 con 0x55e748b2a7f0 2023-10-18T09:19:56.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.828+0000 7f468a59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.828+0000 7f468a59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35042/0 (socket says 172.21.15.61:35042) 2023-10-18T09:19:56.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.828+0000 7f468a59c700 1 -- 172.21.15.61:0/3430572360 learned_addr learned my addr 172.21.15.61:0/3430572360 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.828+0000 7f468ad9d700 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b2ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 -- 172.21.15.61:0/3430572360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 msgr2=0x55e748b2ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b2ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 -- 172.21.15.61:0/3430572360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b29e30 msgr2=0x55e748b2a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b29e30 0x55e748b2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 -- 172.21.15.61:0/3430572360 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e748a393f0 con 0x55e748b294d0 2023-10-18T09:19:56.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468ad9d700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 0x7f46780a3a90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46840060b0 tx=0x7f46840077e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.829+0000 7f468a59c700 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f467c009de0 tx=0x7f467c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ebc955e830e6ad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.830+0000 7f4680ff9700 1 -- 172.21.15.61:0/3430572360 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c009680 con 0x55e748b294d0 2023-10-18T09:19:56.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.830+0000 7f4680ff9700 1 -- 172.21.15.61:0/3430572360 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c013470 con 0x55e748b294d0 2023-10-18T09:19:56.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f4680ff9700 1 -- 172.21.15.61:0/3430572360 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0125f0 con 0x55e748b294d0 2023-10-18T09:19:56.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 -- 172.21.15.61:0/3430572360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 msgr2=0x55e748b298f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f467c009de0 tx=0x7f467c00e4b0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 -- 172.21.15.61:0/3430572360 shutdown_connections 2023-10-18T09:19:56.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b2ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b294d0 0x55e748b298f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 --2- 172.21.15.61:0/3430572360 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b29e30 0x55e748b2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.831+0000 7f46a4456b80 1 -- 172.21.15.61:0/3430572360 >> 172.21.15.61:0/3430572360 conn(0x55e748a9d720 msgr2=0x55e748b23d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.832+0000 7f46a4456b80 1 -- 172.21.15.61:0/3430572360 shutdown_connections 2023-10-18T09:19:56.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.832+0000 7f46a4456b80 1 -- 172.21.15.61:0/3430572360 wait complete. 2023-10-18T09:19:56.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.832+0000 7f46a4456b80 1 -- start start 2023-10-18T09:19:56.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.833+0000 7f46a4456b80 1 Processor -- start 2023-10-18T09:19:56.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b32e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b3a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a7be70 con 0x55e748b29e30 2023-10-18T09:19:56.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a18710 con 0x55e748b294d0 2023-10-18T09:19:56.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.834+0000 7f46a4456b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748a183b0 con 0x55e748b2a7f0 2023-10-18T09:19:56.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.835+0000 7f46a4456b80 10 client.0 _ll_register_callbacks cb 0x55e748962a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:56.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.836+0000 7f468a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b32e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.836+0000 7f4683fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.836+0000 7f4683fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35048/0 (socket says 172.21.15.61:35048) 2023-10-18T09:19:56.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.836+0000 7f4683fff700 1 -- 172.21.15.61:0/2414543762 learned_addr learned my addr 172.21.15.61:0/2414543762 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 -- 172.21.15.61:0/2414543762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 msgr2=0x55e748b3a390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:19:56.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b3a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 -- 172.21.15.61:0/2414543762 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 msgr2=0x55e748b33350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 -- 172.21.15.61:0/2414543762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c0097e0 con 0x55e748b294d0 2023-10-18T09:19:56.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f4683fff700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:56.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.837+0000 7f468a59c700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b32e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50ef76f9ef4ef5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.838+0000 7f4671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:56.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.838+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c01c860 con 0x55e748b294d0 2023-10-18T09:19:56.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.838+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f467c01c9c0 con 0x55e748b294d0 2023-10-18T09:19:56.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.838+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e748a80ef0 con 0x55e748b294d0 2023-10-18T09:19:56.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c013be0 con 0x55e748b294d0 2023-10-18T09:19:56.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f46a4456b80 10 client.25913 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:56.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e7488a6de0 con 0x55e748b294d0 2023-10-18T09:19:56.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e748a80ef0 con 0x55e748b294d0 2023-10-18T09:19:56.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f467c024030 con 0x55e748b294d0 2023-10-18T09:19:56.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f46a4456b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:56.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.839+0000 7f46a4456b80 10 client.25913 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:19:56.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f46a4456b80 20 client.25913 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:56.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748b4fa10 0x55e748b51e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f46a4456b80 4 client.25913 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:56.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e748af88b0 con 0x55e748b4fa10 2023-10-18T09:19:56.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f4683fff700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748b4fa10 0x55e748b51e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f4671ffb700 10 client.25913 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:56.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.840+0000 7f4683fff700 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748b4fa10 0x55e748b51e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6bea442612c4b5d server_cookie=531b447379ea3fbb in_seq=0 out_seq=0 2023-10-18T09:19:56.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.841+0000 7f4671ffb700 1 -- 172.21.15.61:0/2414543762 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e748af88b0 con 0x55e748b4fa10 2023-10-18T09:19:56.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.841+0000 7f4671ffb700 10 client.25913 handle_command_reply: tid=0 2023-10-18T09:19:56.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.841+0000 7f46a4456b80 1 client.25913 shutdown 2023-10-18T09:19:56.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 msgr2=0x55e748b32e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b32e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f467c000c00 tx=0x7f467c00ecf0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 shutdown_connections 2023-10-18T09:19:56.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748b4fa10 0x55e748b51e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b2a7f0 0x55e748b3a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b32e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2414543762 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b29e30 0x55e748b33350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f4671ffb700 0 client.25913 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.842+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 >> 172.21.15.61:0/2414543762 conn(0x55e748a9d720 msgr2=0x55e748b282d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.843+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 shutdown_connections 2023-10-18T09:19:56.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.843+0000 7f46a4456b80 1 -- 172.21.15.61:0/2414543762 wait complete. 2023-10-18T09:19:56.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.844+0000 7f46a4456b80 20 client.25913 trim_cache size 0 max 16384 2023-10-18T09:19:56.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.870+0000 7f46a4456b80 1 Processor -- start 2023-10-18T09:19:56.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.870+0000 7f46a4456b80 1 -- start start 2023-10-18T09:19:56.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7488fc920 con 0x55e748b2a7f0 2023-10-18T09:19:56.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748b24910 con 0x55e748b294d0 2023-10-18T09:19:56.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f46a4456b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748b29c70 con 0x55e748b29e30 2023-10-18T09:19:56.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f468a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.872+0000 7f4683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.873+0000 7f4683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41758/0 (socket says 172.21.15.61:41758) 2023-10-18T09:19:56.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.873+0000 7f4683fff700 1 -- 172.21.15.61:0/2359828454 learned_addr learned my addr 172.21.15.61:0/2359828454 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.873+0000 7f468ad9d700 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b1e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 -- 172.21.15.61:0/2359828454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 msgr2=0x55e748b3a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 -- 172.21.15.61:0/2359828454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 msgr2=0x55e748b1e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b1e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 -- 172.21.15.61:0/2359828454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e74888fc60 con 0x55e748b294d0 2023-10-18T09:19:56.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f4683fff700 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:56.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.874+0000 7f468a59c700 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b3aa40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f467c009030 tx=0x7f467c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf89c87c7f6f0edc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.875+0000 7f46817fa700 1 -- 172.21.15.61:0/2359828454 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c013d90 con 0x55e748b294d0 2023-10-18T09:19:56.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.875+0000 7f46817fa700 1 -- 172.21.15.61:0/2359828454 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c020070 con 0x55e748b294d0 2023-10-18T09:19:56.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.875+0000 7f46817fa700 1 -- 172.21.15.61:0/2359828454 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c013d90 con 0x55e748b294d0 2023-10-18T09:19:56.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.876+0000 7f46a4456b80 1 -- 172.21.15.61:0/2359828454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 msgr2=0x55e748b3aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.876+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b3aa40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f467c009030 tx=0x7f467c0046d0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.876+0000 7f46a4456b80 1 -- 172.21.15.61:0/2359828454 shutdown_connections 2023-10-18T09:19:56.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b1e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b3aa40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2359828454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b3a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 -- 172.21.15.61:0/2359828454 >> 172.21.15.61:0/2359828454 conn(0x55e748a9d720 msgr2=0x55e748b2abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 -- 172.21.15.61:0/2359828454 shutdown_connections 2023-10-18T09:19:56.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.877+0000 7f46a4456b80 1 -- 172.21.15.61:0/2359828454 wait complete. 2023-10-18T09:19:56.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.878+0000 7f46a4456b80 1 -- start start 2023-10-18T09:19:56.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.879+0000 7f46a4456b80 1 Processor -- start 2023-10-18T09:19:56.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748b39be0 con 0x55e748b2a7f0 2023-10-18T09:19:56.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748b39d50 con 0x55e748b294d0 2023-10-18T09:19:56.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e748b39ec0 con 0x55e748b29e30 2023-10-18T09:19:56.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f46a4456b80 10 client.0 _ll_register_callbacks cb 0x55e748962a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:56.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f4683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.883+0000 7f4683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41772/0 (socket says 172.21.15.61:41772) 2023-10-18T09:19:56.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.882+0000 7f468a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.883+0000 7f4683fff700 1 -- 172.21.15.61:0/2143228643 learned_addr learned my addr 172.21.15.61:0/2143228643 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:56.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.883+0000 7f468ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f4683fff700 1 -- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 msgr2=0x55e748b388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f4683fff700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f4683fff700 1 -- 172.21.15.61:0/2143228643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 msgr2=0x55e748b396a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:56.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f4683fff700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b396a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f4683fff700 1 -- 172.21.15.61:0/2143228643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1a4fe0 con 0x55e748b29e30 2023-10-18T09:19:56.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.884+0000 7f468a59c700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:56.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.885+0000 7f4683fff700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4674097a90 tx=0x7f4674009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1cb7b40650f5d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:56.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.885+0000 7f4671ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:56.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.887+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740b9070 con 0x55e748b29e30 2023-10-18T09:19:56.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.887+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f46740a3320 con 0x55e748b29e30 2023-10-18T09:19:56.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.887+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f467c013470 con 0x55e748b29e30 2023-10-18T09:19:56.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.888+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46740b3070 con 0x55e748b29e30 2023-10-18T09:19:56.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.888+0000 7f46a4456b80 10 client.25933 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:56.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.888+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e7488a6de0 con 0x55e748b29e30 2023-10-18T09:19:56.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.888+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f467c013470 con 0x55e748b29e30 2023-10-18T09:19:56.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.888+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f46740a2e90 con 0x55e748b29e30 2023-10-18T09:19:56.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 10 client.25933 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:56.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 10 client.25933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:19:56.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 20 client.25933 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:56.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748ba4420 0x55e748ba6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:56.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 4 client.25933 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:56.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e748a74e50 con 0x55e748ba4420 2023-10-18T09:19:56.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.889+0000 7f468a59c700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748ba4420 0x55e748ba6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:56.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.890+0000 7f468a59c700 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748ba4420 0x55e748ba6840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffe59d5df7ea6fcb server_cookie=2e62e27202790e6 in_seq=0 out_seq=0 2023-10-18T09:19:56.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.890+0000 7f4671ffb700 10 client.25933 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:56.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f4671ffb700 1 -- 172.21.15.61:0/2143228643 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e748a74e50 con 0x55e748ba4420 2023-10-18T09:19:56.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f4671ffb700 10 client.25933 handle_command_reply: tid=0 2023-10-18T09:19:56.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 client.25933 shutdown 2023-10-18T09:19:56.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 msgr2=0x55e748b38de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4674097a90 tx=0x7f4674009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 shutdown_connections 2023-10-18T09:19:56.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x55e748ba4420 0x55e748ba6840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f4671ffb700 0 client.25933 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:19:56.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e748b2a7f0 0x55e748b396a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e748b294d0 0x55e748b388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 --2- 172.21.15.61:0/2143228643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e748b29e30 0x55e748b38de0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.891+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 >> 172.21.15.61:0/2143228643 conn(0x55e748a9d720 msgr2=0x55e748b30d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.892+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 shutdown_connections 2023-10-18T09:19:56.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.892+0000 7f46a4456b80 1 -- 172.21.15.61:0/2143228643 wait complete. 2023-10-18T09:19:56.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.893+0000 7f46a4456b80 20 client.25933 trim_cache size 0 max 16384 2023-10-18T09:19:56.899 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:56.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.899+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 msgr2=0x7f46780a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.899+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 0x7f46780a3a90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f46840060b0 tx=0x7f46840077e0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.899+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 msgr2=0x7f468c19d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:56.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.899+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 0x7f468c19d0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46740098e0 tx=0x7f4674003b90 comp rx=0 tx=0).stop 2023-10-18T09:19:56.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 shutdown_connections 2023-10-18T09:19:56.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f46780a15d0 0x7f46780a3a90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f468c1043c0 0x7f468c1a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f468c102830 0x7f468c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 --2- 172.21.15.61:0/491716307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f468c103a20 0x7f468c19d0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:56.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.900+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 >> 172.21.15.61:0/491716307 conn(0x7f468c0fddc0 msgr2=0x7f468c076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:56.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.901+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 shutdown_connections 2023-10-18T09:19:56.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:56.901+0000 7f468b59e700 1 -- 172.21.15.61:0/491716307 wait complete. 2023-10-18T09:19:56.938 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-18T09:19:57.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.311+0000 7fb9e09ff700 1 -- 172.21.15.61:0/3684020148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc1077f0 msgr2=0x7fb9dc107c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/3684020148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc107c10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b0a0 tx=0x7fb9d400b3b0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 -- 172.21.15.61:0/3684020148 shutdown_connections 2023-10-18T09:19:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/3684020148 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc10f070 0x7fb9dc111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/3684020148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc107c10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/3684020148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc10eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 -- 172.21.15.61:0/3684020148 >> 172.21.15.61:0/3684020148 conn(0x7fb9dc06d7d0 msgr2=0x7fb9dc06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 -- 172.21.15.61:0/3684020148 shutdown_connections 2023-10-18T09:19:57.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.312+0000 7fb9e09ff700 1 -- 172.21.15.61:0/3684020148 wait complete. 2023-10-18T09:19:57.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.313+0000 7fb9e09ff700 1 Processor -- start 2023-10-18T09:19:57.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 -- start start 2023-10-18T09:19:57.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a6b20 con 0x7fb9dc1077f0 2023-10-18T09:19:57.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a6c60 con 0x7fb9dc108150 2023-10-18T09:19:57.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9e09ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc1a6df0 con 0x7fb9dc10f070 2023-10-18T09:19:57.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41776/0 (socket says 172.21.15.61:41776) 2023-10-18T09:19:57.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.314+0000 7fb9dad9d700 1 -- 172.21.15.61:0/2843396091 learned_addr learned my addr 172.21.15.61:0/2843396091 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.315+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.315+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.316+0000 7fb9dad9d700 1 -- 172.21.15.61:0/2843396091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 msgr2=0x7fb9dc19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9dad9d700 1 -- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 msgr2=0x7fb9dc19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9dad9d700 1 -- 172.21.15.61:0/2843396091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009d00 con 0x7fb9dc10f070 2023-10-18T09:19:57.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:57.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:57.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c900 tx=0x7fb9d000cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9cb7fe700 1 -- 172.21.15.61:0/2843396091 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0007b00 con 0x7fb9dc10f070 2023-10-18T09:19:57.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.317+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc108960 con 0x7fb9dc10f070 2023-10-18T09:19:57.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.318+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc108e70 con 0x7fb9dc10f070 2023-10-18T09:19:57.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.319+0000 7fb9cb7fe700 1 -- 172.21.15.61:0/2843396091 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9d000ce50 con 0x7fb9dc10f070 2023-10-18T09:19:57.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.319+0000 7fb9cb7fe700 1 -- 172.21.15.61:0/2843396091 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0018660 con 0x7fb9dc10f070 2023-10-18T09:19:57.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.325+0000 7fb9f4325b80 1 Processor -- start 2023-10-18T09:19:57.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.328+0000 7fb9cb7fe700 1 -- 172.21.15.61:0/2843396091 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb9d000f450 con 0x7fb9dc10f070 2023-10-18T09:19:57.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.328+0000 7fb9f4325b80 1 -- start start 2023-10-18T09:19:57.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.329+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a20c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.329+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.329+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a20ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a17e6e0 con 0x55845a20b5d0 2023-10-18T09:19:57.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a164c60 con 0x55845a20c2d0 2023-10-18T09:19:57.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a162580 con 0x55845a20cc30 2023-10-18T09:19:57.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41782/0 (socket says 172.21.15.61:41782) 2023-10-18T09:19:57.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 -- 172.21.15.61:0/2435113787 learned_addr learned my addr 172.21.15.61:0/2435113787 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a20c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 -- 172.21.15.61:0/2435113787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 msgr2=0x55845a20c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a20c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 -- 172.21.15.61:0/2435113787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 msgr2=0x55845a20ba50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:57.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a20ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 -- 172.21.15.61:0/2435113787 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55845a11b3f0 con 0x55845a20cc30 2023-10-18T09:19:57.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a20c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:57.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b780 tx=0x7fb9d40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c93bcea4c60b47f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.330+0000 7fb9c8ff9700 1 -- 172.21.15.61:0/2435113787 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400e070 con 0x55845a20cc30 2023-10-18T09:19:57.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.331+0000 7fb9c8ff9700 1 -- 172.21.15.61:0/2435113787 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9d4007760 con 0x55845a20cc30 2023-10-18T09:19:57.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.331+0000 7fb9c8ff9700 1 -- 172.21.15.61:0/2435113787 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d401ca90 con 0x55845a20cc30 2023-10-18T09:19:57.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.331+0000 7fb9cb7fe700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 0x7fb9c40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.331+0000 7fb9cb7fe700 1 -- 172.21.15.61:0/2843396091 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9d00cce60 con 0x7fb9dc10f070 2023-10-18T09:19:57.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.332+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2435113787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 msgr2=0x55845a20b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.332+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b780 tx=0x7fb9d40093e0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.332+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2435113787 shutdown_connections 2023-10-18T09:19:57.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.332+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a20ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.333+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a20c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.333+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2435113787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a20b090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.333+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 0x7fb9c40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.333+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2435113787 >> 172.21.15.61:0/2435113787 conn(0x55845a17f720 msgr2=0x55845a205b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.334+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2435113787 shutdown_connections 2023-10-18T09:19:57.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.334+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2435113787 wait complete. 2023-10-18T09:19:57.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.335+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 0x7fb9c40a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc013570 tx=0x7fb9cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.335+0000 7fb9f4325b80 1 -- start start 2023-10-18T09:19:57.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.336+0000 7fb9f4325b80 1 Processor -- start 2023-10-18T09:19:57.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a214a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a15de70 con 0x55845a20cc30 2023-10-18T09:19:57.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a0fa710 con 0x55845a20b5d0 2023-10-18T09:19:57.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a0fa3b0 con 0x55845a20c2d0 2023-10-18T09:19:57.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.337+0000 7fb9f4325b80 10 client.0 _ll_register_callbacks cb 0x55845a044a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:57.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.338+0000 7fb9d9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.338+0000 7fb9d9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41794/0 (socket says 172.21.15.61:41794) 2023-10-18T09:19:57.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.338+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/732048301 learned_addr learned my addr 172.21.15.61:0/732048301 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9da59c700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a214a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/732048301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 msgr2=0x55845a214a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a214a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/732048301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 msgr2=0x55845a21bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.339+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/732048301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc111d20 con 0x55845a20c2d0 2023-10-18T09:19:57.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.342+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011dc0 tx=0x7fb9cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48bbc47d1bd497b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.342+0000 7fb9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:57.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.342+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc01ec20 con 0x55845a20c2d0 2023-10-18T09:19:57.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.342+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9cc01ed80 con 0x55845a20c2d0 2023-10-18T09:19:57.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.343+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55845a162ef0 con 0x55845a20c2d0 2023-10-18T09:19:57.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.343+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9cc01c460 con 0x55845a20c2d0 2023-10-18T09:19:57.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.343+0000 7fb9f4325b80 10 client.25945 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:57.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.343+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558459f88de0 con 0x55845a20c2d0 2023-10-18T09:19:57.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55845a162ef0 con 0x55845a20c2d0 2023-10-18T09:19:57.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb9cc02e070 con 0x55845a20c2d0 2023-10-18T09:19:57.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:57.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 10 client.25945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:19:57.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 20 client.25945 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:57.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a229000 0x55845a22b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 4 client.25945 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:57.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.344+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55845a1da8b0 con 0x55845a229000 2023-10-18T09:19:57.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.345+0000 7fb9da59c700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a229000 0x55845a22b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.349+0000 7fb9c1ffb700 10 client.25945 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:57.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.349+0000 7fb9da59c700 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a229000 0x55845a22b480 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2bea75bbac43fba server_cookie=9487d28ef0046e3c in_seq=0 out_seq=0 2023-10-18T09:19:57.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.350+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/732048301 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55845a1da8b0 con 0x55845a229000 2023-10-18T09:19:57.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.350+0000 7fb9c1ffb700 10 client.25945 handle_command_reply: tid=0 2023-10-18T09:19:57.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.350+0000 7fb9f4325b80 1 client.25945 shutdown 2023-10-18T09:19:57.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.351+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 msgr2=0x55845a214f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.351+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc011dc0 tx=0x7fb9cc004750 comp rx=0 tx=0).stop 2023-10-18T09:19:57.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 shutdown_connections 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a229000 0x55845a22b480 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a214a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/732048301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a214f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9c1ffb700 0 client.25945 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:57.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 >> 172.21.15.61:0/732048301 conn(0x55845a17f720 msgr2=0x55845a20a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 shutdown_connections 2023-10-18T09:19:57.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.352+0000 7fb9f4325b80 1 -- 172.21.15.61:0/732048301 wait complete. 2023-10-18T09:19:57.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.353+0000 7fb9f4325b80 20 client.25945 trim_cache size 0 max 16384 2023-10-18T09:19:57.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.382+0000 7fb9f4325b80 1 Processor -- start 2023-10-18T09:19:57.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.382+0000 7fb9f4325b80 1 -- start start 2023-10-18T09:19:57.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.382+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a2005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.382+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.382+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9d9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9d9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46264/0 (socket says 172.21.15.61:46264) 2023-10-18T09:19:57.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41804/0 (socket says 172.21.15.61:41804) 2023-10-18T09:19:57.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558459fde920 con 0x55845a20b5d0 2023-10-18T09:19:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/1634143054 learned_addr learned my addr 172.21.15.61:0/1634143054 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a202550 con 0x55845a20c2d0 2023-10-18T09:19:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.383+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a162ef0 con 0x55845a20cc30 2023-10-18T09:19:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9da59c700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a2005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/1634143054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 msgr2=0x55845a214a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/1634143054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 msgr2=0x55845a2005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a2005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 -- 172.21.15.61:0/1634143054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558459f71c60 con 0x55845a20c2d0 2023-10-18T09:19:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9da59c700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a2005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.384+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc018040 tx=0x7fb9cc0046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962f37bd132e393a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.385+0000 7fb9c97fa700 1 -- 172.21.15.61:0/1634143054 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc01c460 con 0x55845a20c2d0 2023-10-18T09:19:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.385+0000 7fb9c97fa700 1 -- 172.21.15.61:0/1634143054 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9cc024a70 con 0x55845a20c2d0 2023-10-18T09:19:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.385+0000 7fb9c97fa700 1 -- 172.21.15.61:0/1634143054 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc01abf0 con 0x55845a20c2d0 2023-10-18T09:19:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.386+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 msgr2=0x55845a21bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.386+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc018040 tx=0x7fb9cc0046b0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.386+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 shutdown_connections 2023-10-18T09:19:57.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.386+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20cc30 0x55845a214a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.387+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20b5d0 0x55845a2005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.387+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/1634143054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20c2d0 0x55845a21bf60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.387+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 >> 172.21.15.61:0/1634143054 conn(0x55845a17f720 msgr2=0x55845a205810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.387+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 shutdown_connections 2023-10-18T09:19:57.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.387+0000 7fb9f4325b80 1 -- 172.21.15.61:0/1634143054 wait complete. 2023-10-18T09:19:57.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.388+0000 7fb9f4325b80 1 -- start start 2023-10-18T09:19:57.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.388+0000 7fb9f4325b80 1 Processor -- start 2023-10-18T09:19:57.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a21bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9da59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9da59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46268/0 (socket says 172.21.15.61:46268) 2023-10-18T09:19:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a21aa20 con 0x55845a20cc30 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9d9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a21bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a21ab90 con 0x55845a20b5d0 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55845a21ad50 con 0x55845a20c2d0 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9da59c700 1 -- 172.21.15.61:0/2736736758 learned_addr learned my addr 172.21.15.61:0/2736736758 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.390+0000 7fb9f4325b80 10 client.0 _ll_register_callbacks cb 0x55845a044a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9dad9d700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 -- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 msgr2=0x55845a21bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a21bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 -- 172.21.15.61:0/2736736758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 msgr2=0x55845a21a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 -- 172.21.15.61:0/2736736758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc108e70 con 0x55845a20b5d0 2023-10-18T09:19:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9da59c700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b780 tx=0x7fb9d400bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed0e74b6ebeb2732 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.391+0000 7fb9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.392+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400e040 con 0x55845a20b5d0 2023-10-18T09:19:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.392+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb9d4003c10 con 0x55845a20b5d0 2023-10-18T09:19:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.392+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d4007760 con 0x55845a20b5d0 2023-10-18T09:19:57.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.392+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d4003dd0 con 0x55845a20b5d0 2023-10-18T09:19:57.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 10 client.25931 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:57.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558459f88de0 con 0x55845a20b5d0 2023-10-18T09:19:57.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9d4007760 con 0x55845a20b5d0 2023-10-18T09:19:57.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb9d400b420 con 0x55845a20b5d0 2023-10-18T09:19:57.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 10 client.25931 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:57.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 10 client.25931 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:19:57.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 20 client.25931 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:57.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a27e1d0 0x55845a2805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 4 client.25931 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:57.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.393+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55845a156e50 con 0x55845a27e1d0 2023-10-18T09:19:57.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.394+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a27e1d0 0x55845a2805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.395+0000 7fb9d9d9b700 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a27e1d0 0x55845a2805f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87c2da7b7422b100 server_cookie=8f906cd9e77563d2 in_seq=0 out_seq=0 2023-10-18T09:19:57.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.395+0000 7fb9c1ffb700 10 client.25931 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:57.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.396+0000 7fb9c1ffb700 1 -- 172.21.15.61:0/2736736758 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55845a156e50 con 0x55845a27e1d0 2023-10-18T09:19:57.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.396+0000 7fb9c1ffb700 10 client.25931 handle_command_reply: tid=0 2023-10-18T09:19:57.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.396+0000 7fb9f4325b80 1 client.25931 shutdown 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 msgr2=0x55845a21ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400b780 tx=0x7fb9d400bd80 comp rx=0 tx=0).stop 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 shutdown_connections 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x55845a27e1d0 0x55845a2805f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55845a20cc30 0x55845a21a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55845a20b5d0 0x55845a21ba80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 --2- 172.21.15.61:0/2736736758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55845a20c2d0 0x55845a21bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 >> 172.21.15.61:0/2736736758 conn(0x55845a17f720 msgr2=0x55845a20f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9c1ffb700 0 client.25931 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:19:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.397+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 shutdown_connections 2023-10-18T09:19:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.398+0000 7fb9f4325b80 1 -- 172.21.15.61:0/2736736758 wait complete. 2023-10-18T09:19:57.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.398+0000 7fb9f4325b80 20 client.25931 trim_cache size 0 max 16384 2023-10-18T09:19:57.407 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 msgr2=0x7fb9c40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 0x7fb9c40a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc013570 tx=0x7fb9cc009260 comp rx=0 tx=0).stop 2023-10-18T09:19:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 msgr2=0x7fb9dc1a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000c900 tx=0x7fb9d000cc10 comp rx=0 tx=0).stop 2023-10-18T09:19:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 shutdown_connections 2023-10-18T09:19:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb9c40a1500 0x7fb9c40a39c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9dc10f070 0x7fb9dc1a6550 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb9dc1077f0 0x7fb9dc19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 --2- 172.21.15.61:0/2843396091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9dc108150 0x7fb9dc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.399+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 >> 172.21.15.61:0/2843396091 conn(0x7fb9dc06d7d0 msgr2=0x7fb9dc06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.400+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 shutdown_connections 2023-10-18T09:19:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.400+0000 7fb9e09ff700 1 -- 172.21.15.61:0/2843396091 wait complete. 2023-10-18T09:19:57.435 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T09:19:57.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:57 smithi061 ceph-mon[98573]: pgmap v400: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 753 op/s 2023-10-18T09:19:57.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.799+0000 7f0dc73ff700 1 -- 172.21.15.61:0/407245775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc00fe6d0 msgr2=0x7f0dc00feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.799+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/407245775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc00feaf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098f0 tx=0x7f0db4009c00 comp rx=0 tx=0).stop 2023-10-18T09:19:57.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 -- 172.21.15.61:0/407245775 shutdown_connections 2023-10-18T09:19:57.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/407245775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc0100140 0x7f0dc01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/407245775 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc00feaf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/407245775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc00ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 -- 172.21.15.61:0/407245775 >> 172.21.15.61:0/407245775 conn(0x7f0dc00f9f50 msgr2=0x7f0dc00fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 -- 172.21.15.61:0/407245775 shutdown_connections 2023-10-18T09:19:57.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.800+0000 7f0dc73ff700 1 -- 172.21.15.61:0/407245775 wait complete. 2023-10-18T09:19:57.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.802+0000 7f0dc73ff700 1 Processor -- start 2023-10-18T09:19:57.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 -- start start 2023-10-18T09:19:57.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc01046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 0x7f0dc010bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc010c180 con 0x7f0dc0100140 2023-10-18T09:19:57.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01ad280 con 0x7f0dc00fe6d0 2023-10-18T09:19:57.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.803+0000 7f0dc73ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc01ad3c0 con 0x7f0dc00ff780 2023-10-18T09:19:57.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc01046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41838/0 (socket says 172.21.15.61:41838) 2023-10-18T09:19:57.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 -- 172.21.15.61:0/2643537576 learned_addr learned my addr 172.21.15.61:0/2643537576 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc599c700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 0x7f0dc010bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 -- 172.21.15.61:0/2643537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc00fe6d0 msgr2=0x7f0dc01046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc01046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 -- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 msgr2=0x7f0dc010bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 0x7f0dc010bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc499a700 1 -- 172.21.15.61:0/2643537576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4009520 con 0x7f0dc00ff780 2023-10-18T09:19:57.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.804+0000 7f0dc599c700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 0x7f0dc010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:57.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d980 tx=0x7f0dbc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0db27fc700 1 -- 172.21.15.61:0/2643537576 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc004020 con 0x7f0dc00ff780 2023-10-18T09:19:57.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc01ad650 con 0x7f0dc00ff780 2023-10-18T09:19:57.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0db27fc700 1 -- 172.21.15.61:0/2643537576 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dbc0096d0 con 0x7f0dc00ff780 2023-10-18T09:19:57.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0db27fc700 1 -- 172.21.15.61:0/2643537576 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc003a80 con 0x7f0dc00ff780 2023-10-18T09:19:57.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.805+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc01adbb0 con 0x7f0dc00ff780 2023-10-18T09:19:57.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.811+0000 7f0ddad7ab80 1 Processor -- start 2023-10-18T09:19:57.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 -- start start 2023-10-18T09:19:57.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9436e0 con 0x55beca9d1c30 2023-10-18T09:19:57.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca929c60 con 0x55beca9d12d0 2023-10-18T09:19:57.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.812+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca927580 con 0x55beca9d05d0 2023-10-18T09:19:57.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.813+0000 7f0dc599c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.813+0000 7f0dc599c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46308/0 (socket says 172.21.15.61:46308) 2023-10-18T09:19:57.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.813+0000 7f0dc599c700 1 -- 172.21.15.61:0/560792645 learned_addr learned my addr 172.21.15.61:0/560792645 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.813+0000 7f0db27fc700 1 -- 172.21.15.61:0/2643537576 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f0dbc009840 con 0x7f0dc00ff780 2023-10-18T09:19:57.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.814+0000 7f0dc599c700 1 -- 172.21.15.61:0/560792645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 msgr2=0x55beca9d0a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:57.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.814+0000 7f0dc519b700 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0dc599c700 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9d0a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0dc599c700 1 -- 172.21.15.61:0/560792645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 msgr2=0x55beca9d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0dc599c700 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0dc599c700 1 -- 172.21.15.61:0/560792645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beca8e03f0 con 0x55beca9d12d0 2023-10-18T09:19:57.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0dc519b700 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:57.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.817+0000 7f0db27fc700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 0x7f0dac0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0dc599c700 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009d80 tx=0x7f0db80078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e704966223d24a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0db27fc700 1 -- 172.21.15.61:0/2643537576 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0dbc0cc660 con 0x7f0dc00ff780 2023-10-18T09:19:57.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0dab7fe700 1 -- 172.21.15.61:0/560792645 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8018070 con 0x55beca9d12d0 2023-10-18T09:19:57.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0dab7fe700 1 -- 172.21.15.61:0/560792645 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db800f050 con 0x55beca9d12d0 2023-10-18T09:19:57.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0dab7fe700 1 -- 172.21.15.61:0/560792645 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8014410 con 0x55beca9d12d0 2023-10-18T09:19:57.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.818+0000 7f0dc519b700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 0x7f0dac0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.821+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/560792645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 msgr2=0x55beca9d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009d80 tx=0x7f0db80078a0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/560792645 shutdown_connections 2023-10-18T09:19:57.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9d16f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/560792645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.822+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/560792645 >> 172.21.15.61:0/560792645 conn(0x55beca944720 msgr2=0x55beca9cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.823+0000 7f0dc519b700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 0x7f0dac0a3a40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098c0 tx=0x7f0db40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.823+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/560792645 shutdown_connections 2023-10-18T09:19:57.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.823+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/560792645 wait complete. 2023-10-18T09:19:57.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.824+0000 7f0ddad7ab80 1 -- start start 2023-10-18T09:19:57.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.824+0000 7f0ddad7ab80 1 Processor -- start 2023-10-18T09:19:57.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.825+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.825+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.825+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc499a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46310/0 (socket says 172.21.15.61:46310) 2023-10-18T09:19:57.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc599c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc599c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41864/0 (socket says 172.21.15.61:41864) 2023-10-18T09:19:57.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.826+0000 7f0dc499a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35130/0 (socket says 172.21.15.61:35130) 2023-10-18T09:19:57.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0dc519b700 1 -- 172.21.15.61:0/2816938594 learned_addr learned my addr 172.21.15.61:0/2816938594 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca922e70 con 0x55beca9d12d0 2023-10-18T09:19:57.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca8bf710 con 0x55beca9d05d0 2023-10-18T09:19:57.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca8bf3b0 con 0x55beca9d1c30 2023-10-18T09:19:57.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0ddad7ab80 10 client.0 _ll_register_callbacks cb 0x55beca809a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0dc519b700 1 -- 172.21.15.61:0/2816938594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 msgr2=0x55beca9f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.827+0000 7f0dc519b700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.828+0000 7f0dc519b700 1 -- 172.21.15.61:0/2816938594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 msgr2=0x55beca9eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.828+0000 7f0dc519b700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.828+0000 7f0dc519b700 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db80099e0 con 0x55beca9d05d0 2023-10-18T09:19:57.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.828+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:57.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.829+0000 7f0da8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:57.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.829+0000 7f0dc519b700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009230 tx=0x7f0db4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de013bf6ce16636 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.829+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4005450 con 0x55beca9d05d0 2023-10-18T09:19:57.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.829+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55beca927ef0 con 0x55beca9d05d0 2023-10-18T09:19:57.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.830+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0db40055b0 con 0x55beca9d05d0 2023-10-18T09:19:57.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.830+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0db4005450 con 0x55beca9d05d0 2023-10-18T09:19:57.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.830+0000 7f0ddad7ab80 10 client.25943 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:57.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.830+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55beca74dde0 con 0x55beca9d05d0 2023-10-18T09:19:57.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.831+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55beca927ef0 con 0x55beca9d05d0 2023-10-18T09:19:57.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.831+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0db4039670 con 0x55beca9d05d0 2023-10-18T09:19:57.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.831+0000 7f0ddad7ab80 10 client.25943 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:57.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.831+0000 7f0ddad7ab80 10 client.25943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:57.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0ddad7ab80 20 client.25943 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:57.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55beca9f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0ddad7ab80 4 client.25943 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:57.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55beca99f8b0 con 0x55beca9f6900 2023-10-18T09:19:57.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55beca9f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.832+0000 7f0da8ff9700 10 client.25943 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:57.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.834+0000 7f0dc499a700 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55beca9f8d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b5c263d10366df server_cookie=71a176bbf8704325 in_seq=0 out_seq=0 2023-10-18T09:19:57.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.835+0000 7f0da8ff9700 1 -- 172.21.15.61:0/2816938594 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55beca99f8b0 con 0x55beca9f6900 2023-10-18T09:19:57.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.835+0000 7f0da8ff9700 10 client.25943 handle_command_reply: tid=0 2023-10-18T09:19:57.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.836+0000 7f0ddad7ab80 1 client.25943 shutdown 2023-10-18T09:19:57.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.837+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 msgr2=0x55beca9eabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.837+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009230 tx=0x7f0db4009260 comp rx=0 tx=0).stop 2023-10-18T09:19:57.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.837+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 shutdown_connections 2023-10-18T09:19:57.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.837+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55beca9f8d20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.837+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.838+0000 7f0da8ff9700 0 client.25943 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:57.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.838+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9eabb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.838+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/2816938594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.838+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 >> 172.21.15.61:0/2816938594 conn(0x55beca944720 msgr2=0x55beca9cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.838+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 shutdown_connections 2023-10-18T09:19:57.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.839+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/2816938594 wait complete. 2023-10-18T09:19:57.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.840+0000 7f0ddad7ab80 20 client.25943 trim_cache size 0 max 16384 2023-10-18T09:19:57.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.872+0000 7f0ddad7ab80 1 Processor -- start 2023-10-18T09:19:57.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.872+0000 7f0ddad7ab80 1 -- start start 2023-10-18T09:19:57.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.872+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.872+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:41872/0 (socket says 172.21.15.61:41872) 2023-10-18T09:19:57.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0dc519b700 1 -- 172.21.15.61:0/486101915 learned_addr learned my addr 172.21.15.61:0/486101915 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca7a3920 con 0x55beca9d1c30 2023-10-18T09:19:57.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9cb0f0 con 0x55beca9d12d0 2023-10-18T09:19:57.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.873+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9f32e0 con 0x55beca9d05d0 2023-10-18T09:19:57.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc499a700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc599c700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc499a700 1 -- 172.21.15.61:0/486101915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 msgr2=0x55beca9c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc499a700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc499a700 1 -- 172.21.15.61:0/486101915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 msgr2=0x55beca9d7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.874+0000 7f0dc499a700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.875+0000 7f0dc499a700 1 -- 172.21.15.61:0/486101915 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55beca736c60 con 0x55beca9d12d0 2023-10-18T09:19:57.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.875+0000 7f0dc519b700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:57.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.876+0000 7f0dc499a700 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9f20f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d950 tx=0x7f0dbc0b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de014dbcd72ba3d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.876+0000 7f0dabfff700 1 -- 172.21.15.61:0/486101915 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc009b20 con 0x55beca9d12d0 2023-10-18T09:19:57.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.876+0000 7f0dabfff700 1 -- 172.21.15.61:0/486101915 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0dbc099cd0 con 0x55beca9d12d0 2023-10-18T09:19:57.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.876+0000 7f0dabfff700 1 -- 172.21.15.61:0/486101915 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0b7880 con 0x55beca9d12d0 2023-10-18T09:19:57.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.877+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 msgr2=0x55beca9f20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.877+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9f20f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d950 tx=0x7f0dbc0b9040 comp rx=0 tx=0).stop 2023-10-18T09:19:57.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.878+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 shutdown_connections 2023-10-18T09:19:57.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.878+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d1c30 0x55beca9d7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.878+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d05d0 0x55beca9c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.878+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/486101915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d12d0 0x55beca9f20f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.878+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 >> 172.21.15.61:0/486101915 conn(0x55beca944720 msgr2=0x55beca9d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.879+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 shutdown_connections 2023-10-18T09:19:57.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.879+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/486101915 wait complete. 2023-10-18T09:19:57.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.880+0000 7f0ddad7ab80 1 -- start start 2023-10-18T09:19:57.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.880+0000 7f0ddad7ab80 1 Processor -- start 2023-10-18T09:19:57.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9f04a0 con 0x55beca9d12d0 2023-10-18T09:19:57.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9f0610 con 0x55beca9d05d0 2023-10-18T09:19:57.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55beca9f07d0 con 0x55beca9d1c30 2023-10-18T09:19:57.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.883+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.883+0000 7f0dc519b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46332/0 (socket says 172.21.15.61:46332) 2023-10-18T09:19:57.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.883+0000 7f0dc519b700 1 -- 172.21.15.61:0/1174983738 learned_addr learned my addr 172.21.15.61:0/1174983738 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:57.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.882+0000 7f0ddad7ab80 10 client.0 _ll_register_callbacks cb 0x55beca809a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:57.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.883+0000 7f0dc599c700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc499a700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc519b700 1 -- 172.21.15.61:0/1174983738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 msgr2=0x55beca9eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc519b700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc519b700 1 -- 172.21.15.61:0/1174983738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 msgr2=0x55beca9f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:57.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc519b700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc519b700 1 -- 172.21.15.61:0/1174983738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc01adbb0 con 0x55beca9d05d0 2023-10-18T09:19:57.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc499a700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:57.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.884+0000 7f0dc599c700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:57.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.885+0000 7f0dc519b700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db4005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da5c8135f4df6c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:57.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.885+0000 7f0da8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:57.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db403e070 con 0x55beca9d05d0 2023-10-18T09:19:57.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0db40055b0 con 0x55beca9d05d0 2023-10-18T09:19:57.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0db800f050 con 0x55beca9d05d0 2023-10-18T09:19:57.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0db4003ad0 con 0x55beca9d05d0 2023-10-18T09:19:57.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:57 smithi170 ceph-mon[135150]: pgmap v400: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 753 op/s 2023-10-18T09:19:57.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0ddad7ab80 10 client.25949 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:57.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.886+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55beca774250 con 0x55beca9d05d0 2023-10-18T09:19:57.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0db800f050 con 0x55beca9d05d0 2023-10-18T09:19:57.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f0db4041020 con 0x55beca9d05d0 2023-10-18T09:19:57.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 10 client.25949 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:57.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:57.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 20 client.25949 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:57.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55becaa4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:57.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 4 client.25949 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:19:57.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.887+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55beca91be50 con 0x55beca9f6900 2023-10-18T09:19:57.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.890+0000 7f0dc499a700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55becaa4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:57.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.891+0000 7f0dc499a700 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55becaa4d730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6cd6b6c95b255 server_cookie=35af4cd1b8fc6512 in_seq=0 out_seq=0 2023-10-18T09:19:57.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.891+0000 7f0da8ff9700 10 client.25949 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:57.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0da8ff9700 1 -- 172.21.15.61:0/1174983738 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55beca91be50 con 0x55beca9f6900 2023-10-18T09:19:57.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0da8ff9700 10 client.25949 handle_command_reply: tid=0 2023-10-18T09:19:57.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 client.25949 shutdown 2023-10-18T09:19:57.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 msgr2=0x55beca9f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0db4000c00 tx=0x7f0db4005450 comp rx=0 tx=0).stop 2023-10-18T09:19:57.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 shutdown_connections 2023-10-18T09:19:57.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55beca9f6900 0x55becaa4d730 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55beca9d1c30 0x55beca9eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55beca9d05d0 0x55beca9f18b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 --2- 172.21.15.61:0/1174983738 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55beca9d12d0 0x55beca9f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 >> 172.21.15.61:0/1174983738 conn(0x55beca944720 msgr2=0x55beca9d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.892+0000 7f0da8ff9700 0 client.25949 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:57.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.893+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 shutdown_connections 2023-10-18T09:19:57.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.893+0000 7f0ddad7ab80 1 -- 172.21.15.61:0/1174983738 wait complete. 2023-10-18T09:19:57.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.894+0000 7f0ddad7ab80 20 client.25949 trim_cache size 0 max 16384 2023-10-18T09:19:57.901 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:19:57.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.895+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 msgr2=0x7f0dac0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.895+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 0x7f0dac0a3a40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098c0 tx=0x7f0db40058e0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.895+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 msgr2=0x7f0dc0104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:57.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.895+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc00d980 tx=0x7f0dbc00dc90 comp rx=0 tx=0).stop 2023-10-18T09:19:57.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 shutdown_connections 2023-10-18T09:19:57.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f0dac0a1580 0x7f0dac0a3a40 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098c0 tx=0x7f0db40058e0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f0dc0100140 0x7f0dc010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc00fe6d0 0x7f0dc01046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 --2- 172.21.15.61:0/2643537576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0dc00ff780 0x7f0dc0104be0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 >> 172.21.15.61:0/2643537576 conn(0x7f0dc00f9f50 msgr2=0x7f0dc00fb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:57.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 shutdown_connections 2023-10-18T09:19:57.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:57.896+0000 7f0dc73ff700 1 -- 172.21.15.61:0/2643537576 wait complete. 2023-10-18T09:19:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:57 smithi110 ceph-mon[134818]: pgmap v400: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 22 MiB/s wr, 753 op/s 2023-10-18T09:19:58.938 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T09:19:58.939 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T09:19:59.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:19:58 smithi061 ceph-mon[98573]: pgmap v401: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 733 op/s 2023-10-18T09:19:59.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.277+0000 7fcc933ff700 1 -- 172.21.15.61:0/1457749477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c1086a0 msgr2=0x7fcc8c10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.277+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1457749477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c10ab10 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcc840098f0 tx=0x7fcc84009c00 comp rx=0 tx=0).stop 2023-10-18T09:19:59.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 -- 172.21.15.61:0/1457749477 shutdown_connections 2023-10-18T09:19:59.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1457749477 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c10ab10 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1457749477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1457749477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c0731f0 0x7fcc8c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 -- 172.21.15.61:0/1457749477 >> 172.21.15.61:0/1457749477 conn(0x7fcc8c0fbd40 msgr2=0x7fcc8c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 -- 172.21.15.61:0/1457749477 shutdown_connections 2023-10-18T09:19:59.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.278+0000 7fcc933ff700 1 -- 172.21.15.61:0/1457749477 wait complete. 2023-10-18T09:19:59.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.279+0000 7fcc933ff700 1 Processor -- start 2023-10-18T09:19:59.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 -- start start 2023-10-18T09:19:59.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c0731f0 0x7fcc8c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1a47e0 con 0x7fcc8c0731f0 2023-10-18T09:19:59.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1a4950 con 0x7fcc8c0747d0 2023-10-18T09:19:59.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.280+0000 7fcc933ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1a4b10 con 0x7fcc8c1086a0 2023-10-18T09:19:59.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.281+0000 7fcc9099a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.281+0000 7fcc9099a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:53724/0 (socket says 172.21.15.61:53724) 2023-10-18T09:19:59.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.281+0000 7fcc9099a700 1 -- 172.21.15.61:0/1009745893 learned_addr learned my addr 172.21.15.61:0/1009745893 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.281+0000 7fcc9199c700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.281+0000 7fcc9119b700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c0731f0 0x7fcc8c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 -- 172.21.15.61:0/1009745893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 msgr2=0x7fcc8c19d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 -- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c0731f0 msgr2=0x7fcc8c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c0731f0 0x7fcc8c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 -- 172.21.15.61:0/1009745893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009d00 con 0x7fcc8c1086a0 2023-10-18T09:19:59.288 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9099a700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc9199c700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c1a4210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400fb70 tx=0x7fcc8400bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc8a7fc700 1 -- 172.21.15.61:0/1009745893 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401c070 con 0x7fcc8c1086a0 2023-10-18T09:19:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc8a7fc700 1 -- 172.21.15.61:0/1009745893 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8400bbd0 con 0x7fcc8c1086a0 2023-10-18T09:19:59.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84009520 con 0x7fcc8c1086a0 2023-10-18T09:19:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.282+0000 7fcc8a7fc700 1 -- 172.21.15.61:0/1009745893 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840039f0 con 0x7fcc8c1086a0 2023-10-18T09:19:59.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.283+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c1a5080 con 0x7fcc8c1086a0 2023-10-18T09:19:59.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.288+0000 7fcc8a7fc700 1 -- 172.21.15.61:0/1009745893 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcc84020070 con 0x7fcc8c1086a0 2023-10-18T09:19:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.289+0000 7fcc8a7fc700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 0x7fcc7c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.290+0000 7fcc8a7fc700 1 -- 172.21.15.61:0/1009745893 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc840cd9f0 con 0x7fcc8c1086a0 2023-10-18T09:19:59.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.290+0000 7fcc9119b700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 0x7fcc7c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.292+0000 7fcc9119b700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 0x7fcc7c0a39c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009fd0 tx=0x7fcc8000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.293+0000 7fcca6d2fb80 1 Processor -- start 2023-10-18T09:19:59.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.293+0000 7fcca6d2fb80 1 -- start start 2023-10-18T09:19:59.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4350 0x562df35c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c3650 0x562df35c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df3536760 con 0x562df35c3650 2023-10-18T09:19:59.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df351cce0 con 0x562df35c4350 2023-10-18T09:19:59.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df351a600 con 0x562df35c4cb0 2023-10-18T09:19:59.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcc9099a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcc9099a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44670/0 (socket says 172.21.15.61:44670) 2023-10-18T09:19:59.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.295+0000 7fcc9099a700 1 -- 172.21.15.61:0/942421744 learned_addr learned my addr 172.21.15.61:0/942421744 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9119b700 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4350 0x562df35c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9099a700 1 -- 172.21.15.61:0/942421744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4350 msgr2=0x562df35c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9099a700 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4350 0x562df35c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9099a700 1 -- 172.21.15.61:0/942421744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c3650 msgr2=0x562df35c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9099a700 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c3650 0x562df35c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.296+0000 7fcc9099a700 1 -- 172.21.15.61:0/942421744 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562df34d3470 con 0x562df35c4cb0 2023-10-18T09:19:59.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.297+0000 7fcc9099a700 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc78011680 tx=0x7fcc78011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e476a785c2097992 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.297+0000 7fcc777fe700 1 -- 172.21.15.61:0/942421744 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78004030 con 0x562df35c4cb0 2023-10-18T09:19:59.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.297+0000 7fcc777fe700 1 -- 172.21.15.61:0/942421744 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc78003680 con 0x562df35c4cb0 2023-10-18T09:19:59.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.297+0000 7fcc777fe700 1 -- 172.21.15.61:0/942421744 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800f6e0 con 0x562df35c4cb0 2023-10-18T09:19:59.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/942421744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 msgr2=0x562df35c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc78011680 tx=0x7fcc78011990 comp rx=0 tx=0).stop 2023-10-18T09:19:59.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/942421744 shutdown_connections 2023-10-18T09:19:59.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c3650 0x562df35c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4350 0x562df35c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/942421744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c3110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.298+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/942421744 >> 172.21.15.61:0/942421744 conn(0x562df35377a0 msgr2=0x562df35bdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.300+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/942421744 shutdown_connections 2023-10-18T09:19:59.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.300+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/942421744 wait complete. 2023-10-18T09:19:59.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.301+0000 7fcca6d2fb80 1 -- start start 2023-10-18T09:19:59.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.301+0000 7fcca6d2fb80 1 Processor -- start 2023-10-18T09:19:59.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df35d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df3515ef0 con 0x562df35c4350 2023-10-18T09:19:59.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df34b2790 con 0x562df35c4cb0 2023-10-18T09:19:59.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df34b2430 con 0x562df35c3650 2023-10-18T09:19:59.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcca6d2fb80 10 client.0 _ll_register_callbacks cb 0x562df33fcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:59.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44680/0 (socket says 172.21.15.61:44680) 2023-10-18T09:19:59.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcc9119b700 1 -- 172.21.15.61:0/2115471525 learned_addr learned my addr 172.21.15.61:0/2115471525 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.302+0000 7fcc9199c700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df35d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 -- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 msgr2=0x562df35d4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9099a700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df35d4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 -- 172.21.15.61:0/2115471525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 msgr2=0x562df35cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 -- 172.21.15.61:0/2115471525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc78009e10 con 0x562df35c3650 2023-10-18T09:19:59.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9099a700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:19:59.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9199c700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df35d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:59.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc9119b700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcc80015e40 tx=0x7fcc80004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb93ba7d71258216 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.303+0000 7fcc74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:19:59.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.304+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000e050 con 0x562df35c3650 2023-10-18T09:19:59.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.304+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562df351af70 con 0x562df35c3650 2023-10-18T09:19:59.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.304+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc80016c10 con 0x562df35c3650 2023-10-18T09:19:59.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.305+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc80016dd0 con 0x562df35c3650 2023-10-18T09:19:59.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.305+0000 7fcca6d2fb80 10 client.25975 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:59.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.305+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562df3340e60 con 0x562df35c3650 2023-10-18T09:19:59.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.305+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562df351af70 con 0x562df35c3650 2023-10-18T09:19:59.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcc80014520 con 0x562df35c3650 2023-10-18T09:19:59.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 10 client.25975 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:59.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:59.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 20 client.25975 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:59.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df35e9a50 0x562df35ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 4 client.25975 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:59.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.306+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562df3592930 con 0x562df35e9a50 2023-10-18T09:19:59.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.308+0000 7fcc9099a700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df35e9a50 0x562df35ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.309+0000 7fcc9099a700 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df35e9a50 0x562df35ebe70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72233c9f8636e1ed server_cookie=fc9b659089e82284 in_seq=0 out_seq=0 2023-10-18T09:19:59.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.309+0000 7fcc74ff9700 10 client.25975 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.309+0000 7fcc74ff9700 1 -- 172.21.15.61:0/2115471525 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562df3592930 con 0x562df35e9a50 2023-10-18T09:19:59.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.309+0000 7fcc74ff9700 10 client.25975 handle_command_reply: tid=0 2023-10-18T09:19:59.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.309+0000 7fcca6d2fb80 1 client.25975 shutdown 2023-10-18T09:19:59.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 msgr2=0x562df35ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcc80015e40 tx=0x7fcc80004ad0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 shutdown_connections 2023-10-18T09:19:59.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df35e9a50 0x562df35ebe70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df35d4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35ccb80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/2115471525 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 >> 172.21.15.61:0/2115471525 conn(0x562df35377a0 msgr2=0x562df35c2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.310+0000 7fcc74ff9700 0 client.25975 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.311+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 shutdown_connections 2023-10-18T09:19:59.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.311+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/2115471525 wait complete. 2023-10-18T09:19:59.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.311+0000 7fcca6d2fb80 20 client.25975 trim_cache size 0 max 16384 2023-10-18T09:19:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 45.0 s, 853.6 tps, lat 119.882 ms stddev 169.436 2023-10-18T09:19:59.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.343+0000 7fcca6d2fb80 1 Processor -- start 2023-10-18T09:19:59.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.343+0000 7fcca6d2fb80 1 -- start start 2023-10-18T09:19:59.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.343+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.343+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df33969a0 con 0x562df35c4350 2023-10-18T09:19:59.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df35cc710 con 0x562df35c3650 2023-10-18T09:19:59.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df35be140 con 0x562df35c4cb0 2023-10-18T09:19:59.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9199c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:53764/0 (socket says 172.21.15.61:53764) 2023-10-18T09:19:59.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9199c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44694/0 (socket says 172.21.15.61:44694) 2023-10-18T09:19:59.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 -- 172.21.15.61:0/725675663 learned_addr learned my addr 172.21.15.61:0/725675663 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 -- 172.21.15.61:0/725675663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 msgr2=0x562df35c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 -- 172.21.15.61:0/725675663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 msgr2=0x562df35d40c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:59.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9119b700 1 -- 172.21.15.61:0/725675663 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562df35f0ac0 con 0x562df35c3650 2023-10-18T09:19:59.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.344+0000 7fcc9199c700 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:19:59.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.345+0000 7fcc9119b700 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc80014ed0 tx=0x7fcc80014f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29df0ea23ef7cf13 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.345+0000 7fcc77fff700 1 -- 172.21.15.61:0/725675663 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000e070 con 0x562df35c3650 2023-10-18T09:19:59.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.345+0000 7fcc77fff700 1 -- 172.21.15.61:0/725675663 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8000f040 con 0x562df35c3650 2023-10-18T09:19:59.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.345+0000 7fcc77fff700 1 -- 172.21.15.61:0/725675663 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80021670 con 0x562df35c3650 2023-10-18T09:19:59.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/725675663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 msgr2=0x562df35c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc80014ed0 tx=0x7fcc80014f00 comp rx=0 tx=0).stop 2023-10-18T09:19:59.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/725675663 shutdown_connections 2023-10-18T09:19:59.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c4cb0 0x562df35c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c3650 0x562df35c7350 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/725675663 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/725675663 >> 172.21.15.61:0/725675663 conn(0x562df35377a0 msgr2=0x562df35cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.346+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/725675663 shutdown_connections 2023-10-18T09:19:59.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.347+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/725675663 wait complete. 2023-10-18T09:19:59.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.347+0000 7fcca6d2fb80 1 -- start start 2023-10-18T09:19:59.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.347+0000 7fcca6d2fb80 1 Processor -- start 2023-10-18T09:19:59.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df35d3b20 con 0x562df35c4350 2023-10-18T09:19:59.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df363ddd0 con 0x562df35c4cb0 2023-10-18T09:19:59.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.348+0000 7fcca6d2fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562df363df10 con 0x562df35c3650 2023-10-18T09:19:59.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcc9119b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcca6d2fb80 10 client.0 _ll_register_callbacks cb 0x562df33fcb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:59.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcc9199c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcc9199c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:53774/0 (socket says 172.21.15.61:53774) 2023-10-18T09:19:59.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcc9199c700 1 -- 172.21.15.61:0/32176079 learned_addr learned my addr 172.21.15.61:0/32176079 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.349+0000 7fcc9099a700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.350+0000 7fcc9199c700 1 -- 172.21.15.61:0/32176079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 msgr2=0x562df35d2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.350+0000 7fcc9199c700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35d2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.350+0000 7fcc9199c700 1 -- 172.21.15.61:0/32176079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 msgr2=0x562df35d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.350+0000 7fcc9199c700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.350+0000 7fcc9199c700 1 -- 172.21.15.61:0/32176079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562df3340e60 con 0x562df35c4cb0 2023-10-18T09:19:59.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.351+0000 7fcc9199c700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009810 tx=0x7fcc8409a920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf756d788ef9b39f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.351+0000 7fcc74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:59.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.352+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8409b150 con 0x562df35c4cb0 2023-10-18T09:19:59.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.352+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc8409aa50 con 0x562df35c4cb0 2023-10-18T09:19:59.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.352+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc78003680 con 0x562df35c4cb0 2023-10-18T09:19:59.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.353+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc8409eba0 con 0x562df35c4cb0 2023-10-18T09:19:59.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.353+0000 7fcca6d2fb80 10 client.25964 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:59.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.353+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562df3329ce0 con 0x562df35c4cb0 2023-10-18T09:19:59.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.354+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8407d080 con 0x562df35c4cb0 2023-10-18T09:19:59.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.354+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fcc84099f40 con 0x562df35c4cb0 2023-10-18T09:19:59.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 10 client.25964 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:59.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:59.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 20 client.25964 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:59.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df363f660 0x562df3641a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 4 client.25964 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T09:19:59.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562df350eed0 con 0x562df363f660 2023-10-18T09:19:59.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.355+0000 7fcc9119b700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df363f660 0x562df3641a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.356+0000 7fcc9119b700 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df363f660 0x562df3641a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db7468ce016f914 server_cookie=e512eb99660b2255 in_seq=0 out_seq=0 2023-10-18T09:19:59.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.356+0000 7fcc74ff9700 10 client.25964 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.357+0000 7fcc74ff9700 1 -- 172.21.15.61:0/32176079 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562df350eed0 con 0x562df363f660 2023-10-18T09:19:59.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.358+0000 7fcc74ff9700 10 client.25964 handle_command_reply: tid=0 2023-10-18T09:19:59.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.360+0000 7fcca6d2fb80 1 client.25964 shutdown 2023-10-18T09:19:59.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.360+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 msgr2=0x562df363d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.360+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009810 tx=0x7fcc8409a920 comp rx=0 tx=0).stop 2023-10-18T09:19:59.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 shutdown_connections 2023-10-18T09:19:59.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x562df363f660 0x562df3641a80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562df35c4cb0 0x562df363d890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562df35c3650 0x562df35d2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 --2- 172.21.15.61:0/32176079 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562df35c4350 0x562df35d32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.361+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 >> 172.21.15.61:0/32176079 conn(0x562df35377a0 msgr2=0x562df35c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.362+0000 7fcc74ff9700 0 client.25964 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.362+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 shutdown_connections 2023-10-18T09:19:59.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.362+0000 7fcca6d2fb80 1 -- 172.21.15.61:0/32176079 wait complete. 2023-10-18T09:19:59.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.363+0000 7fcca6d2fb80 20 client.25964 trim_cache size 0 max 16384 2023-10-18T09:19:59.373 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:59.373 INFO:teuthology.orchestra.run.smithi061.stdout: "return_code": 0, 2023-10-18T09:19:59.373 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_tag": "888243c2-792f-41d3-861f-8df7991c36fe", 2023-10-18T09:19:59.374 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": "asynchronous" 2023-10-18T09:19:59.374 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:59.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.364+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 msgr2=0x7fcc7c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.364+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 0x7fcc7c0a39c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009fd0 tx=0x7fcc8000b040 comp rx=0 tx=0).stop 2023-10-18T09:19:59.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.364+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 msgr2=0x7fcc8c1a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.364+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c1a4210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400fb70 tx=0x7fcc8400bf80 comp rx=0 tx=0).stop 2023-10-18T09:19:59.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 shutdown_connections 2023-10-18T09:19:59.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc7c0a1500 0x7fcc7c0a39c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc8c1086a0 0x7fcc8c1a4210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc8c0731f0 0x7fcc8c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 --2- 172.21.15.61:0/1009745893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc8c0747d0 0x7fcc8c19d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.365+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 >> 172.21.15.61:0/1009745893 conn(0x7fcc8c0fbd40 msgr2=0x7fcc8c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.366+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 shutdown_connections 2023-10-18T09:19:59.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.367+0000 7fcc933ff700 1 -- 172.21.15.61:0/1009745893 wait complete. 2023-10-18T09:19:59.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:19:58 smithi170 ceph-mon[135150]: pgmap v401: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 733 op/s 2023-10-18T09:19:59.412 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:19:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:19:59 smithi110 ceph-mon[134818]: pgmap v401: 129 pgs: 129 active+clean; 1.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 21 MiB/s wr, 733 op/s 2023-10-18T09:19:59.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 -- 172.21.15.61:0/1075155549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 msgr2=0x7fd3540ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 --2- 172.21.15.61:0/1075155549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd3540ffab0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd344009990 tx=0x7fd344009ca0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 -- 172.21.15.61:0/1075155549 shutdown_connections 2023-10-18T09:19:59.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 --2- 172.21.15.61:0/1075155549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd354101220 0x7fd354107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 --2- 172.21.15.61:0/1075155549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd3540ffab0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.787+0000 7fd3591ff700 1 --2- 172.21.15.61:0/1075155549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd354100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.788+0000 7fd3591ff700 1 -- 172.21.15.61:0/1075155549 >> 172.21.15.61:0/1075155549 conn(0x7fd3540fac00 msgr2=0x7fd3540fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.788+0000 7fd3591ff700 1 -- 172.21.15.61:0/1075155549 shutdown_connections 2023-10-18T09:19:59.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.788+0000 7fd3591ff700 1 -- 172.21.15.61:0/1075155549 wait complete. 2023-10-18T09:19:59.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.791+0000 7fd3591ff700 1 Processor -- start 2023-10-18T09:19:59.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 -- start start 2023-10-18T09:19:59.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd35410f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd354101220 0x7fd354114d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3541152f0 con 0x7fd354101220 2023-10-18T09:19:59.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354115460 con 0x7fd354100880 2023-10-18T09:19:59.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.792+0000 7fd3591ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd354115620 con 0x7fd3540ff690 2023-10-18T09:19:59.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.793+0000 7fd352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd35410f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.793+0000 7fd35259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:53796/0 (socket says 172.21.15.61:53796) 2023-10-18T09:19:59.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd35410f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44716/0 (socket says 172.21.15.61:44716) 2023-10-18T09:19:59.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 -- 172.21.15.61:0/3174605546 learned_addr learned my addr 172.21.15.61:0/3174605546 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 -- 172.21.15.61:0/3174605546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 msgr2=0x7fd35410f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd35410f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 -- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd354101220 msgr2=0x7fd354114d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd354101220 0x7fd354114d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.794+0000 7fd35259c700 1 -- 172.21.15.61:0/3174605546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3440095f0 con 0x7fd354100880 2023-10-18T09:19:59.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.796+0000 7fd35259c700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.797+0000 7fd33bfff700 1 -- 172.21.15.61:0/3174605546 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348009940 con 0x7fd354100880 2023-10-18T09:19:59.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.797+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3541158a0 con 0x7fd354100880 2023-10-18T09:19:59.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.797+0000 7fd33bfff700 1 -- 172.21.15.61:0/3174605546 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34800b6e0 con 0x7fd354100880 2023-10-18T09:19:59.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.797+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd354115e00 con 0x7fd354100880 2023-10-18T09:19:59.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.798+0000 7fd33bfff700 1 -- 172.21.15.61:0/3174605546 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348009940 con 0x7fd354100880 2023-10-18T09:19:59.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.804+0000 7fd33bfff700 1 -- 172.21.15.61:0/3174605546 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd348010ca0 con 0x7fd354100880 2023-10-18T09:19:59.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.808+0000 7fd33bfff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 0x7fd33c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.808+0000 7fd36cbe9b80 1 Processor -- start 2023-10-18T09:19:59.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.808+0000 7fd33bfff700 1 -- 172.21.15.61:0/3174605546 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3480cc260 con 0x7fd354100880 2023-10-18T09:19:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 -- start start 2023-10-18T09:19:59.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc172d0 0x55ea4cc176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc17c30 0x55ea4cc16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb896e0 con 0x55ea4cc165d0 2023-10-18T09:19:59.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb6fc60 con 0x55ea4cc17c30 2023-10-18T09:19:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.809+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb6d580 con 0x55ea4cc172d0 2023-10-18T09:19:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.810+0000 7fd352d9d700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 0x7fd33c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.810+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.810+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36296/0 (socket says 172.21.15.61:36296) 2023-10-18T09:19:59.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.810+0000 7fd35359e700 1 -- 172.21.15.61:0/2035607351 learned_addr learned my addr 172.21.15.61:0/2035607351 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.811+0000 7fd35359e700 1 -- 172.21.15.61:0/2035607351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc172d0 msgr2=0x55ea4cc176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:19:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.812+0000 7fd35359e700 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc172d0 0x55ea4cc176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.812+0000 7fd35359e700 1 -- 172.21.15.61:0/2035607351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc17c30 msgr2=0x55ea4cc16090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.812+0000 7fd35359e700 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc17c30 0x55ea4cc16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.812+0000 7fd35359e700 1 -- 172.21.15.61:0/2035607351 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea4cb263f0 con 0x55ea4cc165d0 2023-10-18T09:19:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.813+0000 7fd35359e700 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00d190 tx=0x7fd34c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb699a1d38a41b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.814+0000 7fd352d9d700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 0x7fd33c0a39d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd344000c00 tx=0x7fd344003880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.816+0000 7fd3397fa700 1 -- 172.21.15.61:0/2035607351 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c013620 con 0x55ea4cc165d0 2023-10-18T09:19:59.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.816+0000 7fd3397fa700 1 -- 172.21.15.61:0/2035607351 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34c008b10 con 0x55ea4cc165d0 2023-10-18T09:19:59.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.816+0000 7fd3397fa700 1 -- 172.21.15.61:0/2035607351 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c012b00 con 0x55ea4cc165d0 2023-10-18T09:19:59.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.818+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/2035607351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 msgr2=0x55ea4cc16a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.818+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00d190 tx=0x7fd34c00d4a0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/2035607351 shutdown_connections 2023-10-18T09:19:59.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16a50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc172d0 0x55ea4cc176f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/2035607351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc17c30 0x55ea4cc16090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/2035607351 >> 172.21.15.61:0/2035607351 conn(0x55ea4cb8a720 msgr2=0x55ea4cc10b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.820+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/2035607351 shutdown_connections 2023-10-18T09:19:59.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.821+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/2035607351 wait complete. 2023-10-18T09:19:59.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.824+0000 7fd36cbe9b80 1 -- start start 2023-10-18T09:19:59.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.824+0000 7fd36cbe9b80 1 Processor -- start 2023-10-18T09:19:59.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc165d0 0x55ea4cc1fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc17c30 0x55ea4cc27170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb68e70 con 0x55ea4cc172d0 2023-10-18T09:19:59.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb05710 con 0x55ea4cc165d0 2023-10-18T09:19:59.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cb053b0 con 0x55ea4cc17c30 2023-10-18T09:19:59.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.827+0000 7fd36cbe9b80 10 client.0 _ll_register_callbacks cb 0x55ea4ca4fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:59.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd352d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc165d0 0x55ea4cc1fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36310/0 (socket says 172.21.15.61:36310) 2023-10-18T09:19:59.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 -- 172.21.15.61:0/1725980822 learned_addr learned my addr 172.21.15.61:0/1725980822 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 -- 172.21.15.61:0/1725980822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc17c30 msgr2=0x55ea4cc27170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc17c30 0x55ea4cc27170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 -- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc165d0 msgr2=0x55ea4cc1fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc165d0 0x55ea4cc1fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.828+0000 7fd35259c700 1 -- 172.21.15.61:0/1725980822 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c008c90 con 0x55ea4cc172d0 2023-10-18T09:19:59.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.829+0000 7fd35259c700 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd348099250 tx=0x7fd3480992c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df164283e07c3b7a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.830+0000 7fd336ff5700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:19:59.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.830+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3480a15c0 con 0x55ea4cc172d0 2023-10-18T09:19:59.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.830+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3480987b0 con 0x55ea4cc172d0 2023-10-18T09:19:59.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.830+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea4cbe58b0 con 0x55ea4cc172d0 2023-10-18T09:19:59.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.831+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd348098990 con 0x55ea4cc172d0 2023-10-18T09:19:59.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.831+0000 7fd36cbe9b80 10 client.15909 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:59.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.831+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ea4c993de0 con 0x55ea4cc172d0 2023-10-18T09:19:59.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.833+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea4cbe58b0 con 0x55ea4cc172d0 2023-10-18T09:19:59.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.833+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd3480b7270 con 0x55ea4cc172d0 2023-10-18T09:19:59.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:59.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:59.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 20 client.15909 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:59.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc2ba90 0x55ea4cc2deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 4 client.15909 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:19:59.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea4cb61e50 con 0x55ea4cc2ba90 2023-10-18T09:19:59.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.834+0000 7fd352d9d700 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc2ba90 0x55ea4cc2deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.840+0000 7fd336ff5700 10 client.15909 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.840+0000 7fd352d9d700 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc2ba90 0x55ea4cc2deb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=532d16539d768a8d server_cookie=eebc9e6e079bdcd1 in_seq=0 out_seq=0 2023-10-18T09:19:59.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.841+0000 7fd336ff5700 1 -- 172.21.15.61:0/1725980822 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ea4cb61e50 con 0x55ea4cc2ba90 2023-10-18T09:19:59.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.841+0000 7fd336ff5700 10 client.15909 handle_command_reply: tid=0 2023-10-18T09:19:59.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.842+0000 7fd36cbe9b80 1 client.15909 shutdown 2023-10-18T09:19:59.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.844+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 msgr2=0x55ea4cc20180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.844+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd348099250 tx=0x7fd3480992c0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 shutdown_connections 2023-10-18T09:19:59.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc2ba90 0x55ea4cc2deb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc17c30 0x55ea4cc27170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc165d0 0x55ea4cc1fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1725980822 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc172d0 0x55ea4cc20180 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 >> 172.21.15.61:0/1725980822 conn(0x55ea4cb8a720 msgr2=0x55ea4cc15080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.845+0000 7fd336ff5700 0 client.15909 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.846+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 shutdown_connections 2023-10-18T09:19:59.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.846+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1725980822 wait complete. 2023-10-18T09:19:59.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.848+0000 7fd36cbe9b80 20 client.15909 trim_cache size 0 max 16384 2023-10-18T09:19:59.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 Processor -- start 2023-10-18T09:19:59.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 -- start start 2023-10-18T09:19:59.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc1f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4c9e9920 con 0x55ea4cc17c30 2023-10-18T09:19:59.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc12bb0 con 0x55ea4cc172d0 2023-10-18T09:19:59.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.896+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc28eb0 con 0x55ea4cc165d0 2023-10-18T09:19:59.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc1f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.902 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44740/0 (socket says 172.21.15.61:44740) 2023-10-18T09:19:59.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc1f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36320/0 (socket says 172.21.15.61:36320) 2023-10-18T09:19:59.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 -- 172.21.15.61:0/1762055514 learned_addr learned my addr 172.21.15.61:0/1762055514 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd35259c700 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 -- 172.21.15.61:0/1762055514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 msgr2=0x55ea4cc27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc27130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 -- 172.21.15.61:0/1762055514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 msgr2=0x55ea4cc1f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc1f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.897+0000 7fd352d9d700 1 -- 172.21.15.61:0/1762055514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea4c97cc60 con 0x55ea4cc165d0 2023-10-18T09:19:59.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.898+0000 7fd352d9d700 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3541006e0 tx=0x7fd344020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927592f20030b1ba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.898+0000 7fd339ffb700 1 -- 172.21.15.61:0/1762055514 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344032430 con 0x55ea4cc165d0 2023-10-18T09:19:59.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.898+0000 7fd339ffb700 1 -- 172.21.15.61:0/1762055514 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd34402c040 con 0x55ea4cc165d0 2023-10-18T09:19:59.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.898+0000 7fd339ffb700 1 -- 172.21.15.61:0/1762055514 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344024670 con 0x55ea4cc165d0 2023-10-18T09:19:59.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1762055514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 msgr2=0x55ea4cc16090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3541006e0 tx=0x7fd344020000 comp rx=0 tx=0).stop 2023-10-18T09:19:59.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1762055514 shutdown_connections 2023-10-18T09:19:59.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc1f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc16090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/1762055514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.899+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1762055514 >> 172.21.15.61:0/1762055514 conn(0x55ea4cb8a720 msgr2=0x55ea4cc169b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.900+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1762055514 shutdown_connections 2023-10-18T09:19:59.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.900+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/1762055514 wait complete. 2023-10-18T09:19:59.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.900+0000 7fd36cbe9b80 1 -- start start 2023-10-18T09:19:59.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.901+0000 7fd36cbe9b80 1 Processor -- start 2023-10-18T09:19:59.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc25570 con 0x55ea4cc17c30 2023-10-18T09:19:59.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc256e0 con 0x55ea4cc172d0 2023-10-18T09:19:59.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea4cc258a0 con 0x55ea4cc165d0 2023-10-18T09:19:59.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd36cbe9b80 10 client.0 _ll_register_callbacks cb 0x55ea4ca4fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:19:59.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.902+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35359e700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:36332/0 (socket says 172.21.15.61:36332) 2023-10-18T09:19:59.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35359e700 1 -- 172.21.15.61:0/4016426303 learned_addr learned my addr 172.21.15.61:0/4016426303 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:19:59.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd352d9d700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 -- 172.21.15.61:0/4016426303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 msgr2=0x55ea4cc269d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc269d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 -- 172.21.15.61:0/4016426303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 msgr2=0x55ea4cc25030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:19:59.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35259c700 1 -- 172.21.15.61:0/4016426303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd354108420 con 0x55ea4cc172d0 2023-10-18T09:19:59.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.903+0000 7fd35359e700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:19:59.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.904+0000 7fd336ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:19:59.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.904+0000 7fd35259c700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc26f10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3480b75d0 tx=0x7fd348005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea05c2fa987f96a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:19:59.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800b6e0 con 0x55ea4cc172d0 2023-10-18T09:19:59.914 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3480b1030 con 0x55ea4cc172d0 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea4cb6def0 con 0x55ea4cc172d0 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3480952a0 con 0x55ea4cc172d0 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd36cbe9b80 10 client.25979 fetch_fsmap learned FSMap version 32 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.907+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ea4c9ba250 con 0x55ea4cc172d0 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.908+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3480a5020 con 0x55ea4cc172d0 2023-10-18T09:19:59.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fd3480a1130 con 0x55ea4cc172d0 2023-10-18T09:19:59.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 10 client.25979 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:19:59.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 10 client.25979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:19:59.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 20 client.25979 populate_metadata read hostname 'smithi061' 2023-10-18T09:19:59.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc80850 0x55ea4cc82c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:19:59.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 4 client.25979 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:19:59.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea4cb61b70 con 0x55ea4cc80850 2023-10-18T09:19:59.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.909+0000 7fd352d9d700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc80850 0x55ea4cc82c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:19:59.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.910+0000 7fd352d9d700 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc80850 0x55ea4cc82c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308f5c2f659da957 server_cookie=214c74cb3fde4922 in_seq=0 out_seq=0 2023-10-18T09:19:59.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.910+0000 7fd336ff5700 10 client.25979 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.910+0000 7fd336ff5700 1 -- 172.21.15.61:0/4016426303 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ea4cb61b70 con 0x55ea4cc80850 2023-10-18T09:19:59.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.910+0000 7fd336ff5700 10 client.25979 handle_command_reply: tid=0 2023-10-18T09:19:59.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.910+0000 7fd36cbe9b80 1 client.25979 shutdown 2023-10-18T09:19:59.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.911+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 msgr2=0x55ea4cc26f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.911+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc26f10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd3480b75d0 tx=0x7fd348005910 comp rx=0 tx=0).stop 2023-10-18T09:19:59.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 shutdown_connections 2023-10-18T09:19:59.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ea4cc80850 0x55ea4cc82c70 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ea4cc17c30 0x55ea4cc25030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea4cc165d0 0x55ea4cc269d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 --2- 172.21.15.61:0/4016426303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ea4cc172d0 0x55ea4cc26f10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd336ff5700 0 client.25979 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:19:59.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 >> 172.21.15.61:0/4016426303 conn(0x55ea4cb8a720 msgr2=0x55ea4cc1f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 shutdown_connections 2023-10-18T09:19:59.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 1 -- 172.21.15.61:0/4016426303 wait complete. 2023-10-18T09:19:59.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.912+0000 7fd36cbe9b80 20 client.25979 trim_cache size 0 max 16384 2023-10-18T09:19:59.921 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:19:59.922 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-18T09:19:59.922 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:19:59.922 INFO:teuthology.orchestra.run.smithi061.stdout: "888243c2-792f-41d3-861f-8df7991c36fe": { 2023-10-18T09:19:59.922 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:19:59.922 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "888243c2-792f-41d3-861f-8df7991c36fe", 2023-10-18T09:19:59.923 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:19:59.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:19:59.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:19:59.923 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:19:59.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.913+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 msgr2=0x7fd33c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.913+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 0x7fd33c0a39d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd344000c00 tx=0x7fd344003880 comp rx=0 tx=0).stop 2023-10-18T09:19:59.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.913+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 msgr2=0x7fd35410fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.913+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800dc60 comp rx=0 tx=0).stop 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 shutdown_connections 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd33c0a1510 0x7fd33c0a39d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd354101220 0x7fd354114d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3540ff690 0x7fd35410f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 --2- 172.21.15.61:0/3174605546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd354100880 0x7fd35410fd10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:19:59.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 >> 172.21.15.61:0/3174605546 conn(0x7fd3540fac00 msgr2=0x7fd35410c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:19:59.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 shutdown_connections 2023-10-18T09:19:59.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:19:59.914+0000 7fd3591ff700 1 -- 172.21.15.61:0/3174605546 wait complete. 2023-10-18T09:19:59.948 INFO:tasks.cephfs.filesystem:scrub status for tag:888243c2-792f-41d3-861f-8df7991c36fe - {'path': '/', 'tag': '888243c2-792f-41d3-861f-8df7991c36fe', 'options': 'recursive,force'} 2023-10-18T09:20:00.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:00 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:00.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:00 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:00.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:00 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:00.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:00 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:00.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:00 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:00 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:00 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:00.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:00 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:00.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:00 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:00.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:00 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:00.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:00 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:00.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:00 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:00.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:00 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:00.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:00 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:00.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:00 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub queued for path: / 2023-10-18T09:20:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:01.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:01.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: overall HEALTH_OK 2023-10-18T09:20:01.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:01 smithi170 ceph-mon[135150]: pgmap v402: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-18T09:20:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub queued for path: / 2023-10-18T09:20:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:01.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:01.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:01.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:01.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: overall HEALTH_OK 2023-10-18T09:20:01.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:01 smithi110 ceph-mon[134818]: pgmap v402: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-18T09:20:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub queued for path: / 2023-10-18T09:20:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:01.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:01.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:01.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: overall HEALTH_OK 2023-10-18T09:20:01.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:01 smithi061 ceph-mon[98573]: pgmap v402: 129 pgs: 129 active+clean; 1.3 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 28 MiB/s wr, 925 op/s 2023-10-18T09:20:03.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:02 smithi061 ceph-mon[98573]: pgmap v403: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 776 op/s 2023-10-18T09:20:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:02 smithi170 ceph-mon[135150]: pgmap v403: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 776 op/s 2023-10-18T09:20:03.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:02 smithi110 ceph-mon[134818]: pgmap v403: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 MiB/s wr, 776 op/s 2023-10-18T09:20:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 50.0 s, 1426.2 tps, lat 71.326 ms stddev 86.986 2023-10-18T09:20:05.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:05 smithi170 ceph-mon[135150]: pgmap v404: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 24 MiB/s wr, 860 op/s 2023-10-18T09:20:05.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:05 smithi110 ceph-mon[134818]: pgmap v404: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 24 MiB/s wr, 860 op/s 2023-10-18T09:20:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:05 smithi061 ceph-mon[98573]: pgmap v404: 129 pgs: 129 active+clean; 1.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 24 MiB/s wr, 860 op/s 2023-10-18T09:20:06.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:06 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:20:06.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:06 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:20:06.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:06 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:20:06.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:06 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:20:06.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:06 smithi170 ceph-mon[135150]: scrub summary: idle 2023-10-18T09:20:06.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:06 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:20:06.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:06 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:20:06.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:06 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:20:06.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:06 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:20:06.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:06 smithi110 ceph-mon[134818]: scrub summary: idle 2023-10-18T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:06 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:06 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:06 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:20:06.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:06 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:20:06.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:06 smithi061 ceph-mon[98573]: scrub summary: idle 2023-10-18T09:20:07.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:07 smithi170 ceph-mon[135150]: pgmap v405: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 24 MiB/s wr, 848 op/s 2023-10-18T09:20:07.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:07 smithi110 ceph-mon[134818]: pgmap v405: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 24 MiB/s wr, 848 op/s 2023-10-18T09:20:08.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:07 smithi061 ceph-mon[98573]: pgmap v405: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 24 MiB/s wr, 848 op/s 2023-10-18T09:20:09.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:09 smithi061 ceph-mon[98573]: pgmap v406: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 821 op/s 2023-10-18T09:20:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 55.0 s, 1447.8 tps, lat 68.762 ms stddev 86.649 2023-10-18T09:20:09.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:09 smithi170 ceph-mon[135150]: pgmap v406: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 821 op/s 2023-10-18T09:20:09.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:09 smithi110 ceph-mon[134818]: pgmap v406: 129 pgs: 129 active+clean; 1.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 27 MiB/s wr, 821 op/s 2023-10-18T09:20:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:11 smithi170 ceph-mon[135150]: pgmap v407: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 32 MiB/s wr, 1.01k op/s 2023-10-18T09:20:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:11 smithi110 ceph-mon[134818]: pgmap v407: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 32 MiB/s wr, 1.01k op/s 2023-10-18T09:20:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:11 smithi061 ceph-mon[98573]: pgmap v407: 129 pgs: 129 active+clean; 1.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 32 MiB/s wr, 1.01k op/s 2023-10-18T09:20:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:13 smithi170 ceph-mon[135150]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 29 MiB/s wr, 866 op/s 2023-10-18T09:20:13.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:13 smithi110 ceph-mon[134818]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 29 MiB/s wr, 866 op/s 2023-10-18T09:20:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:13 smithi061 ceph-mon[98573]: pgmap v408: 129 pgs: 129 active+clean; 1.5 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 29 MiB/s wr, 866 op/s 2023-10-18T09:20:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 60.0 s, 1282.0 tps, lat 69.491 ms stddev 89.870 2023-10-18T09:20:15.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:15 smithi170 ceph-mon[135150]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 35 MiB/s wr, 1.26k op/s 2023-10-18T09:20:15.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:15 smithi110 ceph-mon[134818]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 35 MiB/s wr, 1.26k op/s 2023-10-18T09:20:15.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:15 smithi061 ceph-mon[98573]: pgmap v409: 129 pgs: 129 active+clean; 1.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 35 MiB/s wr, 1.26k op/s 2023-10-18T09:20:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:17 smithi170 ceph-mon[135150]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 1.63k op/s 2023-10-18T09:20:17.961 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:17 smithi110 ceph-mon[134818]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 1.63k op/s 2023-10-18T09:20:18.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:17 smithi061 ceph-mon[98573]: pgmap v410: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 38 MiB/s wr, 1.63k op/s 2023-10-18T09:20:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 65.0 s, 1382.0 tps, lat 80.352 ms stddev 194.654 2023-10-18T09:20:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:19 smithi170 ceph-mon[135150]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-18T09:20:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:19 smithi110 ceph-mon[134818]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-18T09:20:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:19 smithi061 ceph-mon[98573]: pgmap v411: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2023-10-18T09:20:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:21 smithi170 ceph-mon[135150]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2023-10-18T09:20:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:21 smithi110 ceph-mon[134818]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2023-10-18T09:20:21.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:21 smithi061 ceph-mon[98573]: pgmap v412: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 39 MiB/s wr, 1.99k op/s 2023-10-18T09:20:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:23 smithi170 ceph-mon[135150]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2023-10-18T09:20:23.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:23 smithi110 ceph-mon[134818]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2023-10-18T09:20:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:23 smithi061 ceph-mon[98573]: pgmap v413: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 34 MiB/s wr, 1.79k op/s 2023-10-18T09:20:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 70.0 s, 1841.9 tps, lat 54.223 ms stddev 80.102 2023-10-18T09:20:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:25 smithi170 ceph-mon[135150]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-10-18T09:20:25.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:25 smithi110 ceph-mon[134818]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-10-18T09:20:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:25 smithi061 ceph-mon[98573]: pgmap v414: 129 pgs: 129 active+clean; 1.5 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 32 MiB/s wr, 1.81k op/s 2023-10-18T09:20:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:27 smithi170 ceph-mon[135150]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 1.43k op/s 2023-10-18T09:20:27.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:27 smithi110 ceph-mon[134818]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 1.43k op/s 2023-10-18T09:20:28.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:27 smithi061 ceph-mon[98573]: pgmap v415: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 1.43k op/s 2023-10-18T09:20:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 75.0 s, 1800.2 tps, lat 55.109 ms stddev 72.028 2023-10-18T09:20:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:29 smithi170 ceph-mon[135150]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 MiB/s wr, 995 op/s 2023-10-18T09:20:29.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:29 smithi110 ceph-mon[134818]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 MiB/s wr, 995 op/s 2023-10-18T09:20:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:29 smithi061 ceph-mon[98573]: pgmap v416: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 20 MiB/s wr, 995 op/s 2023-10-18T09:20:29.950 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:20:30.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 -- 172.21.15.61:0/2430907800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00751b0 msgr2=0x7fbcd00755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/2430907800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00751b0 0x7fbcd00755d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009990 tx=0x7fbcc0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 -- 172.21.15.61:0/2430907800 shutdown_connections 2023-10-18T09:20:30.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/2430907800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd0077630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/2430907800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00751b0 0x7fbcd00755d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/2430907800 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00763f0 0x7fbcd0076870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 -- 172.21.15.61:0/2430907800 >> 172.21.15.61:0/2430907800 conn(0x7fbcd00fdf40 msgr2=0x7fbcd0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 -- 172.21.15.61:0/2430907800 shutdown_connections 2023-10-18T09:20:30.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.342+0000 7fbcd57ff700 1 -- 172.21.15.61:0/2430907800 wait complete. 2023-10-18T09:20:30.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.343+0000 7fbcd57ff700 1 Processor -- start 2023-10-18T09:20:30.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.344+0000 7fbcd57ff700 1 -- start start 2023-10-18T09:20:30.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.344+0000 7fbcd57ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00751b0 0x7fbcd019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.344+0000 7fbcd57ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcd57ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd01a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcd57ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01a4680 con 0x7fbcd00751b0 2023-10-18T09:20:30.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcd57ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01a47c0 con 0x7fbcd00763f0 2023-10-18T09:20:30.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcd57ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd01a4950 con 0x7fbcd0076db0 2023-10-18T09:20:30.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00751b0 0x7fbcd019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39572/0 (socket says 172.21.15.61:39572) 2023-10-18T09:20:30.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbcce7fc700 1 -- 172.21.15.61:0/3330645459 learned_addr learned my addr 172.21.15.61:0/3330645459 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.345+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd01a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 -- 172.21.15.61:0/3330645459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 msgr2=0x7fbcd01a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd01a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 -- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00751b0 msgr2=0x7fbcd019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00751b0 0x7fbcd019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 -- 172.21.15.61:0/3330645459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00095f0 con 0x7fbcd00763f0 2023-10-18T09:20:30.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd01a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:30.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.346+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d950 tx=0x7fbcc400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.347+0000 7fbcb7fff700 1 -- 172.21.15.61:0/3330645459 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400f890 con 0x7fbcd00763f0 2023-10-18T09:20:30.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.347+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd01a4c30 con 0x7fbcd00763f0 2023-10-18T09:20:30.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.347+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd01a5190 con 0x7fbcd00763f0 2023-10-18T09:20:30.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.347+0000 7fbcb7fff700 1 -- 172.21.15.61:0/3330645459 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc4010460 con 0x7fbcd00763f0 2023-10-18T09:20:30.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.347+0000 7fbcb7fff700 1 -- 172.21.15.61:0/3330645459 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400e610 con 0x7fbcd00763f0 2023-10-18T09:20:30.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.352+0000 7fbcb7fff700 1 -- 172.21.15.61:0/3330645459 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbcc400e770 con 0x7fbcd00763f0 2023-10-18T09:20:30.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.354+0000 7fbce9177b80 1 Processor -- start 2023-10-18T09:20:30.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 -- start start 2023-10-18T09:20:30.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbcb7fff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 0x7fbcb80a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab3a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab39a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aaac6e0 con 0x55e67ab395d0 2023-10-18T09:20:30.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aa92c60 con 0x55e67ab3a2d0 2023-10-18T09:20:30.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbce9177b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aa90580 con 0x55e67ab3ac30 2023-10-18T09:20:30.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.355+0000 7fbcb7fff700 1 -- 172.21.15.61:0/3330645459 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbcc4010640 con 0x7fbcd00763f0 2023-10-18T09:20:30.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.356+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.356+0000 7fbccf7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab39a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.356+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39656/0 (socket says 172.21.15.61:39656) 2023-10-18T09:20:30.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.356+0000 7fbcce7fc700 1 -- 172.21.15.61:0/2479147971 learned_addr learned my addr 172.21.15.61:0/2479147971 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.356+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 0x7fbcb80a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcceffd700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab3a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 -- 172.21.15.61:0/2479147971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 msgr2=0x55e67ab3a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab3a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 -- 172.21.15.61:0/2479147971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 msgr2=0x55e67ab39a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab39a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 -- 172.21.15.61:0/2479147971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e67aa493f0 con 0x55e67ab3ac30 2023-10-18T09:20:30.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcceffd700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab3a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:30.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab39a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:30.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40a2ed0 tx=0x7fbcc4010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601e295cb7e3f513 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 0x7fbcb80a3a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80076e0 tx=0x7fbcc800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.357+0000 7fbcb57fa700 1 -- 172.21.15.61:0/2479147971 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc409f1c0 con 0x55e67ab3ac30 2023-10-18T09:20:30.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbcb57fa700 1 -- 172.21.15.61:0/2479147971 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc40a3400 con 0x55e67ab3ac30 2023-10-18T09:20:30.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbcb57fa700 1 -- 172.21.15.61:0/2479147971 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc409f1c0 con 0x55e67ab3ac30 2023-10-18T09:20:30.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbce9177b80 1 -- 172.21.15.61:0/2479147971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 msgr2=0x55e67ab39090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40a2ed0 tx=0x7fbcc4010000 comp rx=0 tx=0).stop 2023-10-18T09:20:30.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbce9177b80 1 -- 172.21.15.61:0/2479147971 shutdown_connections 2023-10-18T09:20:30.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.358+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab39a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.359+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab3a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.359+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2479147971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab39090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.359+0000 7fbce9177b80 1 -- 172.21.15.61:0/2479147971 >> 172.21.15.61:0/2479147971 conn(0x55e67aaad720 msgr2=0x55e67ab33b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.359+0000 7fbce9177b80 1 -- 172.21.15.61:0/2479147971 shutdown_connections 2023-10-18T09:20:30.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.359+0000 7fbce9177b80 1 -- 172.21.15.61:0/2479147971 wait complete. 2023-10-18T09:20:30.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.360+0000 7fbce9177b80 1 -- start start 2023-10-18T09:20:30.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.360+0000 7fbce9177b80 1 Processor -- start 2023-10-18T09:20:30.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab42f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 0x55e67ab49fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aa8be70 con 0x55e67ab3a2d0 2023-10-18T09:20:30.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aa28710 con 0x55e67ab3ac30 2023-10-18T09:20:30.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbce9177b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aa283b0 con 0x55e67ab395d0 2023-10-18T09:20:30.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39658/0 (socket says 172.21.15.61:39658) 2023-10-18T09:20:30.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbcceffd700 1 -- 172.21.15.61:0/1853248240 learned_addr learned my addr 172.21.15.61:0/1853248240 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.361+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 0x55e67ab49fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbce9177b80 10 client.0 _ll_register_callbacks cb 0x55e67a972a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:30.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 -- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 msgr2=0x55e67ab49fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 0x55e67ab49fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab42f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 -- 172.21.15.61:0/1853248240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 msgr2=0x55e67ab42f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab42f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 -- 172.21.15.61:0/1853248240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd01a5190 con 0x55e67ab395d0 2023-10-18T09:20:30.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbccf7fe700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 0x55e67ab49fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:30.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab42f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:30.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcceffd700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0015000 tx=0x7fbcc000baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c772e0016fd9a70 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.362+0000 7fbcb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.363+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0027070 con 0x55e67ab395d0 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.363+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbcc0004710 con 0x55e67ab395d0 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.363+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e67aa90ef0 con 0x55e67ab395d0 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.364+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcc00048d0 con 0x55e67ab395d0 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.364+0000 7fbce9177b80 10 client.26005 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:30.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.364+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e67a8b6de0 con 0x55e67ab395d0 2023-10-18T09:20:30.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.364+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e67aa90ef0 con 0x55e67ab395d0 2023-10-18T09:20:30.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fbcc002a020 con 0x55e67ab395d0 2023-10-18T09:20:30.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 10 client.26005 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:30.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 10 client.26005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:30.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 20 client.26005 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:30.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67ab4e8a0 0x55e67ab50cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 4 client.26005 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:30.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.365+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e67ab088b0 con 0x55e67ab4e8a0 2023-10-18T09:20:30.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.366+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67ab4e8a0 0x55e67ab50cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.366+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67ab4e8a0 0x55e67ab50cc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e67173deadf262 server_cookie=6f4530c811f82c31 in_seq=0 out_seq=0 2023-10-18T09:20:30.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.366+0000 7fbcb2ff5700 10 client.26005 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:30.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.367+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/1853248240 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e67ab088b0 con 0x55e67ab4e8a0 2023-10-18T09:20:30.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.367+0000 7fbcb2ff5700 10 client.26005 handle_command_reply: tid=0 2023-10-18T09:20:30.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.367+0000 7fbce9177b80 1 client.26005 shutdown 2023-10-18T09:20:30.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 msgr2=0x55e67ab42a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0015000 tx=0x7fbcc000baa0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 shutdown_connections 2023-10-18T09:20:30.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67ab4e8a0 0x55e67ab50cc0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3ac30 0x55e67ab49fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab395d0 0x55e67ab42a40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 --2- 172.21.15.61:0/1853248240 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab42f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbcb2ff5700 0 client.26005 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:30.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.368+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 >> 172.21.15.61:0/1853248240 conn(0x55e67aaad720 msgr2=0x55e67ab38080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.369+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 shutdown_connections 2023-10-18T09:20:30.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.369+0000 7fbce9177b80 1 -- 172.21.15.61:0/1853248240 wait complete. 2023-10-18T09:20:30.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.369+0000 7fbce9177b80 20 client.26005 trim_cache size 0 max 16384 2023-10-18T09:20:30.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.396+0000 7fbce9177b80 1 Processor -- start 2023-10-18T09:20:30.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 -- start start 2023-10-18T09:20:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67a90c920 con 0x55e67ab395d0 2023-10-18T09:20:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67ab4b190 con 0x55e67ab3a2d0 2023-10-18T09:20:30.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.397+0000 7fbce9177b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67ab34120 con 0x55e67ab3ac30 2023-10-18T09:20:30.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.398+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.398+0000 7fbcce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39608/0 (socket says 172.21.15.61:39608) 2023-10-18T09:20:30.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.398+0000 7fbcce7fc700 1 -- 172.21.15.61:0/436671499 learned_addr learned my addr 172.21.15.61:0/436671499 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 -- 172.21.15.61:0/436671499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 msgr2=0x55e67ab42d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 -- 172.21.15.61:0/436671499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 msgr2=0x55e67ab427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 -- 172.21.15.61:0/436671499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e67a89fc60 con 0x55e67ab3a2d0 2023-10-18T09:20:30.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.399+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4010b80 tx=0x7fbcc409ab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7f7bfef94fc0c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.400+0000 7fbcb5ffb700 1 -- 172.21.15.61:0/436671499 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc409b420 con 0x55e67ab3a2d0 2023-10-18T09:20:30.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.401+0000 7fbcb5ffb700 1 -- 172.21.15.61:0/436671499 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc409f1c0 con 0x55e67ab3a2d0 2023-10-18T09:20:30.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.401+0000 7fbcb5ffb700 1 -- 172.21.15.61:0/436671499 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc409a560 con 0x55e67ab3a2d0 2023-10-18T09:20:30.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.402+0000 7fbce9177b80 1 -- 172.21.15.61:0/436671499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 msgr2=0x55e67ab49ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.402+0000 7fbce9177b80 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4010b80 tx=0x7fbcc409ab00 comp rx=0 tx=0).stop 2023-10-18T09:20:30.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 -- 172.21.15.61:0/436671499 shutdown_connections 2023-10-18T09:20:30.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab42d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab395d0 0x55e67ab427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 --2- 172.21.15.61:0/436671499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab3a2d0 0x55e67ab49ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 -- 172.21.15.61:0/436671499 >> 172.21.15.61:0/436671499 conn(0x55e67aaad720 msgr2=0x55e67ab3d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 -- 172.21.15.61:0/436671499 shutdown_connections 2023-10-18T09:20:30.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.403+0000 7fbce9177b80 1 -- 172.21.15.61:0/436671499 wait complete. 2023-10-18T09:20:30.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.404+0000 7fbce9177b80 1 -- start start 2023-10-18T09:20:30.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.404+0000 7fbce9177b80 1 Processor -- start 2023-10-18T09:20:30.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.406+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.406+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab48410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.406+0000 7fbce9177b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab48950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.406+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.406+0000 7fbcceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39616/0 (socket says 172.21.15.61:39616) 2023-10-18T09:20:30.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.407+0000 7fbcceffd700 1 -- 172.21.15.61:0/2494626958 learned_addr learned my addr 172.21.15.61:0/2494626958 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.407+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67ab48ff0 con 0x55e67ab3a2d0 2023-10-18T09:20:30.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.407+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67ab49160 con 0x55e67ab395d0 2023-10-18T09:20:30.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.407+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e67aba26e0 con 0x55e67ab3ac30 2023-10-18T09:20:30.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.407+0000 7fbce9177b80 10 client.0 _ll_register_callbacks cb 0x55e67a972a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:30.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 -- 172.21.15.61:0/2494626958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 msgr2=0x55e67ab48950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab48950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 -- 172.21.15.61:0/2494626958 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 msgr2=0x55e67ab48410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:30.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab48410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e67aa493f0 con 0x55e67ab395d0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcceffd700 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00098e0 tx=0x7fbcc0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28c68a17620e9e6b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.408+0000 7fbcb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0027040 con 0x55e67ab395d0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbcc0005bc0 con 0x55e67ab395d0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcc40a3400 con 0x55e67ab395d0 2023-10-18T09:20:30.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcc000f880 con 0x55e67ab395d0 2023-10-18T09:20:30.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbce9177b80 10 client.25994 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:30.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e67a8dd250 con 0x55e67ab395d0 2023-10-18T09:20:30.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.409+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcc003e080 con 0x55e67ab395d0 2023-10-18T09:20:30.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fbcc002a070 con 0x55e67ab395d0 2023-10-18T09:20:30.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 10 client.25994 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:30.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:30.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 20 client.25994 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:30.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67aba3e40 0x55e67aba6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 4 client.25994 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:20:30.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e67aa84e50 con 0x55e67aba3e40 2023-10-18T09:20:30.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.410+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67aba3e40 0x55e67aba6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.411+0000 7fbcce7fc700 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67aba3e40 0x55e67aba6260 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6b7ab46cde40a4 server_cookie=f3437964d8a391dc in_seq=0 out_seq=0 2023-10-18T09:20:30.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.411+0000 7fbcb2ff5700 10 client.25994 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.411+0000 7fbcb2ff5700 1 -- 172.21.15.61:0/2494626958 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e67aa84e50 con 0x55e67aba3e40 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.411+0000 7fbcb2ff5700 10 client.25994 handle_command_reply: tid=0 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.411+0000 7fbce9177b80 1 client.25994 shutdown 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 msgr2=0x55e67ab49d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00098e0 tx=0x7fbcc0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 shutdown_connections 2023-10-18T09:20:30.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e67aba3e40 0x55e67aba6260 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e67ab3ac30 0x55e67ab48950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e67ab395d0 0x55e67ab49d60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 --2- 172.21.15.61:0/2494626958 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e67ab3a2d0 0x55e67ab48410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 >> 172.21.15.61:0/2494626958 conn(0x55e67aaad720 msgr2=0x55e67ab427f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbcb2ff5700 0 client.25994 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:30.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 shutdown_connections 2023-10-18T09:20:30.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 1 -- 172.21.15.61:0/2494626958 wait complete. 2023-10-18T09:20:30.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.412+0000 7fbce9177b80 20 client.25994 trim_cache size 0 max 16384 2023-10-18T09:20:30.420 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:20:30.420 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "no active scrubs running", 2023-10-18T09:20:30.420 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": {} 2023-10-18T09:20:30.421 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:20:30.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.413+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 msgr2=0x7fbcb80a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.413+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 0x7fbcb80a3a40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80076e0 tx=0x7fbcc800b040 comp rx=0 tx=0).stop 2023-10-18T09:20:30.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.413+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 msgr2=0x7fbcd019d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.413+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400d950 tx=0x7fbcc400dc60 comp rx=0 tx=0).stop 2023-10-18T09:20:30.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 shutdown_connections 2023-10-18T09:20:30.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbcb80a1580 0x7fbcb80a3a40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd0076db0 0x7fbcd01a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbcd00751b0 0x7fbcd019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 --2- 172.21.15.61:0/3330645459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbcd00763f0 0x7fbcd019d090 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.414+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 >> 172.21.15.61:0/3330645459 conn(0x7fbcd00fdf40 msgr2=0x7fbcd00ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.415+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 shutdown_connections 2023-10-18T09:20:30.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.415+0000 7fbcd57ff700 1 -- 172.21.15.61:0/3330645459 wait complete. 2023-10-18T09:20:30.452 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-18T09:20:30.453 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:20:30.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.811+0000 7f5afebff700 1 -- 172.21.15.61:0/590289407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af81077f0 msgr2=0x7f5af8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.811+0000 7f5afebff700 1 --2- 172.21.15.61:0/590289407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af81077f0 0x7f5af8109c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009990 tx=0x7f5aec009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 -- 172.21.15.61:0/590289407 shutdown_connections 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 --2- 172.21.15.61:0/590289407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af810a5e0 0x7f5af8113480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 --2- 172.21.15.61:0/590289407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af81077f0 0x7f5af8109c20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 --2- 172.21.15.61:0/590289407 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5af810a160 0x7f5af8110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 -- 172.21.15.61:0/590289407 >> 172.21.15.61:0/590289407 conn(0x7f5af806d760 msgr2=0x7f5af806fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:30.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.812+0000 7f5afebff700 1 -- 172.21.15.61:0/590289407 shutdown_connections 2023-10-18T09:20:30.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.813+0000 7f5afebff700 1 -- 172.21.15.61:0/590289407 wait complete. 2023-10-18T09:20:30.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.814+0000 7f5afebff700 1 Processor -- start 2023-10-18T09:20:30.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.814+0000 7f5afebff700 1 -- start start 2023-10-18T09:20:30.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5af810a160 0x7f5af81a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af810a5e0 0x7f5af81ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af81acee0 con 0x7f5af810a160 2023-10-18T09:20:30.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af81ad020 con 0x7f5af810a5e0 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afebff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af81ad1b0 con 0x7f5af81077f0 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afc99b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afc99b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39692/0 (socket says 172.21.15.61:39692) 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.815+0000 7f5afc99b700 1 -- 172.21.15.61:0/1640896987 learned_addr learned my addr 172.21.15.61:0/1640896987 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.816+0000 7f5afc99b700 1 -- 172.21.15.61:0/1640896987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af810a5e0 msgr2=0x7f5af81ac910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T09:20:30.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.816+0000 7f5afc99b700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af810a5e0 0x7f5af81ac910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.816+0000 7f5afc99b700 1 -- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5af810a160 msgr2=0x7f5af81a58f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-18T09:20:30.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.817+0000 7f5afc99b700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5af810a160 0x7f5af81a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:30.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.817+0000 7f5afc99b700 1 -- 172.21.15.61:0/1640896987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec0095f0 con 0x7f5af81077f0 2023-10-18T09:20:30.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.818+0000 7f5afc99b700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005910 tx=0x7f5aec00f7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:30.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.819+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01d070 con 0x7f5af81077f0 2023-10-18T09:20:30.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.819+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5aec00fc90 con 0x7f5af81077f0 2023-10-18T09:20:30.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.819+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec017720 con 0x7f5af81077f0 2023-10-18T09:20:30.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.819+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af81ad430 con 0x7f5af81077f0 2023-10-18T09:20:30.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.819+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af81ad800 con 0x7f5af81077f0 2023-10-18T09:20:30.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.822+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af810e2c0 con 0x7f5af81077f0 2023-10-18T09:20:30.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.825+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5aec022470 con 0x7f5af81077f0 2023-10-18T09:20:30.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.827+0000 7f5af5ffb700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 0x7f5ae80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:30.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.828+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5aec0ce580 con 0x7f5af81077f0 2023-10-18T09:20:30.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.829+0000 7f5af7fff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 0x7f5ae80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:30.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.832+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5aec0978c0 con 0x7f5af81077f0 2023-10-18T09:20:30.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:30.832+0000 7f5af7fff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 0x7f5ae80a3a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005fd0 tx=0x7f5ae4005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.357 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.355+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af8067530 con 0x7f5af81077f0 2023-10-18T09:20:31.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.356+0000 7f5af5ffb700 1 -- 172.21.15.61:0/1640896987 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+11395 (secure 0 0 0) 0x7f5aec09aee0 con 0x7f5af81077f0 2023-10-18T09:20:31.359 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:20:31.359 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:19:16.512081+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:20:31.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 msgr2=0x7f5ae80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 0x7f5ae80a3a90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005fd0 tx=0x7f5ae4005d00 comp rx=0 tx=0).stop 2023-10-18T09:20:31.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 msgr2=0x7f5af81a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5aec005910 tx=0x7f5aec00f7b0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 shutdown_connections 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5ae80a15d0 0x7f5ae80a3a90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5af810a5e0 0x7f5af81ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5af81077f0 0x7f5af81a53b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.360+0000 7f5afebff700 1 --2- 172.21.15.61:0/1640896987 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5af810a160 0x7f5af81a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.361+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 >> 172.21.15.61:0/1640896987 conn(0x7f5af806d760 msgr2=0x7f5af806f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.361+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 shutdown_connections 2023-10-18T09:20:31.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.361+0000 7f5afebff700 1 -- 172.21.15.61:0/1640896987 wait complete. 2023-10-18T09:20:31.370 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 32 2023-10-18T09:20:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:31 smithi170 ceph-mon[135150]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 839 op/s 2023-10-18T09:20:31.398 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:31 smithi061 ceph-mon[98573]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 839 op/s 2023-10-18T09:20:31.403 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2023-10-18T09:20:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:31 smithi110 ceph-mon[134818]: pgmap v417: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 18 MiB/s wr, 839 op/s 2023-10-18T09:20:31.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.771+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2012784614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 msgr2=0x7fdbd006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.771+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2012784614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd006eaa0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80098f0 tx=0x7fdbb8009c00 comp rx=0 tx=0).stop 2023-10-18T09:20:31.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.775+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2012784614 shutdown_connections 2023-10-18T09:20:31.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.775+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2012784614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd006eaa0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.775+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2012784614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 0x7fdbd006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.775+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2012784614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.775+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2012784614 >> 172.21.15.61:0/2012784614 conn(0x7fdbd006b190 msgr2=0x7fdbd006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.779+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2012784614 shutdown_connections 2023-10-18T09:20:31.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.779+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2012784614 wait complete. 2023-10-18T09:20:31.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.781+0000 7fdbd55ff700 1 Processor -- start 2023-10-18T09:20:31.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 -- start start 2023-10-18T09:20:31.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 0x7fdbd01a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd01a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01ad040 con 0x7fdbd006c620 2023-10-18T09:20:31.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01ad180 con 0x7fdbd006e220 2023-10-18T09:20:31.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.782+0000 7fdbd55ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd01ad310 con 0x7fdbd006d860 2023-10-18T09:20:31.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.783+0000 7fdbce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd01a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.783+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.783+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39642/0 (socket says 172.21.15.61:39642) 2023-10-18T09:20:31.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.783+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/2656531027 learned_addr learned my addr 172.21.15.61:0/2656531027 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:31.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.783+0000 7fdbceffd700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 0x7fdbd01a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/2656531027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 msgr2=0x7fdbd01a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd01a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 msgr2=0x7fdbd01a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 0x7fdbd01a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/2656531027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8009520 con 0x7fdbd006e220 2023-10-18T09:20:31.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.784+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:31.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800fb70 tx=0x7fdbb800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbc7fff700 1 -- 172.21.15.61:0/2656531027 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb801c070 con 0x7fdbd006e220 2023-10-18T09:20:31.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbc7fff700 1 -- 172.21.15.61:0/2656531027 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb800f820 con 0x7fdbd006e220 2023-10-18T09:20:31.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbc7fff700 1 -- 172.21.15.61:0/2656531027 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8003a00 con 0x7fdbd006e220 2023-10-18T09:20:31.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd01ad590 con 0x7fdbd006e220 2023-10-18T09:20:31.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.785+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd01ad960 con 0x7fdbd006e220 2023-10-18T09:20:31.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.797+0000 7fdbc7fff700 1 -- 172.21.15.61:0/2656531027 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdbb8020030 con 0x7fdbd006e220 2023-10-18T09:20:31.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.799+0000 7fdbc7fff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 0x7fdbbc0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.800+0000 7fdbc7fff700 1 -- 172.21.15.61:0/2656531027 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbb80cdff0 con 0x7fdbd006e220 2023-10-18T09:20:31.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.802+0000 7fdbe8f5cb80 1 Processor -- start 2023-10-18T09:20:31.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 -- start start 2023-10-18T09:20:31.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b082d0 0x55d590b086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b08c30 0x55d590b07090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590a7a6e0 con 0x55d590b08c30 2023-10-18T09:20:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590a60c60 con 0x55d590b082d0 2023-10-18T09:20:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.803+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590a5e580 con 0x55d590b075d0 2023-10-18T09:20:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.804+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.804+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39716/0 (socket says 172.21.15.61:39716) 2023-10-18T09:20:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.804+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/3405799565 learned_addr learned my addr 172.21.15.61:0/3405799565 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.805+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/3405799565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b082d0 msgr2=0x55d590b086f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:20:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.805+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b082d0 0x55d590b086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.805+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/3405799565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b08c30 msgr2=0x55d590b07090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.805+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b08c30 0x55d590b07090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.805+0000 7fdbcf7fe700 1 -- 172.21.15.61:0/3405799565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d590a173f0 con 0x55d590b075d0 2023-10-18T09:20:31.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.806+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdbb809afe0 tx=0x7fdbb800bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5004e5b922f96b27 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.807+0000 7fdbceffd700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 0x7fdbbc0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.807+0000 7fdbc57fa700 1 -- 172.21.15.61:0/3405799565 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb809b750 con 0x55d590b075d0 2023-10-18T09:20:31.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.807+0000 7fdbc57fa700 1 -- 172.21.15.61:0/3405799565 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb809b050 con 0x55d590b075d0 2023-10-18T09:20:31.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.807+0000 7fdbc57fa700 1 -- 172.21.15.61:0/3405799565 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb809b750 con 0x55d590b075d0 2023-10-18T09:20:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.808+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3405799565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 msgr2=0x55d590b07a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.808+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdbb809afe0 tx=0x7fdbb800bbc0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.809+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3405799565 shutdown_connections 2023-10-18T09:20:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.809+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b075d0 0x55d590b07a50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.809+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b082d0 0x55d590b086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.809+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3405799565 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b08c30 0x55d590b07090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.809+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3405799565 >> 172.21.15.61:0/3405799565 conn(0x55d590a7b720 msgr2=0x55d590b01b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.810+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3405799565 shutdown_connections 2023-10-18T09:20:31.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.810+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3405799565 wait complete. 2023-10-18T09:20:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.811+0000 7fdbe8f5cb80 1 -- start start 2023-10-18T09:20:31.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.811+0000 7fdbe8f5cb80 1 Processor -- start 2023-10-18T09:20:31.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b08c30 0x55d590b17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590a59e70 con 0x55d590b082d0 2023-10-18T09:20:31.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5909f6710 con 0x55d590b075d0 2023-10-18T09:20:31.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5909f63b0 con 0x55d590b08c30 2023-10-18T09:20:31.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbe8f5cb80 10 client.0 _ll_register_callbacks cb 0x55d590940a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:31.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.813+0000 7fdbceffd700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 0x7fdbbc0a3a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8007f90 tx=0x7fdbc8009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.814+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.814+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39658/0 (socket says 172.21.15.61:39658) 2023-10-18T09:20:31.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.814+0000 7fdbceffd700 1 -- 172.21.15.61:0/3481302337 learned_addr learned my addr 172.21.15.61:0/3481302337 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:31.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.814+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b08c30 0x55d590b17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.814+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.815+0000 7fdbceffd700 1 -- 172.21.15.61:0/3481302337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b08c30 msgr2=0x55d590b17a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.815+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b08c30 0x55d590b17a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.815+0000 7fdbceffd700 1 -- 172.21.15.61:0/3481302337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 msgr2=0x55d590b10a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.815+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b10a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.815+0000 7fdbceffd700 1 -- 172.21.15.61:0/3481302337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd01ad960 con 0x55d590b075d0 2023-10-18T09:20:31.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.816+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80147a0 tx=0x7fdbc8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c090d355c61a77 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.816+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:31.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.816+0000 7fdbb67fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.821+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80127e0 con 0x55d590b075d0 2023-10-18T09:20:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.821+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbc8012940 con 0x55d590b075d0 2023-10-18T09:20:31.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.821+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d590a5eef0 con 0x55d590b075d0 2023-10-18T09:20:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.822+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc8012b00 con 0x55d590b075d0 2023-10-18T09:20:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.822+0000 7fdbe8f5cb80 10 client.26006 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:31.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.823+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d590884de0 con 0x55d590b075d0 2023-10-18T09:20:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.824+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d590a5eef0 con 0x55d590b075d0 2023-10-18T09:20:31.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.824+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdbc8027070 con 0x55d590b075d0 2023-10-18T09:20:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 10 client.26006 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 10 client.26006 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:20:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 20 client.26006 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b2d2f0 0x55d590b2f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 4 client.26006 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d590ad68b0 con 0x55d590b2d2f0 2023-10-18T09:20:31.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.825+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b2d2f0 0x55d590b2f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.827+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b2d2f0 0x55d590b2f710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89818861083d67a1 server_cookie=842ec509a45b2460 in_seq=0 out_seq=0 2023-10-18T09:20:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.827+0000 7fdbb67fc700 10 client.26006 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:20:31.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.828+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3481302337 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d590ad68b0 con 0x55d590b2d2f0 2023-10-18T09:20:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.828+0000 7fdbb67fc700 10 client.26006 handle_command_reply: tid=0 2023-10-18T09:20:31.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.828+0000 7fdbe8f5cb80 1 client.26006 shutdown 2023-10-18T09:20:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 msgr2=0x55d590b10510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbc80147a0 tx=0x7fdbc8004ad0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 shutdown_connections 2023-10-18T09:20:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b2d2f0 0x55d590b2f710 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b08c30 0x55d590b17a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b075d0 0x55d590b10510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3481302337 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 >> 172.21.15.61:0/3481302337 conn(0x55d590a7b720 msgr2=0x55d590b06080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.829+0000 7fdbb67fc700 0 client.26006 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:20:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.830+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 shutdown_connections 2023-10-18T09:20:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.831+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3481302337 wait complete. 2023-10-18T09:20:31.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.831+0000 7fdbe8f5cb80 20 client.26006 trim_cache size 0 max 16384 2023-10-18T09:20:31.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.866+0000 7fdbe8f5cb80 1 Processor -- start 2023-10-18T09:20:31.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.866+0000 7fdbe8f5cb80 1 -- start start 2023-10-18T09:20:31.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b17a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 0x55d590b0b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5908da920 con 0x55d590afc5d0 2023-10-18T09:20:31.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590b2a830 con 0x55d590b08c30 2023-10-18T09:20:31.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590b0f180 con 0x55d590b082d0 2023-10-18T09:20:31.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b17a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39736/0 (socket says 172.21.15.61:39736) 2023-10-18T09:20:31.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.867+0000 7fdbceffd700 1 -- 172.21.15.61:0/1445529906 learned_addr learned my addr 172.21.15.61:0/1445529906 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:31.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 0x55d590b0b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 -- 172.21.15.61:0/1445529906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 msgr2=0x55d590b17a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b17a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 -- 172.21.15.61:0/1445529906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 msgr2=0x55d590b0b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 0x55d590b0b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 -- 172.21.15.61:0/1445529906 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d59086dc60 con 0x55d590b082d0 2023-10-18T09:20:31.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 0x55d590b0b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:31.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.868+0000 7fdbceffd700 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8017040 tx=0x7fdbc800d870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc36ef5a445ff209 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.869+0000 7fdbc5ffb700 1 -- 172.21.15.61:0/1445529906 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc801e640 con 0x55d590b082d0 2023-10-18T09:20:31.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.869+0000 7fdbc5ffb700 1 -- 172.21.15.61:0/1445529906 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc8004db0 con 0x55d590b082d0 2023-10-18T09:20:31.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.869+0000 7fdbc5ffb700 1 -- 172.21.15.61:0/1445529906 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8012c20 con 0x55d590b082d0 2023-10-18T09:20:31.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/1445529906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 msgr2=0x55d590b0fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8017040 tx=0x7fdbc800d870 comp rx=0 tx=0).stop 2023-10-18T09:20:31.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/1445529906 shutdown_connections 2023-10-18T09:20:31.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590afc5d0 0x55d590b0b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590b082d0 0x55d590b0fa50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/1445529906 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b17a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.870+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/1445529906 >> 172.21.15.61:0/1445529906 conn(0x55d590a7b720 msgr2=0x55d590b34070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.871+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/1445529906 shutdown_connections 2023-10-18T09:20:31.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.871+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/1445529906 wait complete. 2023-10-18T09:20:31.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.871+0000 7fdbe8f5cb80 1 -- start start 2023-10-18T09:20:31.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.872+0000 7fdbe8f5cb80 1 Processor -- start 2023-10-18T09:20:31.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.872+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.872+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.872+0000 7fdbe8f5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39742/0 (socket says 172.21.15.61:39742) 2023-10-18T09:20:31.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbcf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39680/0 (socket says 172.21.15.61:39680) 2023-10-18T09:20:31.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 -- 172.21.15.61:0/3838007654 learned_addr learned my addr 172.21.15.61:0/3838007654 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbe8f5cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590b17d10 con 0x55d590b082d0 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590b81110 con 0x55d590b08c30 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d590b81250 con 0x55d590afc5d0 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbe8f5cb80 10 client.0 _ll_register_callbacks cb 0x55d590940a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:31.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 -- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 msgr2=0x55d590b16bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 -- 172.21.15.61:0/3838007654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 msgr2=0x55d590b16670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:31.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b16670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbceffd700 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d590884de0 con 0x55d590afc5d0 2023-10-18T09:20:31.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b16670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:31.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.873+0000 7fdbcf7fe700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:31.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.874+0000 7fdbceffd700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbc801e590 tx=0x7fdbc8004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c6be098ab4806e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:31.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.874+0000 7fdbb67fc700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:31.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.874+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc801ea70 con 0x55d590afc5d0 2023-10-18T09:20:31.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.874+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbb809b050 con 0x55d590afc5d0 2023-10-18T09:20:31.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.874+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbc80125e0 con 0x55d590afc5d0 2023-10-18T09:20:31.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.875+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc801e640 con 0x55d590afc5d0 2023-10-18T09:20:31.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.875+0000 7fdbe8f5cb80 10 client.26026 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:31.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.875+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d5908ab250 con 0x55d590afc5d0 2023-10-18T09:20:31.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb809b050 con 0x55d590afc5d0 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fdbc8027040 con 0x55d590afc5d0 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 10 client.26026 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 20 client.26026 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b82770 0x55d590b84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:31.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 4 client.26026 mds_command: new command op to 24629 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:20:31.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.876+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 --> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d590a52e50 con 0x55d590b82770 2023-10-18T09:20:31.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.878+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b82770 0x55d590b84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:31.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbb67fc700 10 client.26026 ms_handle_connect on v2:172.21.15.110:6834/145516333 2023-10-18T09:20:31.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbce7fc700 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b82770 0x55d590b84b90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee2e1a5570d123aa server_cookie=98358527629e1b97 in_seq=0 out_seq=0 2023-10-18T09:20:31.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbb67fc700 1 -- 172.21.15.61:0/3838007654 <== mds.4 v2:172.21.15.110:6834/145516333 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d590a52e50 con 0x55d590b82770 2023-10-18T09:20:31.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbb67fc700 10 client.26026 handle_command_reply: tid=0 2023-10-18T09:20:31.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 client.26026 shutdown 2023-10-18T09:20:31.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 msgr2=0x55d590b16130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbc801e590 tx=0x7fdbc8004f90 comp rx=0 tx=0).stop 2023-10-18T09:20:31.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 shutdown_connections 2023-10-18T09:20:31.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:6834/145516333,v1:172.21.15.110:6835/145516333] conn(0x55d590b82770 0x55d590b84b90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d590b08c30 0x55d590b16bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d590afc5d0 0x55d590b16130 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 --2- 172.21.15.61:0/3838007654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55d590b082d0 0x55d590b16670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 >> 172.21.15.61:0/3838007654 conn(0x55d590a7b720 msgr2=0x55d590b0b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.879+0000 7fdbb67fc700 0 client.26026 ms_handle_reset on v2:172.21.15.110:6834/145516333 2023-10-18T09:20:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.880+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 shutdown_connections 2023-10-18T09:20:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.882+0000 7fdbe8f5cb80 1 -- 172.21.15.61:0/3838007654 wait complete. 2023-10-18T09:20:31.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbe8f5cb80 20 client.26026 trim_cache size 0 max 16384 2023-10-18T09:20:31.890 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:20:31.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 msgr2=0x7fdbbc0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 0x7fdbbc0a3a10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8007f90 tx=0x7fdbc8009040 comp rx=0 tx=0).stop 2023-10-18T09:20:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 msgr2=0x7fdbd01aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800fb70 tx=0x7fdbb800b840 comp rx=0 tx=0).stop 2023-10-18T09:20:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 shutdown_connections 2023-10-18T09:20:31.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdbbc0a1550 0x7fdbbc0a3a10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbd006e220 0x7fdbd01aca70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdbd006c620 0x7fdbd01a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 --2- 172.21.15.61:0/2656531027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbd006d860 0x7fdbd01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.883+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 >> 172.21.15.61:0/2656531027 conn(0x7fdbd006b190 msgr2=0x7fdbd0109730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:31.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.884+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 shutdown_connections 2023-10-18T09:20:31.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:31.884+0000 7fdbd55ff700 1 -- 172.21.15.61:0/2656531027 wait complete. 2023-10-18T09:20:31.917 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2023-10-18T09:20:32.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:32 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1640896987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:20:32.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.305+0000 7faff59ff700 1 -- 172.21.15.61:0/2314797364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff00ff000 msgr2=0x7faff0101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.305+0000 7faff59ff700 1 --2- 172.21.15.61:0/2314797364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff00ff000 0x7faff0101430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009990 tx=0x7fafe0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 -- 172.21.15.61:0/2314797364 shutdown_connections 2023-10-18T09:20:32.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 --2- 172.21.15.61:0/2314797364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff0104340 0x7faff01047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 --2- 172.21.15.61:0/2314797364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff00ff000 0x7faff0101430 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 --2- 172.21.15.61:0/2314797364 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0101970 0x7faff0103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 -- 172.21.15.61:0/2314797364 >> 172.21.15.61:0/2314797364 conn(0x7faff00fac00 msgr2=0x7faff00fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 -- 172.21.15.61:0/2314797364 shutdown_connections 2023-10-18T09:20:32.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.306+0000 7faff59ff700 1 -- 172.21.15.61:0/2314797364 wait complete. 2023-10-18T09:20:32.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 Processor -- start 2023-10-18T09:20:32.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 -- start start 2023-10-18T09:20:32.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff00ff000 0x7faff019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 0x7faff01a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff01a4840 con 0x7faff0104340 2023-10-18T09:20:32.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff01a49b0 con 0x7faff0101970 2023-10-18T09:20:32.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.310+0000 7faff59ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff01a4b70 con 0x7faff00ff000 2023-10-18T09:20:32.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39708/0 (socket says 172.21.15.61:39708) 2023-10-18T09:20:32.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafee7fc700 1 -- 172.21.15.61:0/2736692670 learned_addr learned my addr 172.21.15.61:0/2736692670 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafeeffd700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff00ff000 0x7faff019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafef7fe700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 0x7faff01a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafee7fc700 1 -- 172.21.15.61:0/2736692670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff00ff000 msgr2=0x7faff019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.311+0000 7fafee7fc700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff00ff000 0x7faff019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.312+0000 7fafee7fc700 1 -- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 msgr2=0x7faff01a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.312+0000 7fafee7fc700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 0x7faff01a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.312+0000 7fafee7fc700 1 -- 172.21.15.61:0/2736692670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe00095f0 con 0x7faff0101970 2023-10-18T09:20:32.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.312+0000 7fafef7fe700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 0x7faff01a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:32.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.312+0000 7fafee7fc700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d950 tx=0x7fafe400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.313+0000 7faff49fd700 1 -- 172.21.15.61:0/2736692670 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4009940 con 0x7faff0101970 2023-10-18T09:20:32.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.313+0000 7faff49fd700 1 -- 172.21.15.61:0/2736692670 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe4010460 con 0x7faff0101970 2023-10-18T09:20:32.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.313+0000 7faff49fd700 1 -- 172.21.15.61:0/2736692670 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe400f620 con 0x7faff0101970 2023-10-18T09:20:32.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.313+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff01a4e50 con 0x7faff0101970 2023-10-18T09:20:32.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.314+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff01a5270 con 0x7faff0101970 2023-10-18T09:20:32.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.318+0000 7faff49fd700 1 -- 172.21.15.61:0/2736692670 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fafe40108d0 con 0x7faff0101970 2023-10-18T09:20:32.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.319+0000 7fb009334b80 1 Processor -- start 2023-10-18T09:20:32.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 -- start start 2023-10-18T09:20:32.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b703d6e0 con 0x5618b70cbc30 2023-10-18T09:20:32.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b7023c60 con 0x5618b70ca5d0 2023-10-18T09:20:32.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.320+0000 7fb009334b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b7021580 con 0x5618b70cb2d0 2023-10-18T09:20:32.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39766/0 (socket says 172.21.15.61:39766) 2023-10-18T09:20:32.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 -- 172.21.15.61:0/171811057 learned_addr learned my addr 172.21.15.61:0/171811057 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafeeffd700 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafee7fc700 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 -- 172.21.15.61:0/171811057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 msgr2=0x5618b70caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 -- 172.21.15.61:0/171811057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 msgr2=0x5618b70ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.321+0000 7fafef7fe700 1 -- 172.21.15.61:0/171811057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b6fda3f0 con 0x5618b70cb2d0 2023-10-18T09:20:32.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7fafef7fe700 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009d50 tx=0x7fafe80077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1d70cbfd3d582f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7faff49fd700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 0x7fafd80a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7fafd5ffb700 1 -- 172.21.15.61:0/171811057 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8009ea0 con 0x5618b70cb2d0 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7fafeeffd700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 0x7fafd80a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7fafd5ffb700 1 -- 172.21.15.61:0/171811057 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe8014ac0 con 0x5618b70cb2d0 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.322+0000 7fafd5ffb700 1 -- 172.21.15.61:0/171811057 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8013aa0 con 0x5618b70cb2d0 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.323+0000 7faff49fd700 1 -- 172.21.15.61:0/2736692670 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fafe40ccd90 con 0x7faff0101970 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.323+0000 7fb009334b80 1 -- 172.21.15.61:0/171811057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 msgr2=0x5618b70cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.323+0000 7fb009334b80 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009d50 tx=0x7fafe80077e0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fafeeffd700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 0x7fafd80a3a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b5c0 tx=0x7fafe0003790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 -- 172.21.15.61:0/171811057 shutdown_connections 2023-10-18T09:20:32.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70cb6f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 --2- 172.21.15.61:0/171811057 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 -- 172.21.15.61:0/171811057 >> 172.21.15.61:0/171811057 conn(0x5618b703e720 msgr2=0x5618b70c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.324+0000 7fb009334b80 1 -- 172.21.15.61:0/171811057 shutdown_connections 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.325+0000 7fb009334b80 1 -- 172.21.15.61:0/171811057 wait complete. 2023-10-18T09:20:32.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.325+0000 7fb009334b80 1 -- start start 2023-10-18T09:20:32.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 Processor -- start 2023-10-18T09:20:32.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 0x5618b70d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b701ce70 con 0x5618b70ca5d0 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b6fb9710 con 0x5618b70cbc30 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b6fb93b0 con 0x5618b70cb2d0 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.326+0000 7fb009334b80 10 client.0 _ll_register_callbacks cb 0x5618b6f03a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.327+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.327+0000 7fafef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39724/0 (socket says 172.21.15.61:39724) 2023-10-18T09:20:32.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.327+0000 7fafef7fe700 1 -- 172.21.15.61:0/1421607339 learned_addr learned my addr 172.21.15.61:0/1421607339 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.327+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 0x5618b70d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.327+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 -- 172.21.15.61:0/1421607339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 msgr2=0x5618b70db070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 -- 172.21.15.61:0/1421607339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 msgr2=0x5618b70d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 0x5618b70d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 -- 172.21.15.61:0/1421607339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe8009980 con 0x5618b70cb2d0 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafef7fe700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 0x5618b70d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70d4080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafe40ca590 tx=0x7fafe40996c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1beee3acd18fb550 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafceffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:32.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe409a1c0 con 0x5618b70cb2d0 2023-10-18T09:20:32.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fafe409deb0 con 0x5618b70cb2d0 2023-10-18T09:20:32.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.328+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618b7021ef0 con 0x5618b70cb2d0 2023-10-18T09:20:32.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.329+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafe40a20f0 con 0x5618b70cb2d0 2023-10-18T09:20:32.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.329+0000 7fb009334b80 10 client.26038 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:32.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.329+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5618b6e47de0 con 0x5618b70cb2d0 2023-10-18T09:20:32.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.329+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe40b6020 con 0x5618b70cb2d0 2023-10-18T09:20:32.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fafe40adb40 con 0x5618b70cb2d0 2023-10-18T09:20:32.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 10 client.26038 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:32.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 10 client.26038 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:20:32.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 20 client.26038 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:32.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b70df980 0x5618b70e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 4 client.26038 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.330+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618b70998b0 con 0x5618b70df980 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.331+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b70df980 0x5618b70e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.332+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b70df980 0x5618b70e1da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d5ca9be67a6f2ee server_cookie=43e10b3bc7cdfdec in_seq=0 out_seq=0 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.332+0000 7fafceffd700 10 client.26038 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.333+0000 7fafceffd700 1 -- 172.21.15.61:0/1421607339 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5618b70998b0 con 0x5618b70df980 2023-10-18T09:20:32.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.333+0000 7fafceffd700 10 client.26038 handle_command_reply: tid=0 2023-10-18T09:20:32.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 client.26038 shutdown 2023-10-18T09:20:32.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 msgr2=0x5618b70d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70d4080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fafe40ca590 tx=0x7fafe40996c0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 shutdown_connections 2023-10-18T09:20:32.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cbc30 0x5618b70db070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70ca5d0 0x5618b70d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b70df980 0x5618b70e1da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.334+0000 7fb009334b80 1 --2- 172.21.15.61:0/1421607339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cb2d0 0x5618b70d4080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.335+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 >> 172.21.15.61:0/1421607339 conn(0x5618b703e720 msgr2=0x5618b70c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.335+0000 7fafceffd700 0 client.26038 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:20:32.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.335+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 shutdown_connections 2023-10-18T09:20:32.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.335+0000 7fb009334b80 1 -- 172.21.15.61:0/1421607339 wait complete. 2023-10-18T09:20:32.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.335+0000 7fb009334b80 20 client.26038 trim_cache size 0 max 16384 2023-10-18T09:20:32.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.361+0000 7fb009334b80 1 Processor -- start 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 -- start start 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70ca5d0 0x5618b70db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70bf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b6e9d920 con 0x5618b70cbc30 2023-10-18T09:20:32.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b70c5930 con 0x5618b70cb2d0 2023-10-18T09:20:32.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.362+0000 7fb009334b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b70d36b0 con 0x5618b70ca5d0 2023-10-18T09:20:32.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39738/0 (socket says 172.21.15.61:39738) 2023-10-18T09:20:32.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 -- 172.21.15.61:0/1903286614 learned_addr learned my addr 172.21.15.61:0/1903286614 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafef7fe700 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70bf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 -- 172.21.15.61:0/1903286614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70ca5d0 msgr2=0x5618b70db720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70ca5d0 0x5618b70db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 -- 172.21.15.61:0/1903286614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 msgr2=0x5618b70bf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.364+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70bf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.365+0000 7fafee7fc700 1 -- 172.21.15.61:0/1903286614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b6e30c60 con 0x5618b70cb2d0 2023-10-18T09:20:32.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.365+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafe40980b0 tx=0x7fafe4010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b25f4456530404 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.366+0000 7fafd67fc700 1 -- 172.21.15.61:0/1903286614 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe409deb0 con 0x5618b70cb2d0 2023-10-18T09:20:32.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.367+0000 7fafd67fc700 1 -- 172.21.15.61:0/1903286614 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe40b6070 con 0x5618b70cb2d0 2023-10-18T09:20:32.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.367+0000 7fafd67fc700 1 -- 172.21.15.61:0/1903286614 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe409a220 con 0x5618b70cb2d0 2023-10-18T09:20:32.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.368+0000 7fb009334b80 1 -- 172.21.15.61:0/1903286614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 msgr2=0x5618b70db030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.368+0000 7fb009334b80 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafe40980b0 tx=0x7fafe4010c10 comp rx=0 tx=0).stop 2023-10-18T09:20:32.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.369+0000 7fb009334b80 1 -- 172.21.15.61:0/1903286614 shutdown_connections 2023-10-18T09:20:32.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.369+0000 7fb009334b80 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cbc30 0x5618b70bf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.369+0000 7fb009334b80 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70ca5d0 0x5618b70db720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.369+0000 7fb009334b80 1 --2- 172.21.15.61:0/1903286614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70cb2d0 0x5618b70db030 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.370+0000 7fb009334b80 1 -- 172.21.15.61:0/1903286614 >> 172.21.15.61:0/1903286614 conn(0x5618b703e720 msgr2=0x5618b70ca9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.370+0000 7fb009334b80 1 -- 172.21.15.61:0/1903286614 shutdown_connections 2023-10-18T09:20:32.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.370+0000 7fb009334b80 1 -- 172.21.15.61:0/1903286614 wait complete. 2023-10-18T09:20:32.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.371+0000 7fb009334b80 1 -- start start 2023-10-18T09:20:32.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.371+0000 7fb009334b80 1 Processor -- start 2023-10-18T09:20:32.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.373+0000 7fb009334b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.373+0000 7fb009334b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.373+0000 7fb009334b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cbc30 0x5618b70d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fb009334b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b70da4e0 con 0x5618b70cb2d0 2023-10-18T09:20:32.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fb009334b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b70da650 con 0x5618b70ca5d0 2023-10-18T09:20:32.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fb009334b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b70da810 con 0x5618b70cbc30 2023-10-18T09:20:32.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fb009334b80 10 client.0 _ll_register_callbacks cb 0x5618b6f03a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:32.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fafeeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fafee7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:39440/0 (socket says 172.21.15.61:39440) 2023-10-18T09:20:32.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.374+0000 7fafee7fc700 1 -- 172.21.15.61:0/1957917934 learned_addr learned my addr 172.21.15.61:0/1957917934 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafee7fc700 1 -- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cbc30 msgr2=0x5618b70d9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cbc30 0x5618b70d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafee7fc700 1 -- 172.21.15.61:0/1957917934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 msgr2=0x5618b70d8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70d8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafee7fc700 1 -- 172.21.15.61:0/1957917934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618b6e47de0 con 0x5618b70cb2d0 2023-10-18T09:20:32.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.375+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:32.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.376+0000 7fafceffd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:20:32.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.376+0000 7fafee7fc700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fafe4010fd0 tx=0x7fafe4010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b66bf6f7fabcfad server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.376+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe40a1e60 con 0x5618b70cb2d0 2023-10-18T09:20:32.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.376+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fafe40b7080 con 0x5618b70cb2d0 2023-10-18T09:20:32.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.377+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafe8014ac0 con 0x5618b70cb2d0 2023-10-18T09:20:32.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.377+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafe4099460 con 0x5618b70cb2d0 2023-10-18T09:20:32.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.377+0000 7fb009334b80 10 client.15933 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:32.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.377+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5618b6e6e250 con 0x5618b70cb2d0 2023-10-18T09:20:32.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafe8014ac0 con 0x5618b70cb2d0 2023-10-18T09:20:32.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fafe40b2260 con 0x5618b70cb2d0 2023-10-18T09:20:32.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fb009334b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:32.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fb009334b80 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2023-10-18T09:20:32.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fb009334b80 20 client.15933 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:32.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b7144c50 0x5618b7147040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.378+0000 7fb009334b80 4 client.15933 mds_command: new command op to 24646 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:20:32.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.379+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 --> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618b7015e50 con 0x5618b7144c50 2023-10-18T09:20:32.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.379+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b7144c50 0x5618b7147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.382+0000 7fafeeffd700 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b7144c50 0x5618b7147040 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b4c03205992a0caf server_cookie=c99088f139de47e8 in_seq=0 out_seq=0 2023-10-18T09:20:32.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.382+0000 7fafceffd700 10 client.15933 ms_handle_connect on v2:172.21.15.170:6834/1203136193 2023-10-18T09:20:32.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.382+0000 7fafceffd700 1 -- 172.21.15.61:0/1957917934 <== mds.3 v2:172.21.15.170:6834/1203136193 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618b7015e50 con 0x5618b7144c50 2023-10-18T09:20:32.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.382+0000 7fafceffd700 10 client.15933 handle_command_reply: tid=0 2023-10-18T09:20:32.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.382+0000 7fb009334b80 1 client.15933 shutdown 2023-10-18T09:20:32.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 msgr2=0x5618b70d9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fafe4010fd0 tx=0x7fafe4010cd0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 shutdown_connections 2023-10-18T09:20:32.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:6834/1203136193,v1:172.21.15.170:6835/1203136193] conn(0x5618b7144c50 0x5618b7147040 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5618b70cbc30 0x5618b70d9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5618b70ca5d0 0x5618b70d8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fafceffd700 0 client.15933 ms_handle_reset on v2:172.21.15.170:6834/1203136193 2023-10-18T09:20:32.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 --2- 172.21.15.61:0/1957917934 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5618b70cb2d0 0x5618b70d9030 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.383+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 >> 172.21.15.61:0/1957917934 conn(0x5618b703e720 msgr2=0x5618b70d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.385+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 shutdown_connections 2023-10-18T09:20:32.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.385+0000 7fb009334b80 1 -- 172.21.15.61:0/1957917934 wait complete. 2023-10-18T09:20:32.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.387+0000 7fb009334b80 20 client.15933 trim_cache size 0 max 16384 2023-10-18T09:20:32.390 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:20:32.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.389+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 msgr2=0x7fafd80a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.389+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 0x7fafd80a3a90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fafe000b5c0 tx=0x7fafe0003790 comp rx=0 tx=0).stop 2023-10-18T09:20:32.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.389+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 msgr2=0x7faff019d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.389+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fafe400d950 tx=0x7fafe400dc60 comp rx=0 tx=0).stop 2023-10-18T09:20:32.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:32 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1640896987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:20:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 shutdown_connections 2023-10-18T09:20:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fafd80a15d0 0x7fafd80a3a90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7faff0104340 0x7faff01a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faff00ff000 0x7faff019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 --2- 172.21.15.61:0/2736692670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faff0101970 0x7faff019d290 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.390+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 >> 172.21.15.61:0/2736692670 conn(0x7faff00fac00 msgr2=0x7faff0102370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.391+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 shutdown_connections 2023-10-18T09:20:32.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.391+0000 7faff59ff700 1 -- 172.21.15.61:0/2736692670 wait complete. 2023-10-18T09:20:32.425 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-18T09:20:32.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:32 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1640896987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:20:32.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.792+0000 7f26675ff700 1 -- 172.21.15.61:0/1931192693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 msgr2=0x7f2660105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.792+0000 7f26675ff700 1 --2- 172.21.15.61:0/1931192693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f2660105fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f264c0098f0 tx=0x7f264c009c00 comp rx=0 tx=0).stop 2023-10-18T09:20:32.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 -- 172.21.15.61:0/1931192693 shutdown_connections 2023-10-18T09:20:32.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 --2- 172.21.15.61:0/1931192693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26601064f0 0x7f26601088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 --2- 172.21.15.61:0/1931192693 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2660101290 0x7f2660103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 --2- 172.21.15.61:0/1931192693 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f2660105fb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 -- 172.21.15.61:0/1931192693 >> 172.21.15.61:0/1931192693 conn(0x7f26600fac00 msgr2=0x7f26600fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 -- 172.21.15.61:0/1931192693 shutdown_connections 2023-10-18T09:20:32.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.793+0000 7f26675ff700 1 -- 172.21.15.61:0/1931192693 wait complete. 2023-10-18T09:20:32.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.794+0000 7f26675ff700 1 Processor -- start 2023-10-18T09:20:32.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 -- start start 2023-10-18T09:20:32.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2660101290 0x7f266019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26601064f0 0x7f26601a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26601a4860 con 0x7f26601064f0 2023-10-18T09:20:32.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26601a49d0 con 0x7f2660103bc0 2023-10-18T09:20:32.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f26675ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26601a4b90 con 0x7f2660101290 2023-10-18T09:20:32.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.795+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39752/0 (socket says 172.21.15.61:39752) 2023-10-18T09:20:32.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 -- 172.21.15.61:0/3229274780 learned_addr learned my addr 172.21.15.61:0/3229274780 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 -- 172.21.15.61:0/3229274780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2660101290 msgr2=0x7f266019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2660101290 0x7f266019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 -- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26601064f0 msgr2=0x7f26601a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26601064f0 0x7f26601a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.796+0000 7f2664b9a700 1 -- 172.21.15.61:0/3229274780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c009520 con 0x7f2660103bc0 2023-10-18T09:20:32.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.797+0000 7f2664b9a700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b810 tx=0x7f264c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.801+0000 7f265e7fc700 1 -- 172.21.15.61:0/3229274780 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c005450 con 0x7f2660103bc0 2023-10-18T09:20:32.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.801+0000 7f265e7fc700 1 -- 172.21.15.61:0/3229274780 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f264c00f7d0 con 0x7f2660103bc0 2023-10-18T09:20:32.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.801+0000 7f265e7fc700 1 -- 172.21.15.61:0/3229274780 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c017660 con 0x7f2660103bc0 2023-10-18T09:20:32.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.801+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26601a4e10 con 0x7f2660103bc0 2023-10-18T09:20:32.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.801+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26601a5180 con 0x7f2660103bc0 2023-10-18T09:20:32.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.811+0000 7f267ae8db80 1 Processor -- start 2023-10-18T09:20:32.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.811+0000 7f267ae8db80 1 -- start start 2023-10-18T09:20:32.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.812+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 0x562339a536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.812+0000 7f265e7fc700 1 -- 172.21.15.61:0/3229274780 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f264c003680 con 0x7f2660103bc0 2023-10-18T09:20:32.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.814+0000 7f265e7fc700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 0x7f26500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f265e7fc700 1 -- 172.21.15.61:0/3229274780 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f264c0ce450 con 0x7f2660103bc0 2023-10-18T09:20:32.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f266539b700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 0x7f26500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a52090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 0x562339a52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f267ae8db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399c56e0 con 0x562339a532d0 2023-10-18T09:20:32.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f267ae8db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399abc60 con 0x562339a525d0 2023-10-18T09:20:32.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.815+0000 7f267ae8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399a9580 con 0x562339a53c30 2023-10-18T09:20:32.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.816+0000 7f2665b9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 0x562339a536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.816+0000 7f2665b9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 0x562339a536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:39458/0 (socket says 172.21.15.61:39458) 2023-10-18T09:20:32.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.816+0000 7f2665b9c700 1 -- 172.21.15.61:0/2174218643 learned_addr learned my addr 172.21.15.61:0/2174218643 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.816+0000 7f2665b9c700 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 0x562339a52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.816+0000 7f266539b700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 0x7f26500a39c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26540060b0 tx=0x7f2654006040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 -- 172.21.15.61:0/2174218643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 msgr2=0x562339a52090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a52090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 -- 172.21.15.61:0/2174218643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 msgr2=0x562339a536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 0x562339a536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 -- 172.21.15.61:0/2174218643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623399623f0 con 0x562339a525d0 2023-10-18T09:20:32.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f2665b9c700 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 0x562339a52a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2658013530 tx=0x7f2658009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80cdc112054854de server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.817+0000 7f264b7fe700 1 -- 172.21.15.61:0/2174218643 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658004830 con 0x562339a525d0 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.818+0000 7f264b7fe700 1 -- 172.21.15.61:0/2174218643 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2658004e50 con 0x562339a525d0 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 -- 172.21.15.61:0/2174218643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 msgr2=0x562339a52a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 0x562339a52a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2658013530 tx=0x7f2658009310 comp rx=0 tx=0).stop 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 -- 172.21.15.61:0/2174218643 shutdown_connections 2023-10-18T09:20:32.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a525d0 0x562339a52a50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a532d0 0x562339a536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 --2- 172.21.15.61:0/2174218643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a52090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 -- 172.21.15.61:0/2174218643 >> 172.21.15.61:0/2174218643 conn(0x5623399c6720 msgr2=0x562339a4cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 -- 172.21.15.61:0/2174218643 shutdown_connections 2023-10-18T09:20:32.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.819+0000 7f267ae8db80 1 -- 172.21.15.61:0/2174218643 wait complete. 2023-10-18T09:20:32.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.820+0000 7f267ae8db80 1 -- start start 2023-10-18T09:20:32.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.821+0000 7f267ae8db80 1 Processor -- start 2023-10-18T09:20:32.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39758/0 (socket says 172.21.15.61:39758) 2023-10-18T09:20:32.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f2664b9a700 1 -- 172.21.15.61:0/4149761105 learned_addr learned my addr 172.21.15.61:0/4149761105 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399a4e70 con 0x562339a525d0 2023-10-18T09:20:32.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339941710 con 0x562339a532d0 2023-10-18T09:20:32.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399413b0 con 0x562339a53c30 2023-10-18T09:20:32.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.823+0000 7f267ae8db80 10 client.0 _ll_register_callbacks cb 0x56233988ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:32.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f266539b700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 -- 172.21.15.61:0/4149761105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 msgr2=0x562339a63120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 -- 172.21.15.61:0/4149761105 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 msgr2=0x562339a5bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a5bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2658009dc0 con 0x562339a532d0 2023-10-18T09:20:32.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2664b9a700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f264c09b3e0 tx=0x7f264c09b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459c44e3166eea9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.824+0000 7f2648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:32.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.826+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c09b450 con 0x562339a532d0 2023-10-18T09:20:32.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.826+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f264c09f5a0 con 0x562339a532d0 2023-10-18T09:20:32.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.826+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623399a9ef0 con 0x562339a532d0 2023-10-18T09:20:32.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.828+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f264c0a3750 con 0x562339a532d0 2023-10-18T09:20:32.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.830+0000 7f267ae8db80 10 client.26042 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:32.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.830+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5623397cfde0 con 0x562339a532d0 2023-10-18T09:20:32.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623399a9ef0 con 0x562339a532d0 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f264c09f120 con 0x562339a532d0 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 10 client.26042 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 10 client.26042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 20 client.26042 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339a676c0 0x562339a69ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 4 client.26042 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:32.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.832+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562339a218b0 con 0x562339a676c0 2023-10-18T09:20:32.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.833+0000 7f266539b700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339a676c0 0x562339a69ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.834+0000 7f2648ff9700 10 client.26042 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:20:32.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.834+0000 7f266539b700 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339a676c0 0x562339a69ae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d49a11584b6dfff server_cookie=ec8cf512022ad1fb in_seq=0 out_seq=0 2023-10-18T09:20:32.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.835+0000 7f2648ff9700 1 -- 172.21.15.61:0/4149761105 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562339a218b0 con 0x562339a676c0 2023-10-18T09:20:32.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.835+0000 7f2648ff9700 10 client.26042 handle_command_reply: tid=0 2023-10-18T09:20:32.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.835+0000 7f267ae8db80 1 client.26042 shutdown 2023-10-18T09:20:32.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 msgr2=0x562339a5c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f264c09b3e0 tx=0x7f264c09b040 comp rx=0 tx=0).stop 2023-10-18T09:20:32.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 shutdown_connections 2023-10-18T09:20:32.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339a676c0 0x562339a69ae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a5bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 --2- 172.21.15.61:0/4149761105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a532d0 0x562339a5c0e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 >> 172.21.15.61:0/4149761105 conn(0x5623399c6720 msgr2=0x562339a51080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.836+0000 7f2648ff9700 0 client.26042 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:20:32.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.837+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 shutdown_connections 2023-10-18T09:20:32.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.837+0000 7f267ae8db80 1 -- 172.21.15.61:0/4149761105 wait complete. 2023-10-18T09:20:32.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.837+0000 7f267ae8db80 20 client.26042 trim_cache size 0 max 16384 2023-10-18T09:20:32.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.867+0000 7f267ae8db80 1 Processor -- start 2023-10-18T09:20:32.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.867+0000 7f267ae8db80 1 -- start start 2023-10-18T09:20:32.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.867+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 0x562339a5b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f267ae8db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339a5ac00 con 0x562339a525d0 2023-10-18T09:20:32.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f267ae8db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339825920 con 0x562339a529b0 2023-10-18T09:20:32.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f267ae8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623399a9ef0 con 0x562339a53c30 2023-10-18T09:20:32.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.868+0000 7f266539b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39836/0 (socket says 172.21.15.61:39836) 2023-10-18T09:20:32.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2664b9a700 1 -- 172.21.15.61:0/3621209251 learned_addr learned my addr 172.21.15.61:0/3621209251 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 0x562339a5b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 -- 172.21.15.61:0/3621209251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 msgr2=0x562339a63040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 -- 172.21.15.61:0/3621209251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 msgr2=0x562339a475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.869+0000 7f2665b9c700 1 -- 172.21.15.61:0/3621209251 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623397b8c60 con 0x562339a529b0 2023-10-18T09:20:32.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.870+0000 7f2665b9c700 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 0x562339a5b0b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x562339a49460 tx=0x7f2658005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e00fcee6dee8a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.871+0000 7f264bfff700 1 -- 172.21.15.61:0/3621209251 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580055a0 con 0x562339a529b0 2023-10-18T09:20:32.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.871+0000 7f264bfff700 1 -- 172.21.15.61:0/3621209251 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2658005700 con 0x562339a529b0 2023-10-18T09:20:32.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.872+0000 7f264bfff700 1 -- 172.21.15.61:0/3621209251 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26580043d0 con 0x562339a529b0 2023-10-18T09:20:32.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 -- 172.21.15.61:0/3621209251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 msgr2=0x562339a5b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 0x562339a5b0b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x562339a49460 tx=0x7f2658005fb0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 -- 172.21.15.61:0/3621209251 shutdown_connections 2023-10-18T09:20:32.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a529b0 0x562339a5b0b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 --2- 172.21.15.61:0/3621209251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a53c30 0x562339a63040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.874+0000 7f267ae8db80 1 -- 172.21.15.61:0/3621209251 >> 172.21.15.61:0/3621209251 conn(0x5623399c6720 msgr2=0x562339a4cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.876+0000 7f267ae8db80 1 -- 172.21.15.61:0/3621209251 shutdown_connections 2023-10-18T09:20:32.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.876+0000 7f267ae8db80 1 -- 172.21.15.61:0/3621209251 wait complete. 2023-10-18T09:20:32.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.877+0000 7f267ae8db80 1 -- start start 2023-10-18T09:20:32.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.878+0000 7f267ae8db80 1 Processor -- start 2023-10-18T09:20:32.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.879+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 0x562339a610b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339a61740 con 0x562339a525d0 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339a618b0 con 0x562339a53c30 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x562339a61a70 con 0x562339a529b0 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f267ae8db80 10 client.0 _ll_register_callbacks cb 0x56233988ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f2664b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39838/0 (socket says 172.21.15.61:39838) 2023-10-18T09:20:32.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.880+0000 7f2664b9a700 1 -- 172.21.15.61:0/1081065001 learned_addr learned my addr 172.21.15.61:0/1081065001 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:32.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.881+0000 7f2665b9c700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 0x562339a610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.881+0000 7f266539b700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.881+0000 7f2664b9a700 1 -- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 msgr2=0x562339a610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.881+0000 7f2664b9a700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 0x562339a610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.881+0000 7f2664b9a700 1 -- 172.21.15.61:0/1081065001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 msgr2=0x562339a62a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:32.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.882+0000 7f2664b9a700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a62a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.882+0000 7f2664b9a700 1 -- 172.21.15.61:0/1081065001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623399623f0 con 0x562339a529b0 2023-10-18T09:20:32.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.882+0000 7f266539b700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:32.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.882+0000 7f2665b9c700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 0x562339a610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:32.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.883+0000 7f2664b9a700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f264c09a960 tx=0x7f264c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4606e056c2ca6aee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:32.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.883+0000 7f2648ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:32.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.883+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c09b4d0 con 0x562339a529b0 2023-10-18T09:20:32.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.883+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f264c0a3750 con 0x562339a529b0 2023-10-18T09:20:32.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.884+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56233999de50 con 0x562339a529b0 2023-10-18T09:20:32.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.884+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f264c09b7a0 con 0x562339a529b0 2023-10-18T09:20:32.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.885+0000 7f267ae8db80 10 client.26047 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:32.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.885+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5623397f6250 con 0x562339a529b0 2023-10-18T09:20:32.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.885+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56233999de50 con 0x562339a529b0 2023-10-18T09:20:32.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.885+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f264c0a33e0 con 0x562339a529b0 2023-10-18T09:20:32.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 10 client.26047 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 10 client.26047 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 20 client.26047 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339abc3c0 0x562339abe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 4 client.26047 mds_command: new command op to 24653 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 --> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56233999db70 con 0x562339abc3c0 2023-10-18T09:20:32.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.886+0000 7f266539b700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339abc3c0 0x562339abe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:32.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.887+0000 7f2648ff9700 10 client.26047 ms_handle_connect on v2:172.21.15.110:6836/2750873982 2023-10-18T09:20:32.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.887+0000 7f266539b700 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339abc3c0 0x562339abe7b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e2d462663fd61d4 server_cookie=6eb133b02cadbaa9 in_seq=0 out_seq=0 2023-10-18T09:20:32.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.887+0000 7f2648ff9700 1 -- 172.21.15.61:0/1081065001 <== mds.2 v2:172.21.15.110:6836/2750873982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56233999db70 con 0x562339abc3c0 2023-10-18T09:20:32.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.887+0000 7f2648ff9700 10 client.26047 handle_command_reply: tid=0 2023-10-18T09:20:32.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.887+0000 7f267ae8db80 1 client.26047 shutdown 2023-10-18T09:20:32.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 msgr2=0x562339a62f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f264c09a960 tx=0x7f264c009c00 comp rx=0 tx=0).stop 2023-10-18T09:20:32.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 shutdown_connections 2023-10-18T09:20:32.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:6836/2750873982,v1:172.21.15.110:6837/2750873982] conn(0x562339abc3c0 0x562339abe7b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562339a53c30 0x562339a610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x562339a525d0 0x562339a62a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 --2- 172.21.15.61:0/1081065001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562339a529b0 0x562339a62f40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 >> 172.21.15.61:0/1081065001 conn(0x5623399c6720 msgr2=0x562339a5b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.890+0000 7f2648ff9700 0 client.26047 ms_handle_reset on v2:172.21.15.110:6836/2750873982 2023-10-18T09:20:32.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.891+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 shutdown_connections 2023-10-18T09:20:32.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.891+0000 7f267ae8db80 1 -- 172.21.15.61:0/1081065001 wait complete. 2023-10-18T09:20:32.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.892+0000 7f267ae8db80 20 client.26047 trim_cache size 0 max 16384 2023-10-18T09:20:32.896 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:20:32.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 msgr2=0x7f26500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 0x7f26500a39c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f26540060b0 tx=0x7f2654006040 comp rx=0 tx=0).stop 2023-10-18T09:20:32.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 msgr2=0x7f266019d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:32.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b810 tx=0x7f264c00b8f0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 shutdown_connections 2023-10-18T09:20:32.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f26500a1500 0x7f26500a39c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f26601064f0 0x7f26601a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2660101290 0x7f266019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 --2- 172.21.15.61:0/3229274780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2660103bc0 0x7f266019d2b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:32.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.893+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 >> 172.21.15.61:0/3229274780 conn(0x7f26600fac00 msgr2=0x7f2660103140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:32.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.894+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 shutdown_connections 2023-10-18T09:20:32.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:32.894+0000 7f26675ff700 1 -- 172.21.15.61:0/3229274780 wait complete. 2023-10-18T09:20:32.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-18T09:20:33.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:33 smithi061 ceph-mon[98573]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 606 op/s 2023-10-18T09:20:33.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.290+0000 7fb6c599d700 1 -- 172.21.15.61:0/3411553162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 msgr2=0x7fb6b809e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.290+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3411553162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b809e200 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb6c006a1e0 tx=0x7fb6c0068f00 comp rx=0 tx=0).stop 2023-10-18T09:20:33.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.290+0000 7fb6c599d700 1 -- 172.21.15.61:0/3411553162 shutdown_connections 2023-10-18T09:20:33.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.290+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3411553162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b809e200 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.290+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3411553162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b80961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.291+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3411553162 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b8097410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.291+0000 7fb6c599d700 1 -- 172.21.15.61:0/3411553162 >> 172.21.15.61:0/3411553162 conn(0x7fb6b8091310 msgr2=0x7fb6b8093770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.291+0000 7fb6c599d700 1 -- 172.21.15.61:0/3411553162 shutdown_connections 2023-10-18T09:20:33.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.291+0000 7fb6c599d700 1 -- 172.21.15.61:0/3411553162 wait complete. 2023-10-18T09:20:33.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.292+0000 7fb6c599d700 1 Processor -- start 2023-10-18T09:20:33.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.293+0000 7fb6c599d700 1 -- start start 2023-10-18T09:20:33.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.295+0000 7fb6c599d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.295+0000 7fb6c599d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b812a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.295+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.295+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39870/0 (socket says 172.21.15.61:39870) 2023-10-18T09:20:33.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c499b700 1 -- 172.21.15.61:0/3393066856 learned_addr learned my addr 172.21.15.61:0/3393066856 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b81310e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b81316b0 con 0x7fb6b8096fb0 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8131820 con 0x7fb6b8097950 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b81319e0 con 0x7fb6b8095dc0 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c519c700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b81310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.296+0000 7fb6c499b700 1 -- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 msgr2=0x7fb6b81310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.297+0000 7fb6c499b700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b81310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.297+0000 7fb6c499b700 1 -- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 msgr2=0x7fb6b812a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.297+0000 7fb6bffff700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b812a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6c499b700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b812a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6c499b700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c004ed50 con 0x7fb6b8095dc0 2023-10-18T09:20:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6c519c700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b81310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6c499b700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b550 tx=0x7fb6b400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6bffff700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b812a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6bdffb700 1 -- 172.21.15.61:0/3393066856 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4011830 con 0x7fb6b8095dc0 2023-10-18T09:20:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6bdffb700 1 -- 172.21.15.61:0/3393066856 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6b4011e50 con 0x7fb6b8095dc0 2023-10-18T09:20:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6bdffb700 1 -- 172.21.15.61:0/3393066856 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400f2e0 con 0x7fb6b8095dc0 2023-10-18T09:20:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.298+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b8131c60 con 0x7fb6b8095dc0 2023-10-18T09:20:33.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.299+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b81321a0 con 0x7fb6b8095dc0 2023-10-18T09:20:33.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.305+0000 7fb6bdffb700 1 -- 172.21.15.61:0/3393066856 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb6b4011990 con 0x7fb6b8095dc0 2023-10-18T09:20:33.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.308+0000 7fb6da59ab80 1 Processor -- start 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.308+0000 7fb6da59ab80 1 -- start start 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75c9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75c106e0 con 0x555c75c9d4d0 2023-10-18T09:20:33.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75bf6c60 con 0x555c75c9de30 2023-10-18T09:20:33.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75bf4580 con 0x555c75c9e7f0 2023-10-18T09:20:33.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39884/0 (socket says 172.21.15.61:39884) 2023-10-18T09:20:33.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.309+0000 7fb6bffff700 1 -- 172.21.15.61:0/2196680233 learned_addr learned my addr 172.21.15.61:0/2196680233 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.310+0000 7fb6bffff700 1 -- 172.21.15.61:0/2196680233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 msgr2=0x555c75c9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.310+0000 7fb6c519c700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75c9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.311+0000 7fb6bdffb700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 0x7fb6b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.311+0000 7fb6c499b700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6bffff700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c9e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6bffff700 1 -- 172.21.15.61:0/2196680233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 msgr2=0x555c75c9d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6bffff700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75c9d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6bffff700 1 -- 172.21.15.61:0/2196680233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c75bad3f0 con 0x555c75c9e7f0 2023-10-18T09:20:33.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6c519c700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75c9d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6bdffb700 1 -- 172.21.15.61:0/3393066856 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6b40ccb50 con 0x7fb6b8095dc0 2023-10-18T09:20:33.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.312+0000 7fb6c519c700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 0x7fb6b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.313+0000 7fb6bffff700 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00bca0 tx=0x7fb6ac00d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa6c03bba95171f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.314+0000 7fb6c519c700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 0x7fb6b00a3940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0063a70 tx=0x7fb6c0066970 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.316+0000 7fb6aaffd700 1 -- 172.21.15.61:0/2196680233 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00de80 con 0x555c75c9e7f0 2023-10-18T09:20:33.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.316+0000 7fb6aaffd700 1 -- 172.21.15.61:0/2196680233 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ac0178e0 con 0x555c75c9e7f0 2023-10-18T09:20:33.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.316+0000 7fb6aaffd700 1 -- 172.21.15.61:0/2196680233 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00fac0 con 0x555c75c9e7f0 2023-10-18T09:20:33.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.317+0000 7fb6da59ab80 1 -- 172.21.15.61:0/2196680233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 msgr2=0x555c75c9ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.317+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00bca0 tx=0x7fb6ac00d340 comp rx=0 tx=0).stop 2023-10-18T09:20:33.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 -- 172.21.15.61:0/2196680233 shutdown_connections 2023-10-18T09:20:33.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9e7f0 0x555c75c9ec50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75c9d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/2196680233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 -- 172.21.15.61:0/2196680233 >> 172.21.15.61:0/2196680233 conn(0x555c75c11720 msgr2=0x555c75c97d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 -- 172.21.15.61:0/2196680233 shutdown_connections 2023-10-18T09:20:33.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.318+0000 7fb6da59ab80 1 -- 172.21.15.61:0/2196680233 wait complete. 2023-10-18T09:20:33.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.319+0000 7fb6da59ab80 1 -- start start 2023-10-18T09:20:33.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.319+0000 7fb6da59ab80 1 Processor -- start 2023-10-18T09:20:33.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75cb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9de30 0x555c75cb8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75befe70 con 0x555c75c9d4d0 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75b8c710 con 0x555c75c9e7f0 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75b8c3b0 con 0x555c75c9de30 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.320+0000 7fb6da59ab80 10 client.0 _ll_register_callbacks cb 0x555c75ad6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:33.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9de30 0x555c75cb8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39804/0 (socket says 172.21.15.61:39804) 2023-10-18T09:20:33.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 -- 172.21.15.61:0/1039295205 learned_addr learned my addr 172.21.15.61:0/1039295205 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75cb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 -- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9de30 msgr2=0x555c75cb8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9de30 0x555c75cb8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 -- 172.21.15.61:0/1039295205 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 msgr2=0x555c75cb7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75cb7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.321+0000 7fb6c519c700 1 -- 172.21.15.61:0/1039295205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac00b8d0 con 0x555c75c9e7f0 2023-10-18T09:20:33.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.322+0000 7fb6c519c700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0064030 tx=0x7fb6c0076040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6eab766fbd8e52 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.322+0000 7fb69ffff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.322+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0068a10 con 0x555c75c9e7f0 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.322+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6c0071430 con 0x555c75c9e7f0 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.323+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c75bf4ef0 con 0x555c75c9e7f0 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.324+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6c00715f0 con 0x555c75c9e7f0 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.324+0000 7fb6da59ab80 10 client.26054 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:33.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.324+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555c75a1ade0 con 0x555c75c9e7f0 2023-10-18T09:20:33.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c75bf4ef0 con 0x555c75c9e7f0 2023-10-18T09:20:33.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb6c00977f0 con 0x555c75c9e7f0 2023-10-18T09:20:33.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 10 client.26054 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:33.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 10 client.26054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:20:33.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 20 client.26054 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:33.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75cc3ae0 0x555c75cc5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 4 client.26054 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:33.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c75c6c8b0 con 0x555c75cc3ae0 2023-10-18T09:20:33.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.325+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75cc3ae0 0x555c75cc5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.326+0000 7fb69ffff700 10 client.26054 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:20:33.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.326+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75cc3ae0 0x555c75cc5f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bce8c1a3b666d56 server_cookie=961edc152f8c810 in_seq=0 out_seq=0 2023-10-18T09:20:33.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb69ffff700 1 -- 172.21.15.61:0/1039295205 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c75c6c8b0 con 0x555c75cc3ae0 2023-10-18T09:20:33.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb69ffff700 10 client.26054 handle_command_reply: tid=0 2023-10-18T09:20:33.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 client.26054 shutdown 2023-10-18T09:20:33.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 msgr2=0x555c75cbf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0064030 tx=0x7fb6c0076040 comp rx=0 tx=0).stop 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 shutdown_connections 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75cc3ae0 0x555c75cc5f00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbf270 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9d4d0 0x555c75cb7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1039295205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9de30 0x555c75cb8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.327+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 >> 172.21.15.61:0/1039295205 conn(0x555c75c11720 msgr2=0x555c75c9c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.328+0000 7fb69ffff700 0 client.26054 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:20:33.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.328+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 shutdown_connections 2023-10-18T09:20:33.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.328+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1039295205 wait complete. 2023-10-18T09:20:33.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.328+0000 7fb6da59ab80 20 client.26054 trim_cache size 0 max 16384 2023-10-18T09:20:33.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 Processor -- start 2023-10-18T09:20:33.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 -- start start 2023-10-18T09:20:33.358 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c92820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 0x555c75cbf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75a70920 con 0x555c75c9e7f0 2023-10-18T09:20:33.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75cc03f0 con 0x555c75c9de30 2023-10-18T09:20:33.359 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75c99010 con 0x555c75c9d4d0 2023-10-18T09:20:33.360 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39900/0 (socket says 172.21.15.61:39900) 2023-10-18T09:20:33.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.356+0000 7fb6c499b700 1 -- 172.21.15.61:0/1446813177 learned_addr learned my addr 172.21.15.61:0/1446813177 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6bffff700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c92820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c499b700 1 -- 172.21.15.61:0/1446813177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 msgr2=0x555c75c92820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c92820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c499b700 1 -- 172.21.15.61:0/1446813177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 msgr2=0x555c75cbf190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:20:33.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c519c700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 0x555c75cbf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 0x555c75cbf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c499b700 1 -- 172.21.15.61:0/1446813177 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c75a03c60 con 0x555c75c9d4d0 2023-10-18T09:20:33.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.357+0000 7fb6c519c700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 0x555c75cbf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.358+0000 7fb6c499b700 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b520 tx=0x7fb6b40ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f651ef0d211ad1d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.358+0000 7fb6ab7fe700 1 -- 172.21.15.61:0/1446813177 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40b1070 con 0x555c75c9d4d0 2023-10-18T09:20:33.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.358+0000 7fb6ab7fe700 1 -- 172.21.15.61:0/1446813177 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555c75c6c8b0 con 0x555c75c9d4d0 2023-10-18T09:20:33.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.358+0000 7fb6ab7fe700 1 -- 172.21.15.61:0/1446813177 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4097c60 con 0x555c75c9d4d0 2023-10-18T09:20:33.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1446813177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 msgr2=0x555c75cbf920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b520 tx=0x7fb6b40ce040 comp rx=0 tx=0).stop 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1446813177 shutdown_connections 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9e7f0 0x555c75cbf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbf920 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/1446813177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9de30 0x555c75c92820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.359+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1446813177 >> 172.21.15.61:0/1446813177 conn(0x555c75c11720 msgr2=0x555c75cca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.360+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1446813177 shutdown_connections 2023-10-18T09:20:33.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.360+0000 7fb6da59ab80 1 -- 172.21.15.61:0/1446813177 wait complete. 2023-10-18T09:20:33.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.361+0000 7fb6da59ab80 1 -- start start 2023-10-18T09:20:33.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.361+0000 7fb6da59ab80 1 Processor -- start 2023-10-18T09:20:33.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9de30 0x555c75cbd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75cbe0b0 con 0x555c75c9de30 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75cbe220 con 0x555c75c9e7f0 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c75cbe3e0 con 0x555c75c9d4d0 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6da59ab80 10 client.0 _ll_register_callbacks cb 0x555c75ad6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:33.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39910/0 (socket says 172.21.15.61:39910) 2023-10-18T09:20:33.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 -- 172.21.15.61:0/540938236 learned_addr learned my addr 172.21.15.61:0/540938236 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 -- 172.21.15.61:0/540938236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 msgr2=0x555c75cbdb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:20:33.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbdb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 -- 172.21.15.61:0/540938236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9de30 msgr2=0x555c75cbd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9de30 0x555c75cbd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.362+0000 7fb6c499b700 1 -- 172.21.15.61:0/540938236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b81321a0 con 0x555c75c9d4d0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb6c499b700 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40ca350 tx=0x7fb6b40cce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f05663b3b03831 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb69ffff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40b1050 con 0x555c75c9d4d0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6b40a1eb0 con 0x555c75c9d4d0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6ac0178e0 con 0x555c75c9d4d0 2023-10-18T09:20:33.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b4097ec0 con 0x555c75c9d4d0 2023-10-18T09:20:33.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb6da59ab80 10 client.26062 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:33.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.363+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555c75a41250 con 0x555c75c9d4d0 2023-10-18T09:20:33.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ac0178e0 con 0x555c75c9d4d0 2023-10-18T09:20:33.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb6b4099010 con 0x555c75c9d4d0 2023-10-18T09:20:33.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 10 client.26062 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:33.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 10 client.26062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-18T09:20:33.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 20 client.26062 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:33.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75d18e80 0x555c75d1b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 4 client.26062 mds_command: new command op to 24676 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:20:33.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.364+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 --> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c75be8e50 con 0x555c75d18e80 2023-10-18T09:20:33.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.365+0000 7fb6bffff700 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75d18e80 0x555c75d1b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.367+0000 7fb69ffff700 10 client.26062 ms_handle_connect on v2:172.21.15.170:6838/2453939791 2023-10-18T09:20:33.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.367+0000 7fb6bffff700 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75d18e80 0x555c75d1b2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c4b6e12c721e92b server_cookie=8c29f3ff7bfea1ac in_seq=0 out_seq=0 2023-10-18T09:20:33.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb69ffff700 1 -- 172.21.15.61:0/540938236 <== mds.1 v2:172.21.15.170:6838/2453939791 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c75be8e50 con 0x555c75d18e80 2023-10-18T09:20:33.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb69ffff700 10 client.26062 handle_command_reply: tid=0 2023-10-18T09:20:33.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 client.26062 shutdown 2023-10-18T09:20:33.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 msgr2=0x555c75cbeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40ca350 tx=0x7fb6b40cce10 comp rx=0 tx=0).stop 2023-10-18T09:20:33.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 shutdown_connections 2023-10-18T09:20:33.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:6838/2453939791,v1:172.21.15.170:6839/2453939791] conn(0x555c75d18e80 0x555c75d1b2a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555c75c9e7f0 0x555c75cbdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555c75c9d4d0 0x555c75cbeef0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 --2- 172.21.15.61:0/540938236 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x555c75c9de30 0x555c75cbd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb69ffff700 0 client.26062 ms_handle_reset on v2:172.21.15.170:6838/2453939791 2023-10-18T09:20:33.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.368+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 >> 172.21.15.61:0/540938236 conn(0x555c75c11720 msgr2=0x555c75ca4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.369+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 shutdown_connections 2023-10-18T09:20:33.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.369+0000 7fb6da59ab80 1 -- 172.21.15.61:0/540938236 wait complete. 2023-10-18T09:20:33.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.369+0000 7fb6da59ab80 20 client.26062 trim_cache size 0 max 16384 2023-10-18T09:20:33.378 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:20:33.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 msgr2=0x7fb6b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 0x7fb6b00a3940 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0063a70 tx=0x7fb6c0066970 comp rx=0 tx=0).stop 2023-10-18T09:20:33.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 msgr2=0x7fb6b8129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400b550 tx=0x7fb6b400b910 comp rx=0 tx=0).stop 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 shutdown_connections 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6b00a1480 0x7fb6b00a3940 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6b8097950 0x7fb6b81310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6b8095dc0 0x7fb6b8129e30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 --2- 172.21.15.61:0/3393066856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6b8096fb0 0x7fb6b812a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 >> 172.21.15.61:0/3393066856 conn(0x7fb6b8091310 msgr2=0x7fb6b80920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.370+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 shutdown_connections 2023-10-18T09:20:33.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.371+0000 7fb6c599d700 1 -- 172.21.15.61:0/3393066856 wait complete. 2023-10-18T09:20:33.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:33 smithi170 ceph-mon[135150]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 606 op/s 2023-10-18T09:20:33.404 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-18T09:20:33.461 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:33 smithi110 ceph-mon[134818]: pgmap v418: 129 pgs: 129 active+clean; 1.5 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 13 MiB/s wr, 606 op/s 2023-10-18T09:20:33.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 -- 172.21.15.61:0/4059773689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809c3a0 msgr2=0x7f8a8809c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 --2- 172.21.15.61:0/4059773689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809c3a0 0x7f8a8809c7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009920 tx=0x7f8a80009c30 comp rx=0 tx=0).stop 2023-10-18T09:20:33.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 -- 172.21.15.61:0/4059773689 shutdown_connections 2023-10-18T09:20:33.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 --2- 172.21.15.61:0/4059773689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 0x7f8a880a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 --2- 172.21.15.61:0/4059773689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809c3a0 0x7f8a8809c7c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 --2- 172.21.15.61:0/4059773689 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809d590 0x7f8a8809d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.794+0000 7f8a9699d700 1 -- 172.21.15.61:0/4059773689 >> 172.21.15.61:0/4059773689 conn(0x7f8a8800b340 msgr2=0x7f8a8800b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.795+0000 7f8a9699d700 1 -- 172.21.15.61:0/4059773689 shutdown_connections 2023-10-18T09:20:33.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.795+0000 7f8a9699d700 1 -- 172.21.15.61:0/4059773689 wait complete. 2023-10-18T09:20:33.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.797+0000 7f8a9699d700 1 Processor -- start 2023-10-18T09:20:33.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.797+0000 7f8a9699d700 1 -- start start 2023-10-18T09:20:33.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 0x7f8a88132410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 0x7f8a88139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a88139f40 con 0x7f8a8809c3a0 2023-10-18T09:20:33.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8814b090 con 0x7f8a8809d590 2023-10-18T09:20:33.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9699d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8814b220 con 0x7f8a8809df30 2023-10-18T09:20:33.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39836/0 (socket says 172.21.15.61:39836) 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9519a700 1 -- 172.21.15.61:0/2252597951 learned_addr learned my addr 172.21.15.61:0/2252597951 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.798+0000 7f8a9619c700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 0x7f8a88139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 -- 172.21.15.61:0/2252597951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 msgr2=0x7f8a88139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 0x7f8a88139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 -- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 msgr2=0x7f8a88132410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9599b700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 0x7f8a88132410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 0x7f8a88132410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 -- 172.21.15.61:0/2252597951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009520 con 0x7f8a8809d590 2023-10-18T09:20:33.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9599b700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 0x7f8a88132410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e9a0 tx=0x7f8a8c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a86ffd700 1 -- 172.21.15.61:0/2252597951 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c009790 con 0x7f8a8809d590 2023-10-18T09:20:33.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a86ffd700 1 -- 172.21.15.61:0/2252597951 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c00cbd0 con 0x7f8a8809d590 2023-10-18T09:20:33.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.799+0000 7f8a86ffd700 1 -- 172.21.15.61:0/2252597951 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c018680 con 0x7f8a8809d590 2023-10-18T09:20:33.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.800+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8814b500 con 0x7f8a8809d590 2023-10-18T09:20:33.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.800+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8814b9c0 con 0x7f8a8809d590 2023-10-18T09:20:33.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.807+0000 7f8a86ffd700 1 -- 172.21.15.61:0/2252597951 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8a8c00cd40 con 0x7f8a8809d590 2023-10-18T09:20:33.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 Processor -- start 2023-10-18T09:20:33.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 -- start start 2023-10-18T09:20:33.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5653047694d0 0x5653047698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565304769e30 0x56530476a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653046dc6e0 con 0x5653047694d0 2023-10-18T09:20:33.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653046c2c60 con 0x56530476a7f0 2023-10-18T09:20:33.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.809+0000 7f8aab558b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653046c0580 con 0x565304769e30 2023-10-18T09:20:33.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39842/0 (socket says 172.21.15.61:39842) 2023-10-18T09:20:33.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 -- 172.21.15.61:0/2320531133 learned_addr learned my addr 172.21.15.61:0/2320531133 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9619c700 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5653047694d0 0x5653047698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 -- 172.21.15.61:0/2320531133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565304769e30 msgr2=0x56530476a2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:20:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565304769e30 0x56530476a2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 -- 172.21.15.61:0/2320531133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5653047694d0 msgr2=0x5653047698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5653047694d0 0x5653047698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.810+0000 7f8a9519a700 1 -- 172.21.15.61:0/2320531133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653046793f0 con 0x56530476a7f0 2023-10-18T09:20:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a9519a700 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0d0fd0 tx=0x7f8a8c0d53c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25bbc185fc86142 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a7bfff700 1 -- 172.21.15.61:0/2320531133 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0d5ab0 con 0x56530476a7f0 2023-10-18T09:20:33.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a7bfff700 1 -- 172.21.15.61:0/2320531133 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c0d5c10 con 0x56530476a7f0 2023-10-18T09:20:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a7bfff700 1 -- 172.21.15.61:0/2320531133 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0db660 con 0x56530476a7f0 2023-10-18T09:20:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a86ffd700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 0x7f8a7c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.811+0000 7f8a86ffd700 1 -- 172.21.15.61:0/2252597951 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8a8c014070 con 0x7f8a8809d590 2023-10-18T09:20:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 -- 172.21.15.61:0/2320531133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 msgr2=0x56530476ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c0d0fd0 tx=0x7f8a8c0d53c0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8a9619c700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 0x7f8a7c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 -- 172.21.15.61:0/2320531133 shutdown_connections 2023-10-18T09:20:33.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x56530476ac50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5653047694d0 0x5653047698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 --2- 172.21.15.61:0/2320531133 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x565304769e30 0x56530476a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 -- 172.21.15.61:0/2320531133 >> 172.21.15.61:0/2320531133 conn(0x5653046dd720 msgr2=0x565304763d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.812+0000 7f8aab558b80 1 -- 172.21.15.61:0/2320531133 shutdown_connections 2023-10-18T09:20:33.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.813+0000 7f8aab558b80 1 -- 172.21.15.61:0/2320531133 wait complete. 2023-10-18T09:20:33.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.813+0000 7f8a9619c700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 0x7f8a7c0a39c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a90000eb0 tx=0x7f8a90076040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.814+0000 7f8aab558b80 1 -- start start 2023-10-18T09:20:33.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.814+0000 7f8aab558b80 1 Processor -- start 2023-10-18T09:20:33.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304772ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x56530477a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653046bbe70 con 0x56530476a7f0 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x565304658710 con 0x565304769e30 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653046583b0 con 0x5653047694d0 2023-10-18T09:20:33.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.817+0000 7f8aab558b80 10 client.0 _ll_register_callbacks cb 0x5653045a2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:33.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39856/0 (socket says 172.21.15.61:39856) 2023-10-18T09:20:33.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 -- 172.21.15.61:0/1566951511 learned_addr learned my addr 172.21.15.61:0/1566951511 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9619c700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x56530477a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.823 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 -- 172.21.15.61:0/1566951511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 msgr2=0x565304772ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:20:33.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304772ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 -- 172.21.15.61:0/1566951511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 msgr2=0x56530477a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x56530477a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.818+0000 7f8a9519a700 1 -- 172.21.15.61:0/1566951511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8814b9c0 con 0x565304769e30 2023-10-18T09:20:33.824 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.819+0000 7f8a9519a700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c096720 tx=0x7f8a8c0d0980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a1d4662e77c543d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.819+0000 7f8a797fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:33.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.819+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0d5ab0 con 0x565304769e30 2023-10-18T09:20:33.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.819+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8a8c09a920 con 0x565304769e30 2023-10-18T09:20:33.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.820+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5653046c0ef0 con 0x565304769e30 2023-10-18T09:20:33.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.820+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a8c0d0c60 con 0x565304769e30 2023-10-18T09:20:33.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.821+0000 7f8aab558b80 10 client.26069 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:33.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.821+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5653044e6de0 con 0x565304769e30 2023-10-18T09:20:33.826 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.821+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5653046c0ef0 con 0x565304769e30 2023-10-18T09:20:33.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8a8c09a480 con 0x565304769e30 2023-10-18T09:20:33.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:33.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 20 client.26069 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56530477e7f0 0x565304780c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 4 client.26069 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.822+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653047388b0 con 0x56530477e7f0 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.823+0000 7f8a9599b700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56530477e7f0 0x565304780c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.825+0000 7f8a9599b700 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56530477e7f0 0x565304780c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224a01e6485de4f4 server_cookie=7cb6a74b046984d9 in_seq=0 out_seq=0 2023-10-18T09:20:33.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.825+0000 7f8a797fa700 10 client.26069 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.825+0000 7f8a797fa700 1 -- 172.21.15.61:0/1566951511 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5653047388b0 con 0x56530477e7f0 2023-10-18T09:20:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8a797fa700 10 client.26069 handle_command_reply: tid=0 2023-10-18T09:20:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 client.26069 shutdown 2023-10-18T09:20:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 msgr2=0x5653047731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c096720 tx=0x7f8a8c0d0980 comp rx=0 tx=0).stop 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 shutdown_connections 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56530477e7f0 0x565304780c10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x56530477a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304772ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 --2- 172.21.15.61:0/1566951511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x5653047731e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 >> 172.21.15.61:0/1566951511 conn(0x5653046dd720 msgr2=0x5653047682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.826+0000 7f8a797fa700 0 client.26069 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:33.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.827+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 shutdown_connections 2023-10-18T09:20:33.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.827+0000 7f8aab558b80 1 -- 172.21.15.61:0/1566951511 wait complete. 2023-10-18T09:20:33.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.827+0000 7f8aab558b80 20 client.26069 trim_cache size 0 max 16384 2023-10-18T09:20:33.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 Processor -- start 2023-10-18T09:20:33.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 -- start start 2023-10-18T09:20:33.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x56530475e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x565304771e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56530453c920 con 0x56530476a7f0 2023-10-18T09:20:33.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x565304769140 con 0x565304769e30 2023-10-18T09:20:33.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.852+0000 7f8aab558b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x565304773e90 con 0x5653047694d0 2023-10-18T09:20:33.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39868/0 (socket says 172.21.15.61:39868) 2023-10-18T09:20:33.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 -- 172.21.15.61:0/3864433815 learned_addr learned my addr 172.21.15.61:0/3864433815 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9619c700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x565304771e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 -- 172.21.15.61:0/3864433815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 msgr2=0x56530475e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9599b700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x56530475e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x56530475e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 -- 172.21.15.61:0/3864433815 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 msgr2=0x565304771e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x565304771e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 -- 172.21.15.61:0/3864433815 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653044cfc60 con 0x565304769e30 2023-10-18T09:20:33.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9619c700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x565304771e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:33.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9599b700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x56530475e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:33.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.853+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c096c40 tx=0x7f8a8c0d53c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1769605a30c48354 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8a84bff700 1 -- 172.21.15.61:0/3864433815 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c096560 con 0x565304769e30 2023-10-18T09:20:33.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8a84bff700 1 -- 172.21.15.61:0/3864433815 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c099ff0 con 0x565304769e30 2023-10-18T09:20:33.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8a84bff700 1 -- 172.21.15.61:0/3864433815 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c09a220 con 0x565304769e30 2023-10-18T09:20:33.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 -- 172.21.15.61:0/3864433815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 msgr2=0x56530477a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c096c40 tx=0x7f8a8c0d53c0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 -- 172.21.15.61:0/3864433815 shutdown_connections 2023-10-18T09:20:33.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56530476a7f0 0x565304771e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x56530475e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3864433815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x565304769e30 0x56530477a1e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 -- 172.21.15.61:0/3864433815 >> 172.21.15.61:0/3864433815 conn(0x5653046dd720 msgr2=0x565304772ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.854+0000 7f8aab558b80 1 -- 172.21.15.61:0/3864433815 shutdown_connections 2023-10-18T09:20:33.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.855+0000 7f8aab558b80 1 -- 172.21.15.61:0/3864433815 wait complete. 2023-10-18T09:20:33.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.855+0000 7f8aab558b80 1 -- start start 2023-10-18T09:20:33.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.855+0000 7f8aab558b80 1 Processor -- start 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304777b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x565304769e30 0x565304778040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653047794c0 con 0x565304769e30 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x565304779630 con 0x56530476a7f0 2023-10-18T09:20:33.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653047797f0 con 0x5653047694d0 2023-10-18T09:20:33.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8aab558b80 10 client.0 _ll_register_callbacks cb 0x5653045a2a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:33.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9619c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304777b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9619c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39880/0 (socket says 172.21.15.61:39880) 2023-10-18T09:20:33.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9619c700 1 -- 172.21.15.61:0/3329408396 learned_addr learned my addr 172.21.15.61:0/3329408396 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x565304769e30 0x565304778040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 -- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 msgr2=0x565304778f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 -- 172.21.15.61:0/3329408396 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x565304769e30 msgr2=0x565304778040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x565304769e30 0x565304778040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.856+0000 7f8a9599b700 1 -- 172.21.15.61:0/3329408396 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653044e6de0 con 0x5653047694d0 2023-10-18T09:20:33.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a9619c700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:33.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a9599b700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304777b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000f9e0 tx=0x7f8a8000bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65c38f147ce807f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:33.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a797fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:33.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8001c040 con 0x5653047694d0 2023-10-18T09:20:33.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a8c0d5c10 con 0x5653047694d0 2023-10-18T09:20:33.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8a8000f460 con 0x5653047694d0 2023-10-18T09:20:33.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a800235c0 con 0x5653047694d0 2023-10-18T09:20:33.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8aab558b80 10 client.26071 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:33.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.857+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56530450d250 con 0x5653047694d0 2023-10-18T09:20:33.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a8c0d5c10 con 0x5653047694d0 2023-10-18T09:20:33.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f8a80015020 con 0x5653047694d0 2023-10-18T09:20:33.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:33.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:33.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 20 client.26071 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:33.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5653047d3070 0x5653047d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 4 client.26071 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5653046b4e50 con 0x5653047d3070 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.858+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5653047d3070 0x5653047d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8a9519a700 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5653047d3070 0x5653047d5490 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a3131c6bed2a9c server_cookie=75de55b4d8222678 in_seq=0 out_seq=0 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8a797fa700 10 client.26071 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:33.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8a797fa700 1 -- 172.21.15.61:0/3329408396 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5653046b4e50 con 0x5653047d3070 2023-10-18T09:20:33.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8a797fa700 10 client.26071 handle_command_reply: tid=0 2023-10-18T09:20:33.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 client.26071 shutdown 2023-10-18T09:20:33.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 msgr2=0x565304777b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304777b00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000f9e0 tx=0x7f8a8000bf20 comp rx=0 tx=0).stop 2023-10-18T09:20:33.873 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 shutdown_connections 2023-10-18T09:20:33.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5653047d3070 0x5653047d5490 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56530476a7f0 0x565304778f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653047694d0 0x565304777b00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 --2- 172.21.15.61:0/3329408396 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x565304769e30 0x565304778040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 >> 172.21.15.61:0/3329408396 conn(0x5653046dd720 msgr2=0x5653047682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.859+0000 7f8a797fa700 0 client.26071 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:33.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 shutdown_connections 2023-10-18T09:20:33.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8aab558b80 1 -- 172.21.15.61:0/3329408396 wait complete. 2023-10-18T09:20:33.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8aab558b80 20 client.26071 trim_cache size 0 max 16384 2023-10-18T09:20:33.876 INFO:teuthology.orchestra.run.smithi061.stdout:[] 2023-10-18T09:20:33.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 msgr2=0x7f8a7c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 0x7f8a7c0a39c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a90000eb0 tx=0x7f8a90076040 comp rx=0 tx=0).stop 2023-10-18T09:20:33.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 msgr2=0x7f8a88132950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:33.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.860+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00e9a0 tx=0x7f8a8c00ecb0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 shutdown_connections 2023-10-18T09:20:33.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8a7c0a1500 0x7f8a7c0a39c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a8809df30 0x7f8a88139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8a8809c3a0 0x7f8a88132410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 --2- 172.21.15.61:0/2252597951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a8809d590 0x7f8a88132950 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:33.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 >> 172.21.15.61:0/2252597951 conn(0x7f8a8800b340 msgr2=0x7f8a880a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:33.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 shutdown_connections 2023-10-18T09:20:33.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:33.861+0000 7f8a9699d700 1 -- 172.21.15.61:0/2252597951 wait complete. 2023-10-18T09:20:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 80.0 s, 1640.6 tps, lat 61.685 ms stddev 79.309 2023-10-18T09:20:34.882 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-18T09:20:34.882 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-18T09:20:35.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 -- 172.21.15.61:0/1339148768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68280ff000 msgr2=0x7f68280ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1339148768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68280ff000 0x7f68280ff420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6818009990 tx=0x7f6818009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 -- 172.21.15.61:0/1339148768 shutdown_connections 2023-10-18T09:20:35.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1339148768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6828100320 0x7f68281007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1339148768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68280ff000 0x7f68280ff420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1339148768 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68280ff960 0x7f68280ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 -- 172.21.15.61:0/1339148768 >> 172.21.15.61:0/1339148768 conn(0x7f68280fac00 msgr2=0x7f68280fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.230+0000 7f682f1ff700 1 -- 172.21.15.61:0/1339148768 shutdown_connections 2023-10-18T09:20:35.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.231+0000 7f682f1ff700 1 -- 172.21.15.61:0/1339148768 wait complete. 2023-10-18T09:20:35.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.232+0000 7f682f1ff700 1 Processor -- start 2023-10-18T09:20:35.235 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.232+0000 7f682f1ff700 1 -- start start 2023-10-18T09:20:35.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68280ff960 0x7f6828198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6828100320 0x7f682819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a04d0 con 0x7f68280ff960 2023-10-18T09:20:35.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a0610 con 0x7f68280ff000 2023-10-18T09:20:35.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682f1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68281a07a0 con 0x7f6828100320 2023-10-18T09:20:35.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39892/0 (socket says 172.21.15.61:39892) 2023-10-18T09:20:35.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682cf9b700 1 -- 172.21.15.61:0/1146101702 learned_addr learned my addr 172.21.15.61:0/1146101702 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.233+0000 7f682d79c700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6828100320 0x7f682819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 -- 172.21.15.61:0/1146101702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6828100320 msgr2=0x7f682819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6828100320 0x7f682819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 -- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68280ff960 msgr2=0x7f6828198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68280ff960 0x7f6828198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 -- 172.21.15.61:0/1146101702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820009c40 con 0x7f68280ff000 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6818000c00 tx=0x7f681800f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f6825ffb700 1 -- 172.21.15.61:0/1146101702 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681801d070 con 0x7f68280ff000 2023-10-18T09:20:35.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f6825ffb700 1 -- 172.21.15.61:0/1146101702 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f681800fb70 con 0x7f68280ff000 2023-10-18T09:20:35.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.234+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68180095f0 con 0x7f68280ff000 2023-10-18T09:20:35.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.235+0000 7f6825ffb700 1 -- 172.21.15.61:0/1146101702 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800e3e0 con 0x7f68280ff000 2023-10-18T09:20:35.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.235+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68281014a0 con 0x7f68280ff000 2023-10-18T09:20:35.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.241+0000 7f6825ffb700 1 -- 172.21.15.61:0/1146101702 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f68180173f0 con 0x7f68280ff000 2023-10-18T09:20:35.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.242+0000 7f6842a9ab80 1 Processor -- start 2023-10-18T09:20:35.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.243+0000 7f6842a9ab80 1 -- start start 2023-10-18T09:20:35.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576ecc760 con 0x55b576f5a350 2023-10-18T09:20:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576eb2ce0 con 0x55b576f59650 2023-10-18T09:20:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576eb0600 con 0x55b576f5acb0 2023-10-18T09:20:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f59110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39992/0 (socket says 172.21.15.61:39992) 2023-10-18T09:20:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f682cf9b700 1 -- 172.21.15.61:0/2577560836 learned_addr learned my addr 172.21.15.61:0/2577560836 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.244+0000 7f6827fff700 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 -- 172.21.15.61:0/2577560836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 msgr2=0x55b576f59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 -- 172.21.15.61:0/2577560836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 msgr2=0x55b576f5a770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:20:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f5a770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 -- 172.21.15.61:0/2577560836 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b576e69470 con 0x55b576f59650 2023-10-18T09:20:35.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f6827fff700 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f59ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097f0 tx=0x7f681c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be61190cb7fabc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f680effd700 1 -- 172.21.15.61:0/2577560836 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011830 con 0x55b576f59650 2023-10-18T09:20:35.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f680effd700 1 -- 172.21.15.61:0/2577560836 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f681c011e50 con 0x55b576f59650 2023-10-18T09:20:35.251 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.245+0000 7f680effd700 1 -- 172.21.15.61:0/2577560836 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c010380 con 0x55b576f59650 2023-10-18T09:20:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.246+0000 7f6825ffb700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 0x7f68100ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.246+0000 7f6825ffb700 1 -- 172.21.15.61:0/1146101702 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f68180ce6c0 con 0x7f68280ff000 2023-10-18T09:20:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.246+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2577560836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 msgr2=0x55b576f59ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.246+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f59ad0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097f0 tx=0x7f681c009b00 comp rx=0 tx=0).stop 2023-10-18T09:20:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.246+0000 7f682d79c700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 0x7f68100ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.247+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2577560836 shutdown_connections 2023-10-18T09:20:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.247+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f59ad0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.247+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.247+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2577560836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.247+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2577560836 >> 172.21.15.61:0/2577560836 conn(0x55b576ecd7a0 msgr2=0x55b576f53bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.248+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2577560836 shutdown_connections 2023-10-18T09:20:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.248+0000 7f682d79c700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 0x7f68100ac2d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6820009fd0 tx=0x7f682000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.248+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2577560836 wait complete. 2023-10-18T09:20:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.249+0000 7f6842a9ab80 1 -- start start 2023-10-18T09:20:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.250+0000 7f6842a9ab80 1 Processor -- start 2023-10-18T09:20:35.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:35 smithi061 ceph-mon[98573]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-10-18T09:20:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 0x55b576f62b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 0x55b576f630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f6a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576eabef0 con 0x55b576f59650 2023-10-18T09:20:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576e48790 con 0x55b576f5a350 2023-10-18T09:20:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576e48430 con 0x55b576f5acb0 2023-10-18T09:20:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.251+0000 7f6827fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 0x55b576f630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f6a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f6827fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 0x55b576f630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39908/0 (socket says 172.21.15.61:39908) 2023-10-18T09:20:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f6827fff700 1 -- 172.21.15.61:0/1419310497 learned_addr learned my addr 172.21.15.61:0/1419310497 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f6842a9ab80 10 client.0 _ll_register_callbacks cb 0x55b576d92b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:35.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 0x55b576f62b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 -- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 msgr2=0x55b576f630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 0x55b576f630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.260 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 -- 172.21.15.61:0/1419310497 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 msgr2=0x55b576f62b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 0x55b576f62b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682d79c700 1 -- 172.21.15.61:0/1419310497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c009420 con 0x55b576f5acb0 2023-10-18T09:20:35.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.252+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 0x55b576f62b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:35.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.253+0000 7f682d79c700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f6a110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f682000fe00 tx=0x7f6820004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999c413d5c8fbd63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.261 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.253+0000 7f6807fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:20:35.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.254+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682001c040 con 0x55b576f5acb0 2023-10-18T09:20:35.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.254+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6820017ce0 con 0x55b576f5acb0 2023-10-18T09:20:35.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.254+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b576eb0f70 con 0x55b576f5acb0 2023-10-18T09:20:35.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.254+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f682000e040 con 0x55b576f5acb0 2023-10-18T09:20:35.262 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.254+0000 7f6842a9ab80 10 client.26083 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:35.263 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b576cd6e60 con 0x55b576f5acb0 2023-10-18T09:20:35.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b576eb0f70 con 0x55b576f5acb0 2023-10-18T09:20:35.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f6820015480 con 0x55b576f5acb0 2023-10-18T09:20:35.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 10 client.26083 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:35.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 10 client.26083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 20 client.26083 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f6eb50 0x55b576f70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 4 client.26083 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.255+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b576f28930 con 0x55b576f6eb50 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.256+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f6eb50 0x55b576f70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.265 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.256+0000 7f682cf9b700 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f6eb50 0x55b576f70f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=517ec5ceeddf7507 server_cookie=3a4a55dd636de99e in_seq=0 out_seq=0 2023-10-18T09:20:35.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.256+0000 7f6807fff700 10 client.26083 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.257+0000 7f6807fff700 1 -- 172.21.15.61:0/1419310497 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b576f28930 con 0x55b576f6eb50 2023-10-18T09:20:35.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.257+0000 7f6807fff700 10 client.26083 handle_command_reply: tid=0 2023-10-18T09:20:35.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.258+0000 7f6842a9ab80 1 client.26083 shutdown 2023-10-18T09:20:35.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 msgr2=0x55b576f6a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f6a110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f682000fe00 tx=0x7f6820004ad0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 shutdown_connections 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f6eb50 0x55b576f70f90 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5acb0 0x55b576f6a110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f59650 0x55b576f62b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/1419310497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f5a350 0x55b576f630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 >> 172.21.15.61:0/1419310497 conn(0x55b576ecd7a0 msgr2=0x55b576f53ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6807fff700 0 client.26083 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.259+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 shutdown_connections 2023-10-18T09:20:35.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.260+0000 7f6842a9ab80 1 -- 172.21.15.61:0/1419310497 wait complete. 2023-10-18T09:20:35.268 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.260+0000 7f6842a9ab80 20 client.26083 trim_cache size 0 max 16384 2023-10-18T09:20:35.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.296+0000 7f6842a9ab80 1 Processor -- start 2023-10-18T09:20:35.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.300+0000 7f6842a9ab80 1 -- start start 2023-10-18T09:20:35.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576ecd7a0 0x55b576f59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5a350 0x55b576f6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576d2c9a0 con 0x55b576ecd7a0 2023-10-18T09:20:35.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576f63be0 con 0x55b576f59650 2023-10-18T09:20:35.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.301+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576f6be20 con 0x55b576f5a350 2023-10-18T09:20:35.304 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.302+0000 7f6827fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.302+0000 7f6827fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39924/0 (socket says 172.21.15.61:39924) 2023-10-18T09:20:35.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.302+0000 7f6827fff700 1 -- 172.21.15.61:0/2295271758 learned_addr learned my addr 172.21.15.61:0/2295271758 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.302+0000 7f682cf9b700 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576ecd7a0 0x55b576f59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.305 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.303+0000 7f6827fff700 1 -- 172.21.15.61:0/2295271758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5a350 msgr2=0x55b576f6a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.303+0000 7f6827fff700 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5a350 0x55b576f6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.303+0000 7f6827fff700 1 -- 172.21.15.61:0/2295271758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576ecd7a0 msgr2=0x55b576f59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.303+0000 7f6827fff700 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576ecd7a0 0x55b576f59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.306 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.303+0000 7f6827fff700 1 -- 172.21.15.61:0/2295271758 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b576cbfce0 con 0x55b576f59650 2023-10-18T09:20:35.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.304+0000 7f6827fff700 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f681c006c90 tx=0x7f681c0048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bd24b6a67620cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.305+0000 7f680f7fe700 1 -- 172.21.15.61:0/2295271758 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c01f070 con 0x55b576f59650 2023-10-18T09:20:35.307 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.305+0000 7f680f7fe700 1 -- 172.21.15.61:0/2295271758 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f681c00b820 con 0x55b576f59650 2023-10-18T09:20:35.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.305+0000 7f680f7fe700 1 -- 172.21.15.61:0/2295271758 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011bb0 con 0x55b576f59650 2023-10-18T09:20:35.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2295271758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 msgr2=0x55b576f631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f681c006c90 tx=0x7f681c0048a0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2295271758 shutdown_connections 2023-10-18T09:20:35.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f5a350 0x55b576f6a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576ecd7a0 0x55b576f59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2295271758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576f59650 0x55b576f631d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.306+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2295271758 >> 172.21.15.61:0/2295271758 conn(0x55b576f53f00 msgr2=0x55b576f4d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.307+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2295271758 shutdown_connections 2023-10-18T09:20:35.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.308+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2295271758 wait complete. 2023-10-18T09:20:35.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.308+0000 7f6842a9ab80 1 -- start start 2023-10-18T09:20:35.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.309+0000 7f6842a9ab80 1 Processor -- start 2023-10-18T09:20:35.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f59650 0x55b576f679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576f68f00 con 0x55b576f5a350 2023-10-18T09:20:35.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576f69070 con 0x55b576ecd7a0 2023-10-18T09:20:35.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b576f69230 con 0x55b576f59650 2023-10-18T09:20:35.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.310+0000 7f6842a9ab80 10 client.0 _ll_register_callbacks cb 0x55b576d92b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:35.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.311+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.311+0000 7f6827fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f59650 0x55b576f679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.311+0000 7f682cf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39932/0 (socket says 172.21.15.61:39932) 2023-10-18T09:20:35.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.311+0000 7f682cf9b700 1 -- 172.21.15.61:0/2494273210 learned_addr learned my addr 172.21.15.61:0/2494273210 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.312+0000 7f682cf9b700 1 -- 172.21.15.61:0/2494273210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f59650 msgr2=0x55b576f679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.312+0000 7f682cf9b700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f59650 0x55b576f679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.312+0000 7f682cf9b700 1 -- 172.21.15.61:0/2494273210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 msgr2=0x55b576f689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f682d79c700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f682cf9b700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f682cf9b700 1 -- 172.21.15.61:0/2494273210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b576e69470 con 0x55b576ecd7a0 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f682d79c700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:35.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f682cf9b700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f681809b650 tx=0x7f6818009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd3e265821998074 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.313+0000 7f6807fff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:35.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.314+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681809f810 con 0x55b576ecd7a0 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.315+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f68180b6020 con 0x55b576ecd7a0 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.315+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f681c011e50 con 0x55b576ecd7a0 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.315+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68180a39c0 con 0x55b576ecd7a0 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.316+0000 7f6842a9ab80 10 client.26093 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.316+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b576cfd2d0 con 0x55b576ecd7a0 2023-10-18T09:20:35.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.316+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68180a7020 con 0x55b576ecd7a0 2023-10-18T09:20:35.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f68180b9320 con 0x55b576ecd7a0 2023-10-18T09:20:35.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6842a9ab80 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:35.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6842a9ab80 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:35.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6842a9ab80 20 client.26093 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:35.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f5acb0 0x55b576fc6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.317+0000 7f6842a9ab80 4 client.26093 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-18T09:20:35.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.318+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b576ea4ed0 con 0x55b576f5acb0 2023-10-18T09:20:35.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.318+0000 7f6827fff700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f5acb0 0x55b576fc6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.321+0000 7f6827fff700 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f5acb0 0x55b576fc6d30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26fa381f4f0cc13d server_cookie=d8cd2cf829559835 in_seq=0 out_seq=0 2023-10-18T09:20:35.323 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.321+0000 7f6807fff700 10 client.26093 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.322+0000 7f6807fff700 1 -- 172.21.15.61:0/2494273210 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b576ea4ed0 con 0x55b576f5acb0 2023-10-18T09:20:35.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.322+0000 7f6807fff700 10 client.26093 handle_command_reply: tid=0 2023-10-18T09:20:35.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.322+0000 7f6842a9ab80 1 client.26093 shutdown 2023-10-18T09:20:35.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.322+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 msgr2=0x55b576f67470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.322+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f681809b650 tx=0x7f6818009310 comp rx=0 tx=0).stop 2023-10-18T09:20:35.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 shutdown_connections 2023-10-18T09:20:35.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55b576f5acb0 0x55b576fc6d30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55b576f5a350 0x55b576f689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b576ecd7a0 0x55b576f67470 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 --2- 172.21.15.61:0/2494273210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b576f59650 0x55b576f679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 >> 172.21.15.61:0/2494273210 conn(0x55b576f53f00 msgr2=0x55b576f58100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6807fff700 0 client.26093 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 shutdown_connections 2023-10-18T09:20:35.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.323+0000 7f6842a9ab80 1 -- 172.21.15.61:0/2494273210 wait complete. 2023-10-18T09:20:35.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.324+0000 7f6842a9ab80 20 client.26093 trim_cache size 0 max 16384 2023-10-18T09:20:35.328 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:20:35.328 INFO:teuthology.orchestra.run.smithi061.stdout: "return_code": 0, 2023-10-18T09:20:35.328 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:20:35.328 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": "asynchronous" 2023-10-18T09:20:35.329 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:20:35.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 msgr2=0x7f68100ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 0x7f68100ac2d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6820009fd0 tx=0x7f682000b040 comp rx=0 tx=0).stop 2023-10-18T09:20:35.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 msgr2=0x7f68281989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6818000c00 tx=0x7f681800f460 comp rx=0 tx=0).stop 2023-10-18T09:20:35.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 shutdown_connections 2023-10-18T09:20:35.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f68100a9e10 0x7f68100ac2d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6828100320 0x7f682819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68280ff000 0x7f68281989a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 --2- 172.21.15.61:0/1146101702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f68280ff960 0x7f6828198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.325+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 >> 172.21.15.61:0/1146101702 conn(0x7f68280fac00 msgr2=0x7f68280fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.326+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 shutdown_connections 2023-10-18T09:20:35.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.326+0000 7f682f1ff700 1 -- 172.21.15.61:0/1146101702 wait complete. 2023-10-18T09:20:35.365 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:20:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:35 smithi170 ceph-mon[135150]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-10-18T09:20:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:35 smithi110 ceph-mon[134818]: pgmap v419: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 13 MiB/s wr, 654 op/s 2023-10-18T09:20:35.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 -- 172.21.15.61:0/3998108653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 msgr2=0x7ffad410eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/3998108653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad410eb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009990 tx=0x7ffac0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 -- 172.21.15.61:0/3998108653 shutdown_connections 2023-10-18T09:20:35.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/3998108653 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 0x7ffad4111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/3998108653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.752+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/3998108653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad410eb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.756 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.753+0000 7ffadc1ff700 1 -- 172.21.15.61:0/3998108653 >> 172.21.15.61:0/3998108653 conn(0x7ffad406d7d0 msgr2=0x7ffad406fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.755+0000 7ffadc1ff700 1 -- 172.21.15.61:0/3998108653 shutdown_connections 2023-10-18T09:20:35.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.755+0000 7ffadc1ff700 1 -- 172.21.15.61:0/3998108653 wait complete. 2023-10-18T09:20:35.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.756+0000 7ffadc1ff700 1 Processor -- start 2023-10-18T09:20:35.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.757+0000 7ffadc1ff700 1 -- start start 2023-10-18T09:20:35.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad41185d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 0x7ffad41b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad411f680 con 0x7ffad410f070 2023-10-18T09:20:35.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39950/0 (socket says 172.21.15.61:39950) 2023-10-18T09:20:35.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad411f7f0 con 0x7ffad41077f0 2023-10-18T09:20:35.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1784838395 learned_addr learned my addr 172.21.15.61:0/1784838395 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad41b5cf0 con 0x7ffad4108150 2023-10-18T09:20:35.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.758+0000 7ffad979a700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad41185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffada79c700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 0x7ffad41b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1784838395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 msgr2=0x7ffad41185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad41185d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 msgr2=0x7ffad41b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 0x7ffad41b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1784838395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00095f0 con 0x7ffad41077f0 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad979a700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad41185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.759+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffac800ba50 tx=0x7ffac800bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.760+0000 7ffad2ffd700 1 -- 172.21.15.61:0/1784838395 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800c950 con 0x7ffad41077f0 2023-10-18T09:20:35.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.760+0000 7ffad2ffd700 1 -- 172.21.15.61:0/1784838395 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8014440 con 0x7ffad41077f0 2023-10-18T09:20:35.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.760+0000 7ffad2ffd700 1 -- 172.21.15.61:0/1784838395 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80075d0 con 0x7ffad41077f0 2023-10-18T09:20:35.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.760+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad41b5ef0 con 0x7ffad41077f0 2023-10-18T09:20:35.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.761+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad41b62c0 con 0x7ffad41077f0 2023-10-18T09:20:35.774 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.771+0000 7ffad2ffd700 1 -- 172.21.15.61:0/1784838395 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ffac80145b0 con 0x7ffad41077f0 2023-10-18T09:20:35.778 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.775+0000 7ffaefab2b80 1 Processor -- start 2023-10-18T09:20:35.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.777+0000 7ffaefab2b80 1 -- start start 2023-10-18T09:20:35.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8bda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f8306e0 con 0x55ab6f8bec30 2023-10-18T09:20:35.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f816c60 con 0x55ab6f8be2d0 2023-10-18T09:20:35.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f814580 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffad2ffd700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 0x7ffac40a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39964/0 (socket says 172.21.15.61:39964) 2023-10-18T09:20:35.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.778+0000 7ffad9f9b700 1 -- 172.21.15.61:0/232188167 learned_addr learned my addr 172.21.15.61:0/232188167 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad2ffd700 1 -- 172.21.15.61:0/1784838395 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ffac8006e40 con 0x7ffad41077f0 2023-10-18T09:20:35.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 -- 172.21.15.61:0/232188167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 msgr2=0x55ab6f8bda50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:20:35.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8bda50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 -- 172.21.15.61:0/232188167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 msgr2=0x55ab6f8bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 -- 172.21.15.61:0/232188167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab6f7cd3f0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.784 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.779+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffac8095a60 tx=0x7ffac806c5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2175b0006515859e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.780+0000 7ffabffff700 1 -- 172.21.15.61:0/232188167 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8097db0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.780+0000 7ffabffff700 1 -- 172.21.15.61:0/232188167 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8097bc0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.780+0000 7ffabffff700 1 -- 172.21.15.61:0/232188167 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac809d6b0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.780+0000 7ffada79c700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 0x7ffac40a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.785 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 -- 172.21.15.61:0/232188167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 msgr2=0x55ab6f8be6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.786 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffac8095a60 tx=0x7ffac806c5a0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffada79c700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 0x7ffac40a3a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b800 tx=0x7ffacc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 -- 172.21.15.61:0/232188167 shutdown_connections 2023-10-18T09:20:35.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8bda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8be6f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.787 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/232188167 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.781+0000 7ffaefab2b80 1 -- 172.21.15.61:0/232188167 >> 172.21.15.61:0/232188167 conn(0x55ab6f831720 msgr2=0x55ab6f8b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.782+0000 7ffaefab2b80 1 -- 172.21.15.61:0/232188167 shutdown_connections 2023-10-18T09:20:35.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.782+0000 7ffaefab2b80 1 -- 172.21.15.61:0/232188167 wait complete. 2023-10-18T09:20:35.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.782+0000 7ffaefab2b80 1 -- start start 2023-10-18T09:20:35.788 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.783+0000 7ffaefab2b80 1 Processor -- start 2023-10-18T09:20:35.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f80fe70 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f7ac710 con 0x55ab6f8be2d0 2023-10-18T09:20:35.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f7ac3b0 con 0x55ab6f8bec30 2023-10-18T09:20:35.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.785+0000 7ffaefab2b80 10 client.0 _ll_register_callbacks cb 0x55ab6f6f6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:35.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:39652/0 (socket says 172.21.15.61:39652) 2023-10-18T09:20:35.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1960738469 learned_addr learned my addr 172.21.15.61:0/1960738469 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffada79c700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad979a700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1960738469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 msgr2=0x55ab6f8cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 msgr2=0x55ab6f8c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad9f9b700 1 -- 172.21.15.61:0/1960738469 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad41b62c0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffad979a700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.786+0000 7ffada79c700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffac8014ce0 tx=0x7ffac8098070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2be3f8785a7e25 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffabd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:20:35.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac809d6b0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ffac8097d60 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab6f814ef0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac8098a50 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.787+0000 7ffaefab2b80 10 client.15969 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:35.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.788+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ab6f63ade0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.788+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac80b6030 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.790+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ffac806c150 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.790+0000 7ffaefab2b80 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:35.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.791+0000 7ffaefab2b80 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:35.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.791+0000 7ffaefab2b80 20 client.15969 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:35.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.791+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f8d25c0 0x55ab6f8d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.791+0000 7ffaefab2b80 4 client.15969 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:20:35.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.791+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab6f88c8b0 con 0x55ab6f8d25c0 2023-10-18T09:20:35.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.792+0000 7ffad979a700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f8d25c0 0x55ab6f8d49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.792+0000 7ffad979a700 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f8d25c0 0x55ab6f8d49e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26223603878811ed server_cookie=bd2cf0b4ae2d028f in_seq=0 out_seq=0 2023-10-18T09:20:35.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.792+0000 7ffabd7fa700 10 client.15969 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.793+0000 7ffabd7fa700 1 -- 172.21.15.61:0/1960738469 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ab6f88c8b0 con 0x55ab6f8d25c0 2023-10-18T09:20:35.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.793+0000 7ffabd7fa700 10 client.15969 handle_command_reply: tid=0 2023-10-18T09:20:35.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.793+0000 7ffaefab2b80 1 client.15969 shutdown 2023-10-18T09:20:35.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.794+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 msgr2=0x55ab6f8c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.794+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffac8014ce0 tx=0x7ffac8098070 comp rx=0 tx=0).stop 2023-10-18T09:20:35.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 shutdown_connections 2023-10-18T09:20:35.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f8d25c0 0x55ab6f8d49e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8c6a70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffabd7fa700 0 client.15969 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/1960738469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 >> 172.21.15.61:0/1960738469 conn(0x55ab6f831720 msgr2=0x55ab6f8bc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 shutdown_connections 2023-10-18T09:20:35.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.795+0000 7ffaefab2b80 1 -- 172.21.15.61:0/1960738469 wait complete. 2023-10-18T09:20:35.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.796+0000 7ffaefab2b80 20 client.15969 trim_cache size 0 max 16384 2023-10-18T09:20:35.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 Processor -- start 2023-10-18T09:20:35.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 -- start start 2023-10-18T09:20:35.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f690920 con 0x55ab6f8bec30 2023-10-18T09:20:35.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f8c7ae0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f88c8e0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.827+0000 7ffad979a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39970/0 (socket says 172.21.15.61:39970) 2023-10-18T09:20:35.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 -- 172.21.15.61:0/2737743878 learned_addr learned my addr 172.21.15.61:0/2737743878 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffada79c700 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 -- 172.21.15.61:0/2737743878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 msgr2=0x55ab6f8b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 -- 172.21.15.61:0/2737743878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 msgr2=0x55ab6f8b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 -- 172.21.15.61:0/2737743878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab6f8c1e50 con 0x55ab6f8be2d0 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.828+0000 7ffad979a700 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72e8f15c6b16e0b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.829+0000 7ffad0ff9700 1 -- 172.21.15.61:0/2737743878 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac001d040 con 0x55ab6f8be2d0 2023-10-18T09:20:35.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.829+0000 7ffad0ff9700 1 -- 172.21.15.61:0/2737743878 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac00263a0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.829+0000 7ffad0ff9700 1 -- 172.21.15.61:0/2737743878 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00052d0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 -- 172.21.15.61:0/2737743878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 msgr2=0x55ab6f8cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffac0000c00 tx=0x7ffac0009290 comp rx=0 tx=0).stop 2023-10-18T09:20:35.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 -- 172.21.15.61:0/2737743878 shutdown_connections 2023-10-18T09:20:35.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8bec30 0x55ab6f8b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/2737743878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdf80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 -- 172.21.15.61:0/2737743878 >> 172.21.15.61:0/2737743878 conn(0x55ab6f831720 msgr2=0x55ab6f8bd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 -- 172.21.15.61:0/2737743878 shutdown_connections 2023-10-18T09:20:35.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.832+0000 7ffaefab2b80 1 -- 172.21.15.61:0/2737743878 wait complete. 2023-10-18T09:20:35.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.833+0000 7ffaefab2b80 1 -- start start 2023-10-18T09:20:35.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.834+0000 7ffaefab2b80 1 Processor -- start 2023-10-18T09:20:35.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f8cc9c0 con 0x55ab6f8be2d0 2023-10-18T09:20:35.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f8ccb00 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab6f8cccc0 con 0x55ab6f8bec30 2023-10-18T09:20:35.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffaefab2b80 10 client.0 _ll_register_callbacks cb 0x55ab6f6f6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:20:35.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffad9f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39974/0 (socket says 172.21.15.61:39974) 2023-10-18T09:20:35.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.835+0000 7ffad9f9b700 1 -- 172.21.15.61:0/3199831839 learned_addr learned my addr 172.21.15.61:0/3199831839 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:20:35.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 -- 172.21.15.61:0/3199831839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 msgr2=0x55ab6f8cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 -- 172.21.15.61:0/3199831839 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8be2d0 msgr2=0x55ab6f8cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:20:35.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 -- 172.21.15.61:0/3199831839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4111d20 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffad9f9b700 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c450 tx=0x7ffac80c9d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bbee1c0b2b57017 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:20:35.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffabd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:20:35.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.836+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8066020 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ffac809d6b0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac8097bc0 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac80ad120 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 10 client.26108 fetch_fsmap learned FSMap version 32 2023-10-18T09:20:35.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ab6f623c60 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac807c020 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7ffac8095350 con 0x55ab6f8bd5d0 2023-10-18T09:20:35.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:20:35.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:20:35.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 20 client.26108 populate_metadata read hostname 'smithi061' 2023-10-18T09:20:35.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f9273b0 0x55ab6f9297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:20:35.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 4 client.26108 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:20:35.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.837+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab6f808e50 con 0x55ab6f9273b0 2023-10-18T09:20:35.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.838+0000 7ffad979a700 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f9273b0 0x55ab6f9297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:20:35.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.839+0000 7ffad979a700 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f9273b0 0x55ab6f9297d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e592ae9a6cfe2f server_cookie=d004830b706029b0 in_seq=0 out_seq=0 2023-10-18T09:20:35.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.839+0000 7ffabd7fa700 10 client.26108 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.839+0000 7ffabd7fa700 1 -- 172.21.15.61:0/3199831839 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55ab6f808e50 con 0x55ab6f9273b0 2023-10-18T09:20:35.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.839+0000 7ffabd7fa700 10 client.26108 handle_command_reply: tid=0 2023-10-18T09:20:35.846 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.839+0000 7ffaefab2b80 1 client.26108 shutdown 2023-10-18T09:20:35.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 msgr2=0x55ab6f8cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffac800c450 tx=0x7ffac80c9d90 comp rx=0 tx=0).stop 2023-10-18T09:20:35.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 shutdown_connections 2023-10-18T09:20:35.847 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55ab6f9273b0 0x55ab6f9297d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab6f8bec30 0x55ab6f8cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ab6f8bd5d0 0x55ab6f8cdaa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 --2- 172.21.15.61:0/3199831839 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55ab6f8be2d0 0x55ab6f8cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffabd7fa700 0 client.26108 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 >> 172.21.15.61:0/3199831839 conn(0x55ab6f831720 msgr2=0x55ab6f8b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.848 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.840+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 shutdown_connections 2023-10-18T09:20:35.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.841+0000 7ffaefab2b80 1 -- 172.21.15.61:0/3199831839 wait complete. 2023-10-18T09:20:35.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.841+0000 7ffaefab2b80 20 client.26108 trim_cache size 0 max 16384 2023-10-18T09:20:35.849 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:20:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:20:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:20:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:20:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:20:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:20:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:20:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:20:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:20:35.851 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:20:35.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 msgr2=0x7ffac40a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 0x7ffac40a3a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00b800 tx=0x7ffacc008040 comp rx=0 tx=0).stop 2023-10-18T09:20:35.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 msgr2=0x7ffad4118090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:20:35.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffac800ba50 tx=0x7ffac800bd60 comp rx=0 tx=0).stop 2023-10-18T09:20:35.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 shutdown_connections 2023-10-18T09:20:35.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ffac40a15d0 0x7ffac40a3a90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ffad410f070 0x7ffad41b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffad41077f0 0x7ffad4118090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 --2- 172.21.15.61:0/1784838395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffad4108150 0x7ffad41185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:20:35.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 >> 172.21.15.61:0/1784838395 conn(0x7ffad406d7d0 msgr2=0x7ffad406f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:20:35.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.843+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 shutdown_connections 2023-10-18T09:20:35.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:20:35.844+0000 7ffadc1ff700 1 -- 172.21.15.61:0/1784838395 wait complete. 2023-10-18T09:20:35.875 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:20:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:36.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:36 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:36.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:36 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:36.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:36.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:36 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: scrub queued for path: / 2023-10-18T09:20:37.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: scrub summary: active paths [/] 2023-10-18T09:20:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:37 smithi170 ceph-mon[135150]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-10-18T09:20:37.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: scrub queued for path: / 2023-10-18T09:20:37.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:37.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: scrub summary: active paths [/] 2023-10-18T09:20:37.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:37 smithi110 ceph-mon[134818]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-10-18T09:20:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: scrub queued for path: / 2023-10-18T09:20:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: scrub summary: active paths [/] 2023-10-18T09:20:37.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: scrub summary: idle+waiting paths [/] 2023-10-18T09:20:37.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:37 smithi061 ceph-mon[98573]: pgmap v420: 129 pgs: 129 active+clean; 1.5 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 623 op/s 2023-10-18T09:20:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 85.0 s, 1900.1 tps, lat 52.566 ms stddev 61.876 2023-10-18T09:20:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:39 smithi170 ceph-mon[135150]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.7 MiB/s wr, 565 op/s 2023-10-18T09:20:39.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:39 smithi110 ceph-mon[134818]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.7 MiB/s wr, 565 op/s 2023-10-18T09:20:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:39 smithi061 ceph-mon[98573]: pgmap v421: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 9.7 MiB/s wr, 565 op/s 2023-10-18T09:20:41.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:41 smithi170 ceph-mon[135150]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 MiB/s wr, 724 op/s 2023-10-18T09:20:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:41 smithi110 ceph-mon[134818]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 MiB/s wr, 724 op/s 2023-10-18T09:20:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:41 smithi061 ceph-mon[98573]: pgmap v422: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 12 MiB/s wr, 724 op/s 2023-10-18T09:20:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:43 smithi170 ceph-mon[135150]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 615 op/s 2023-10-18T09:20:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:43 smithi110 ceph-mon[134818]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 615 op/s 2023-10-18T09:20:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:43 smithi061 ceph-mon[98573]: pgmap v423: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.1 MiB/s wr, 615 op/s 2023-10-18T09:20:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 90.0 s, 1896.8 tps, lat 52.389 ms stddev 64.312 2023-10-18T09:20:45.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:45 smithi170 ceph-mon[135150]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 705 op/s 2023-10-18T09:20:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:45 smithi110 ceph-mon[134818]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 705 op/s 2023-10-18T09:20:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:45 smithi061 ceph-mon[98573]: pgmap v424: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 705 op/s 2023-10-18T09:20:47.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:47 smithi170 ceph-mon[135150]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-18T09:20:47.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:47 smithi110 ceph-mon[134818]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-18T09:20:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:47 smithi061 ceph-mon[98573]: pgmap v425: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-18T09:20:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 95.0 s, 1104.0 tps, lat 90.018 ms stddev 119.252 2023-10-18T09:20:49.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:49 smithi170 ceph-mon[135150]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-10-18T09:20:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:49 smithi110 ceph-mon[134818]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-10-18T09:20:49.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:49 smithi061 ceph-mon[98573]: pgmap v426: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 701 op/s 2023-10-18T09:20:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:51 smithi170 ceph-mon[135150]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 912 op/s 2023-10-18T09:20:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:51 smithi110 ceph-mon[134818]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 912 op/s 2023-10-18T09:20:51.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:51 smithi061 ceph-mon[98573]: pgmap v427: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 912 op/s 2023-10-18T09:20:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:53 smithi170 ceph-mon[135150]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 807 op/s 2023-10-18T09:20:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:53 smithi110 ceph-mon[134818]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 807 op/s 2023-10-18T09:20:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:53 smithi061 ceph-mon[98573]: pgmap v428: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 MiB/s wr, 807 op/s 2023-10-18T09:20:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 100.0 s, 1497.8 tps, lat 67.524 ms stddev 80.447 2023-10-18T09:20:54.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:54 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:20:54.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:54 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:20:54.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:54 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:20:55.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:55.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:55 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:55.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:55 smithi170 ceph-mon[135150]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 864 op/s 2023-10-18T09:20:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:55 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:55.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:55 smithi110 ceph-mon[134818]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 864 op/s 2023-10-18T09:20:56.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:56.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:55 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:56.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:55 smithi061 ceph-mon[98573]: pgmap v429: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 864 op/s 2023-10-18T09:20:56.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:20:56.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:20:56.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:56.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:20:56.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:20:56.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:57.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:20:57.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:20:57.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:20:57.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:57 smithi170 ceph-mon[135150]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 846 op/s 2023-10-18T09:20:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:57 smithi110 ceph-mon[134818]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 846 op/s 2023-10-18T09:20:58.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:57 smithi061 ceph-mon[98573]: pgmap v430: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 16 MiB/s wr, 846 op/s 2023-10-18T09:20:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 105.0 s, 1475.6 tps, lat 66.003 ms stddev 77.880 2023-10-18T09:20:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:20:59 smithi170 ceph-mon[135150]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 789 op/s 2023-10-18T09:20:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:20:59 smithi110 ceph-mon[134818]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 789 op/s 2023-10-18T09:20:59.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:20:59 smithi061 ceph-mon[98573]: pgmap v431: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 16 MiB/s wr, 789 op/s 2023-10-18T09:21:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:01 smithi170 ceph-mon[135150]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 864 op/s 2023-10-18T09:21:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:01 smithi110 ceph-mon[134818]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 864 op/s 2023-10-18T09:21:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:01 smithi061 ceph-mon[98573]: pgmap v432: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 18 MiB/s wr, 864 op/s 2023-10-18T09:21:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:03 smithi170 ceph-mon[135150]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 664 op/s 2023-10-18T09:21:03.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:03 smithi110 ceph-mon[134818]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 664 op/s 2023-10-18T09:21:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:03 smithi061 ceph-mon[98573]: pgmap v433: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 14 MiB/s wr, 664 op/s 2023-10-18T09:21:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 110.0 s, 1293.7 tps, lat 78.802 ms stddev 91.267 2023-10-18T09:21:05.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:05 smithi170 ceph-mon[135150]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 708 op/s 2023-10-18T09:21:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:05 smithi110 ceph-mon[134818]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 708 op/s 2023-10-18T09:21:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:05 smithi061 ceph-mon[98573]: pgmap v434: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 MiB/s wr, 708 op/s 2023-10-18T09:21:05.876 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:21:06.308 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 -- 172.21.15.61:0/3873078990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 msgr2=0x7fb6e0105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 --2- 172.21.15.61:0/3873078990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e0105dd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009990 tx=0x7fb6c8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 -- 172.21.15.61:0/3873078990 shutdown_connections 2023-10-18T09:21:06.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 --2- 172.21.15.61:0/3873078990 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e0106310 0x7fb6e0108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 --2- 172.21.15.61:0/3873078990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e01010b0 0x7fb6e01034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.309 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 --2- 172.21.15.61:0/3873078990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e0105dd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.306+0000 7fb6dffff700 1 -- 172.21.15.61:0/3873078990 >> 172.21.15.61:0/3873078990 conn(0x7fb6e00faa80 msgr2=0x7fb6e00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.310 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.307+0000 7fb6dffff700 1 -- 172.21.15.61:0/3873078990 shutdown_connections 2023-10-18T09:21:06.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.307+0000 7fb6dffff700 1 -- 172.21.15.61:0/3873078990 wait complete. 2023-10-18T09:21:06.311 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.308+0000 7fb6dffff700 1 Processor -- start 2023-10-18T09:21:06.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.309+0000 7fb6dffff700 1 -- start start 2023-10-18T09:21:06.312 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.310+0000 7fb6dffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 0x7fb6e0194180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.310+0000 7fb6dffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.310+0000 7fb6dffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 0x7fb6e01a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.310+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.310+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46884/0 (socket says 172.21.15.61:46884) 2023-10-18T09:21:06.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 0x7fb6e01a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 0x7fb6e01a4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:32996/0 (socket says 172.21.15.61:32996) 2023-10-18T09:21:06.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6d67fc700 1 -- 172.21.15.61:0/77478470 learned_addr learned my addr 172.21.15.61:0/77478470 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:06.315 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e0194f60 con 0x7fb6e01010b0 2023-10-18T09:21:06.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01a48c0 con 0x7fb6e01039e0 2023-10-18T09:21:06.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e01a4a00 con 0x7fb6e0106310 2023-10-18T09:21:06.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6deffd700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 0x7fb6e0194180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6d67fc700 1 -- 172.21.15.61:0/77478470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 msgr2=0x7fb6e01a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.316 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 0x7fb6e01a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6d67fc700 1 -- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 msgr2=0x7fb6e0194180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.311+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 0x7fb6e0194180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.312+0000 7fb6d67fc700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c80095f0 con 0x7fb6e01039e0 2023-10-18T09:21:06.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.312+0000 7fb6deffd700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 0x7fb6e0194180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:21:06.317 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.312+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005910 tx=0x7fb6c8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.312+0000 7fb6dcff9700 1 -- 172.21.15.61:0/77478470 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c801d070 con 0x7fb6e01039e0 2023-10-18T09:21:06.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.313+0000 7fb6dcff9700 1 -- 172.21.15.61:0/77478470 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c8021410 con 0x7fb6e01039e0 2023-10-18T09:21:06.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.313+0000 7fb6dcff9700 1 -- 172.21.15.61:0/77478470 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c801d070 con 0x7fb6e01039e0 2023-10-18T09:21:06.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.313+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e01a4c50 con 0x7fb6e01039e0 2023-10-18T09:21:06.318 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.313+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e01a4fc0 con 0x7fb6e01039e0 2023-10-18T09:21:06.324 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.321+0000 7fb6dcff9700 1 -- 172.21.15.61:0/77478470 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb6c802a3f0 con 0x7fb6e01039e0 2023-10-18T09:21:06.326 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.324+0000 7fb6f916cb80 1 Processor -- start 2023-10-18T09:21:06.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.325+0000 7fb6dcff9700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 0x7fb6cc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.325+0000 7fb6dcff9700 1 -- 172.21.15.61:0/77478470 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6c80cdc30 con 0x7fb6e01039e0 2023-10-18T09:21:06.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.326+0000 7fb6f916cb80 1 -- start start 2023-10-18T09:21:06.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.326+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 0x7fb6cc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 0x557d18fd58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 0x557d18fd6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18f486e0 con 0x557d18fd67f0 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18f2ec60 con 0x557d18fd5e30 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18f2c580 con 0x557d18fd54d0 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46898/0 (socket says 172.21.15.61:46898) 2023-10-18T09:21:06.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 -- 172.21.15.61:0/4152272720 learned_addr learned my addr 172.21.15.61:0/4152272720 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:06.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6deffd700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 0x557d18fd58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 0x557d18fd6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 -- 172.21.15.61:0/4152272720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 msgr2=0x557d18fd58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.332 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 0x557d18fd58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.333 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.327+0000 7fb6d67fc700 1 -- 172.21.15.61:0/4152272720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 msgr2=0x557d18fd6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.328+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 0x557d18fd6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.328+0000 7fb6d67fc700 1 -- 172.21.15.61:0/4152272720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d18ee53f0 con 0x557d18fd5e30 2023-10-18T09:21:06.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.328+0000 7fb6deffd700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 0x557d18fd58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:21:06.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.328+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 0x557d18fd6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:21:06.334 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.328+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8098040 tx=0x7fb6c8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aae3a21de1e9e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.329+0000 7fb6d57fa700 1 -- 172.21.15.61:0/4152272720 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c809c240 con 0x557d18fd5e30 2023-10-18T09:21:06.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.329+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 0x7fb6cc0a3a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009e80 tx=0x7fb6d80098c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.329+0000 7fb6d57fa700 1 -- 172.21.15.61:0/4152272720 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c809bb40 con 0x557d18fd5e30 2023-10-18T09:21:06.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.329+0000 7fb6d57fa700 1 -- 172.21.15.61:0/4152272720 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c809c000 con 0x557d18fd5e30 2023-10-18T09:21:06.335 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.330+0000 7fb6f916cb80 1 -- 172.21.15.61:0/4152272720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 msgr2=0x557d18fd62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.330+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8098040 tx=0x7fb6c8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.331+0000 7fb6f916cb80 1 -- 172.21.15.61:0/4152272720 shutdown_connections 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.331+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd67f0 0x557d18fd6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.331+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd54d0 0x557d18fd58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.331+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/4152272720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fd62b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.336 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.331+0000 7fb6f916cb80 1 -- 172.21.15.61:0/4152272720 >> 172.21.15.61:0/4152272720 conn(0x557d18f49720 msgr2=0x557d18fcfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.332+0000 7fb6f916cb80 1 -- 172.21.15.61:0/4152272720 shutdown_connections 2023-10-18T09:21:06.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.332+0000 7fb6f916cb80 1 -- 172.21.15.61:0/4152272720 wait complete. 2023-10-18T09:21:06.337 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.333+0000 7fb6f916cb80 1 -- start start 2023-10-18T09:21:06.338 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.333+0000 7fb6f916cb80 1 Processor -- start 2023-10-18T09:21:06.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd5e30 0x557d18fdeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18f27e70 con 0x557d18fd5e30 2023-10-18T09:21:06.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18ec4710 con 0x557d18fd54d0 2023-10-18T09:21:06.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18ec43b0 con 0x557d18fd67f0 2023-10-18T09:21:06.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6f916cb80 10 client.0 _ll_register_callbacks cb 0x557d18e0ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:21:06.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6deffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46900/0 (socket says 172.21.15.61:46900) 2023-10-18T09:21:06.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.337+0000 7fb6deffd700 1 -- 172.21.15.61:0/3045157452 learned_addr learned my addr 172.21.15.61:0/3045157452 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:06.342 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.338+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.338+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd5e30 0x557d18fdeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.338+0000 7fb6deffd700 1 -- 172.21.15.61:0/3045157452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 msgr2=0x557d18fe5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.338+0000 7fb6deffd700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.338+0000 7fb6deffd700 1 -- 172.21.15.61:0/3045157452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd5e30 msgr2=0x557d18fdeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6deffd700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd5e30 0x557d18fdeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6deffd700 1 -- 172.21.15.61:0/3045157452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e01a4fc0 con 0x557d18fd54d0 2023-10-18T09:21:06.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:21:06.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6deffd700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0007f50 tx=0x7fb6d0007f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d9da977a9b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6c67fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:21:06.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.339+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000e070 con 0x557d18fd54d0 2023-10-18T09:21:06.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.340+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6d0012c60 con 0x557d18fd54d0 2023-10-18T09:21:06.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.340+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d18f2cef0 con 0x557d18fd54d0 2023-10-18T09:21:06.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.340+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6d0012e20 con 0x557d18fd54d0 2023-10-18T09:21:06.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.340+0000 7fb6f916cb80 10 client.26120 fetch_fsmap learned FSMap version 32 2023-10-18T09:21:06.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.340+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d18d52de0 con 0x557d18fd54d0 2023-10-18T09:21:06.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d18f2cef0 con 0x557d18fd54d0 2023-10-18T09:21:06.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb6d0014040 con 0x557d18fd54d0 2023-10-18T09:21:06.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 10 client.26120 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:21:06.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 10 client.26120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:21:06.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 20 client.26120 populate_metadata read hostname 'smithi061' 2023-10-18T09:21:06.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d18fea7f0 0x557d18fecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 4 client.26120 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:21:06.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.341+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d18fa48b0 con 0x557d18fea7f0 2023-10-18T09:21:06.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.342+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d18fea7f0 0x557d18fecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.342+0000 7fb6c67fc700 10 client.26120 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:06.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.342+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d18fea7f0 0x557d18fecc10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60c1912b057750 server_cookie=59db90a1547bd5f4 in_seq=0 out_seq=0 2023-10-18T09:21:06.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6c67fc700 1 -- 172.21.15.61:0/3045157452 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557d18fa48b0 con 0x557d18fea7f0 2023-10-18T09:21:06.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6c67fc700 10 client.26120 handle_command_reply: tid=0 2023-10-18T09:21:06.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 client.26120 shutdown 2023-10-18T09:21:06.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 msgr2=0x557d18fde970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0007f50 tx=0x7fb6d0007f80 comp rx=0 tx=0).stop 2023-10-18T09:21:06.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 shutdown_connections 2023-10-18T09:21:06.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d18fea7f0 0x557d18fecc10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd54d0 0x557d18fde970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/3045157452 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd5e30 0x557d18fdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.352 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 >> 172.21.15.61:0/3045157452 conn(0x557d18f49720 msgr2=0x557d18fd42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.343+0000 7fb6c67fc700 0 client.26120 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:06.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.344+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 shutdown_connections 2023-10-18T09:21:06.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.344+0000 7fb6f916cb80 1 -- 172.21.15.61:0/3045157452 wait complete. 2023-10-18T09:21:06.353 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.344+0000 7fb6f916cb80 20 client.26120 trim_cache size 0 max 16384 2023-10-18T09:21:06.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.379+0000 7fb6f916cb80 1 Processor -- start 2023-10-18T09:21:06.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.379+0000 7fb6f916cb80 1 -- start start 2023-10-18T09:21:06.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fd42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 0x557d18fe5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 0x557d18fd4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18da8920 con 0x557d18fd54d0 2023-10-18T09:21:06.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18fde500 con 0x557d18fd67f0 2023-10-18T09:21:06.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.380+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d18fe7070 con 0x557d18fd5e30 2023-10-18T09:21:06.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.381+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 0x557d18fd4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.381+0000 7fb6df7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 0x557d18fd4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46912/0 (socket says 172.21.15.61:46912) 2023-10-18T09:21:06.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.381+0000 7fb6df7fe700 1 -- 172.21.15.61:0/829466567 learned_addr learned my addr 172.21.15.61:0/829466567 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:06.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.381+0000 7fb6deffd700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fd42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 0x557d18fe5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 -- 172.21.15.61:0/829466567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 msgr2=0x557d18fe5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 0x557d18fe5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 -- 172.21.15.61:0/829466567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 msgr2=0x557d18fd4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 0x557d18fd4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 -- 172.21.15.61:0/829466567 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d18d3bc60 con 0x557d18fd54d0 2023-10-18T09:21:06.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 0x557d18fe5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:21:06.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.382+0000 7fb6deffd700 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fd42d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0016730 tx=0x7fb6d0006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb565137329fc5a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.383+0000 7fb6d5ffb700 1 -- 172.21.15.61:0/829466567 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d000e040 con 0x557d18fd54d0 2023-10-18T09:21:06.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.383+0000 7fb6d5ffb700 1 -- 172.21.15.61:0/829466567 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d000f040 con 0x557d18fd54d0 2023-10-18T09:21:06.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.383+0000 7fb6d5ffb700 1 -- 172.21.15.61:0/829466567 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00054d0 con 0x557d18fd54d0 2023-10-18T09:21:06.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 -- 172.21.15.61:0/829466567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 msgr2=0x557d18fd42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fd42d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0016730 tx=0x7fb6d0006150 comp rx=0 tx=0).stop 2023-10-18T09:21:06.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 -- 172.21.15.61:0/829466567 shutdown_connections 2023-10-18T09:21:06.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd67f0 0x557d18fd4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fd42d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/829466567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd5e30 0x557d18fe5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.384+0000 7fb6f916cb80 1 -- 172.21.15.61:0/829466567 >> 172.21.15.61:0/829466567 conn(0x557d18f49720 msgr2=0x557d18fdcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.385+0000 7fb6f916cb80 1 -- 172.21.15.61:0/829466567 shutdown_connections 2023-10-18T09:21:06.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.385+0000 7fb6f916cb80 1 -- 172.21.15.61:0/829466567 wait complete. 2023-10-18T09:21:06.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.386+0000 7fb6f916cb80 1 -- start start 2023-10-18T09:21:06.390 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.386+0000 7fb6f916cb80 1 Processor -- start 2023-10-18T09:21:06.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fe4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1903e610 con 0x557d18fd54d0 2023-10-18T09:21:06.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1903e750 con 0x557d18fd5e30 2023-10-18T09:21:06.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.387+0000 7fb6f916cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d1903e890 con 0x557d18fd67f0 2023-10-18T09:21:06.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6f916cb80 10 client.0 _ll_register_callbacks cb 0x557d18e0ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:21:06.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46924/0 (socket says 172.21.15.61:46924) 2023-10-18T09:21:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6d67fc700 1 -- 172.21.15.61:0/2735815319 learned_addr learned my addr 172.21.15.61:0/2735815319 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6deffd700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fe4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.388+0000 7fb6df7fe700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.389+0000 7fb6d67fc700 1 -- 172.21.15.61:0/2735815319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 msgr2=0x557d18fe51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.389+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.389+0000 7fb6d67fc700 1 -- 172.21.15.61:0/2735815319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 msgr2=0x557d18fe4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.389+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fe4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.389+0000 7fb6d67fc700 1 -- 172.21.15.61:0/2735815319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d18d52de0 con 0x557d18fd5e30 2023-10-18T09:21:06.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.390+0000 7fb6d67fc700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c809bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6fc1019a11d20e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.390+0000 7fb6c67fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.391+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8097e80 con 0x557d18fd5e30 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.391+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb6c80a3e40 con 0x557d18fd5e30 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.392+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6c809bb40 con 0x557d18fd5e30 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.392+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6c809fc90 con 0x557d18fd5e30 2023-10-18T09:21:06.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.392+0000 7fb6f916cb80 10 client.26126 fetch_fsmap learned FSMap version 32 2023-10-18T09:21:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.392+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557d18d79250 con 0x557d18fd5e30 2023-10-18T09:21:06.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.392+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6c809bb40 con 0x557d18fd5e30 2023-10-18T09:21:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.393+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 32) v1 ==== 7922+0+0 (secure 0 0 0) 0x7fb6c809bda0 con 0x557d18fd5e30 2023-10-18T09:21:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.393+0000 7fb6f916cb80 10 client.26126 fetch_fsmap finished waiting for FSMap version 32 2023-10-18T09:21:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.393+0000 7fb6f916cb80 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:21:06.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.393+0000 7fb6f916cb80 20 client.26126 populate_metadata read hostname 'smithi061' 2023-10-18T09:21:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.393+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d1903ffc0 0x557d190423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.394+0000 7fb6f916cb80 4 client.26126 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:21:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.394+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557d18f20e50 con 0x557d1903ffc0 2023-10-18T09:21:06.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.395+0000 7fb6deffd700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d1903ffc0 0x557d190423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.397+0000 7fb6deffd700 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d1903ffc0 0x557d190423e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3aac28cd0f8302 server_cookie=291192515cf4ad7c in_seq=0 out_seq=0 2023-10-18T09:21:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.397+0000 7fb6c67fc700 10 client.26126 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.397+0000 7fb6c67fc700 1 -- 172.21.15.61:0/2735815319 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557d18f20e50 con 0x557d1903ffc0 2023-10-18T09:21:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.397+0000 7fb6c67fc700 10 client.26126 handle_command_reply: tid=0 2023-10-18T09:21:06.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.397+0000 7fb6f916cb80 1 client.26126 shutdown 2023-10-18T09:21:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.398+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 msgr2=0x557d18fe4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.398+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c809bcb0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 shutdown_connections 2023-10-18T09:21:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557d1903ffc0 0x557d190423e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557d18fd67f0 0x557d18fe51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557d18fd54d0 0x557d18fe4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 --2- 172.21.15.61:0/2735815319 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557d18fd5e30 0x557d18fe4c70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 >> 172.21.15.61:0/2735815319 conn(0x557d18f49720 msgr2=0x557d18fdc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.399+0000 7fb6c67fc700 0 client.26126 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:06.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.400+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 shutdown_connections 2023-10-18T09:21:06.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.402+0000 7fb6f916cb80 1 -- 172.21.15.61:0/2735815319 wait complete. 2023-10-18T09:21:06.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.402+0000 7fb6f916cb80 20 client.26126 trim_cache size 0 max 16384 2023-10-18T09:21:06.405 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:21:06.405 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:21:06.405 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:21:06.406 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:21:06.406 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:21:06.406 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:21:06.406 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:21:06.406 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:21:06.407 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:21:06.407 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 msgr2=0x7fb6cc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 0x7fb6cc0a3a50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009e80 tx=0x7fb6d80098c0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 msgr2=0x7fb6e01946c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8005910 tx=0x7fb6c8004730 comp rx=0 tx=0).stop 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 shutdown_connections 2023-10-18T09:21:06.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb6cc0a1590 0x7fb6cc0a3a50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6e0106310 0x7fb6e01a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb6e01010b0 0x7fb6e0194180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 --2- 172.21.15.61:0/77478470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6e01039e0 0x7fb6e01946c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:06.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.404+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 >> 172.21.15.61:0/77478470 conn(0x7fb6e00faa80 msgr2=0x7fb6e0100820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:06.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.405+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 shutdown_connections 2023-10-18T09:21:06.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:06.405+0000 7fb6dffff700 1 -- 172.21.15.61:0/77478470 wait complete. 2023-10-18T09:21:06.453 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:21:07.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:07 smithi110 ceph-mon[134818]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 705 op/s 2023-10-18T09:21:08.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:07 smithi061 ceph-mon[98573]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 705 op/s 2023-10-18T09:21:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:07 smithi170 ceph-mon[135150]: pgmap v435: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 MiB/s wr, 705 op/s 2023-10-18T09:21:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 115.0 s, 692.2 tps, lat 142.429 ms stddev 388.295 2023-10-18T09:21:09.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:09 smithi170 ceph-mon[135150]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-18T09:21:09.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:09 smithi110 ceph-mon[134818]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-18T09:21:09.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:09 smithi061 ceph-mon[98573]: pgmap v436: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 17 MiB/s wr, 890 op/s 2023-10-18T09:21:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:11 smithi170 ceph-mon[135150]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-18T09:21:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:11 smithi110 ceph-mon[134818]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-18T09:21:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:11 smithi061 ceph-mon[98573]: pgmap v437: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.74k op/s 2023-10-18T09:21:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:13 smithi170 ceph-mon[135150]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-18T09:21:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:13 smithi110 ceph-mon[134818]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-18T09:21:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:13 smithi061 ceph-mon[98573]: pgmap v438: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 1.80k op/s 2023-10-18T09:21:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 120.0 s, 1417.9 tps, lat 70.237 ms stddev 90.433 2023-10-18T09:21:15.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:15 smithi170 ceph-mon[135150]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.83k op/s 2023-10-18T09:21:15.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:15 smithi110 ceph-mon[134818]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.83k op/s 2023-10-18T09:21:15.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:15 smithi061 ceph-mon[98573]: pgmap v439: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 MiB/s wr, 1.83k op/s 2023-10-18T09:21:17.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:17 smithi170 ceph-mon[135150]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-18T09:21:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:17 smithi110 ceph-mon[134818]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-18T09:21:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:17 smithi061 ceph-mon[98573]: pgmap v440: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 MiB/s wr, 1.76k op/s 2023-10-18T09:21:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 125.0 s, 1470.2 tps, lat 68.706 ms stddev 81.889 2023-10-18T09:21:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:19 smithi170 ceph-mon[135150]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2023-10-18T09:21:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:19 smithi110 ceph-mon[134818]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2023-10-18T09:21:19.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:19 smithi061 ceph-mon[98573]: pgmap v441: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2023-10-18T09:21:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:21 smithi170 ceph-mon[135150]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2023-10-18T09:21:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:21 smithi110 ceph-mon[134818]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2023-10-18T09:21:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:21 smithi061 ceph-mon[98573]: pgmap v442: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 1.59k op/s 2023-10-18T09:21:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:23 smithi170 ceph-mon[135150]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-18T09:21:23.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:23 smithi110 ceph-mon[134818]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-18T09:21:23.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:23 smithi061 ceph-mon[98573]: pgmap v443: 129 pgs: 129 active+clean; 1.5 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 MiB/s wr, 732 op/s 2023-10-18T09:21:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 130.0 s, 1364.6 tps, lat 73.619 ms stddev 106.513 2023-10-18T09:21:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:25 smithi170 ceph-mon[135150]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:25 smithi110 ceph-mon[134818]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:25 smithi061 ceph-mon[98573]: pgmap v444: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:27.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:27 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:27.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:27 smithi170 ceph-mon[135150]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:27.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:27 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:27.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:27 smithi110 ceph-mon[134818]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:27 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:28.006 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:27 smithi061 ceph-mon[98573]: pgmap v445: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 626 op/s 2023-10-18T09:21:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 135.0 s, 1525.0 tps, lat 65.779 ms stddev 92.432 2023-10-18T09:21:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:29 smithi170 ceph-mon[135150]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 577 op/s 2023-10-18T09:21:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:29 smithi110 ceph-mon[134818]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 577 op/s 2023-10-18T09:21:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:29 smithi061 ceph-mon[98573]: pgmap v446: 129 pgs: 129 active+clean; 1.5 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 9.5 MiB/s wr, 577 op/s 2023-10-18T09:21:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:31 smithi170 ceph-mon[135150]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:21:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:31 smithi110 ceph-mon[134818]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:21:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:31 smithi061 ceph-mon[98573]: pgmap v447: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:21:33.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:33 smithi170 ceph-mon[135150]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 587 op/s 2023-10-18T09:21:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:33 smithi110 ceph-mon[134818]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 587 op/s 2023-10-18T09:21:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:33 smithi061 ceph-mon[98573]: pgmap v448: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 587 op/s 2023-10-18T09:21:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 140.0 s, 1531.4 tps, lat 65.170 ms stddev 83.371 2023-10-18T09:21:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:35 smithi170 ceph-mon[135150]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-10-18T09:21:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:35 smithi110 ceph-mon[134818]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-10-18T09:21:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:35 smithi061 ceph-mon[98573]: pgmap v449: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 645 op/s 2023-10-18T09:21:36.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:21:36.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 -- 172.21.15.61:0/488640991 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 msgr2=0x7f80c4100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/488640991 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c4100ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80b80099c0 tx=0x7f80b8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 -- 172.21.15.61:0/488640991 shutdown_connections 2023-10-18T09:21:36.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/488640991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 0x7f80c4107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/488640991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c40ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/488640991 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c4100ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.798+0000 7f80cb1ff700 1 -- 172.21.15.61:0/488640991 >> 172.21.15.61:0/488640991 conn(0x7f80c40fac00 msgr2=0x7f80c40fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.799+0000 7f80cb1ff700 1 -- 172.21.15.61:0/488640991 shutdown_connections 2023-10-18T09:21:36.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.800+0000 7f80cb1ff700 1 -- 172.21.15.61:0/488640991 wait complete. 2023-10-18T09:21:36.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.802+0000 7f80cb1ff700 1 Processor -- start 2023-10-18T09:21:36.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.802+0000 7f80cb1ff700 1 -- start start 2023-10-18T09:21:36.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c41946d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 0x7f80c4190d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41914c0 con 0x7f80c4100880 2023-10-18T09:21:36.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4191630 con 0x7f80c40ff690 2023-10-18T09:21:36.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80cb1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41917f0 con 0x7f80c4101220 2023-10-18T09:21:36.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.804+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.805+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:58334/0 (socket says 172.21.15.61:58334) 2023-10-18T09:21:36.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.805+0000 7f80c8f9b700 1 -- 172.21.15.61:0/3477605276 learned_addr learned my addr 172.21.15.61:0/3477605276 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:36.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.805+0000 7f80c979c700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 0x7f80c4190d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.805+0000 7f80c3fff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c41946d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c8f9b700 1 -- 172.21.15.61:0/3477605276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 msgr2=0x7f80c4190d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 0x7f80c4190d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c8f9b700 1 -- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 msgr2=0x7f80c41946d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c41946d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c8f9b700 1 -- 172.21.15.61:0/3477605276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80095f0 con 0x7f80c40ff690 2023-10-18T09:21:36.810 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.806+0000 7f80c3fff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c41946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:21:36.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.807+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b500 tx=0x7f80b000b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.807+0000 7f80c1ffb700 1 -- 172.21.15.61:0/3477605276 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0011830 con 0x7f80c40ff690 2023-10-18T09:21:36.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.807+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c41986e0 con 0x7f80c40ff690 2023-10-18T09:21:36.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.808+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c4198c10 con 0x7f80c40ff690 2023-10-18T09:21:36.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.809+0000 7f80c1ffb700 1 -- 172.21.15.61:0/3477605276 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b0011e50 con 0x7f80c40ff690 2023-10-18T09:21:36.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.809+0000 7f80c1ffb700 1 -- 172.21.15.61:0/3477605276 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b000f2e0 con 0x7f80c40ff690 2023-10-18T09:21:36.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.819+0000 7f80c1ffb700 1 -- 172.21.15.61:0/3477605276 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f80b000f440 con 0x7f80c40ff690 2023-10-18T09:21:36.825 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.823+0000 7f80deb1cb80 1 Processor -- start 2023-10-18T09:21:36.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.823+0000 7f80deb1cb80 1 -- start start 2023-10-18T09:21:36.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e75c30 0x564d36e74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.827 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80c1ffb700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 0x7f80b40b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80c1ffb700 1 -- 172.21.15.61:0/3477605276 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f80b00cca10 con 0x7f80c40ff690 2023-10-18T09:21:36.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.824+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36de76e0 con 0x564d36e752d0 2023-10-18T09:21:36.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39388/0 (socket says 172.21.15.61:39388) 2023-10-18T09:21:36.828 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 -- 172.21.15.61:0/2999672187 learned_addr learned my addr 172.21.15.61:0/2999672187 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:36.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36dcdc60 con 0x564d36e75c30 2023-10-18T09:21:36.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36dcb580 con 0x564d36e745d0 2023-10-18T09:21:36.829 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e75c30 0x564d36e74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c979c700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 0x7f80b40b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 -- 172.21.15.61:0/2999672187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e75c30 msgr2=0x564d36e74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e75c30 0x564d36e74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 -- 172.21.15.61:0/2999672187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 msgr2=0x564d36e756f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 -- 172.21.15.61:0/2999672187 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d36d843f0 con 0x564d36e745d0 2023-10-18T09:21:36.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.825+0000 7f80c3fff700 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80b8000c00 tx=0x7f80b80050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577126c12d8b98ba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.826+0000 7f80aeffd700 1 -- 172.21.15.61:0/2999672187 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8005560 con 0x564d36e745d0 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.826+0000 7f80aeffd700 1 -- 172.21.15.61:0/2999672187 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b801e040 con 0x564d36e745d0 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.826+0000 7f80aeffd700 1 -- 172.21.15.61:0/2999672187 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003df0 con 0x564d36e745d0 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 msgr2=0x564d36e74a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f80b8000c00 tx=0x7f80b80050c0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80c979c700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 0x7f80b40b4950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00b5b0 tx=0x7f80bc009cd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 shutdown_connections 2023-10-18T09:21:36.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e756f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/2999672187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e75c30 0x564d36e74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.827+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 >> 172.21.15.61:0/2999672187 conn(0x564d36de8720 msgr2=0x564d36e6eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.828+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 shutdown_connections 2023-10-18T09:21:36.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.828+0000 7f80deb1cb80 1 -- 172.21.15.61:0/2999672187 wait complete. 2023-10-18T09:21:36.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.828+0000 7f80deb1cb80 1 -- start start 2023-10-18T09:21:36.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.829+0000 7f80deb1cb80 1 Processor -- start 2023-10-18T09:21:36.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.830+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.830+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.830+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36e85000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36dc6e70 con 0x564d36e752d0 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36d63710 con 0x564d36e745d0 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36d633b0 con 0x564d36e75c30 2023-10-18T09:21:36.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e7da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:58352/0 (socket says 172.21.15.61:58352) 2023-10-18T09:21:36.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80c8f9b700 1 -- 172.21.15.61:0/152039685 learned_addr learned my addr 172.21.15.61:0/152039685 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:36.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80deb1cb80 10 client.0 _ll_register_callbacks cb 0x564d36cada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:21:36.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80c979c700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36e85000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.831+0000 7f80c3fff700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.832+0000 7f80c979c700 1 -- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 msgr2=0x564d36e7da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.832+0000 7f80c979c700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e7da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.832+0000 7f80c979c700 1 -- 172.21.15.61:0/152039685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 msgr2=0x564d36e7dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.832+0000 7f80c979c700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e7dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.832+0000 7f80c979c700 1 -- 172.21.15.61:0/152039685 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c4108420 con 0x564d36e75c30 2023-10-18T09:21:36.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.833+0000 7f80c979c700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36e85000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00bd00 tx=0x7f80bc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d33b77447eb8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.833+0000 7f80a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:21:36.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.833+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc014760 con 0x564d36e75c30 2023-10-18T09:21:36.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.833+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f80bc006e40 con 0x564d36e75c30 2023-10-18T09:21:36.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.833+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d36dcbef0 con 0x564d36e75c30 2023-10-18T09:21:36.840 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.834+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80bc0148c0 con 0x564d36e75c30 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.834+0000 7f80deb1cb80 10 client.26113 fetch_fsmap learned FSMap version 34 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.834+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564d36bf1de0 con 0x564d36e75c30 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d36dcbef0 con 0x564d36e75c30 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f80bc020a60 con 0x564d36e75c30 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 10 client.26113 fetch_fsmap finished waiting for FSMap version 34 2023-10-18T09:21:36.841 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 10 client.26113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:21:36.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 20 client.26113 populate_metadata read hostname 'smithi061' 2023-10-18T09:21:36.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36e899e0 0x564d36e8be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 4 client.26113 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:21:36.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d36e438b0 con 0x564d36e899e0 2023-10-18T09:21:36.842 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.835+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36e899e0 0x564d36e8be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.836+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36e899e0 0x564d36e8be00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b245cc89408e04 server_cookie=8636123c76f89033 in_seq=0 out_seq=0 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.836+0000 7f80a3fff700 10 client.26113 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80a3fff700 1 -- 172.21.15.61:0/152039685 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d36e438b0 con 0x564d36e899e0 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80a3fff700 10 client.26113 handle_command_reply: tid=0 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 client.26113 shutdown 2023-10-18T09:21:36.843 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 msgr2=0x564d36e85000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36e85000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00bd00 tx=0x7f80bc004ad0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 shutdown_connections 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36e899e0 0x564d36e8be00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36e85000 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e7da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.844 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/152039685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e7dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80a3fff700 0 client.26113 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:36.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.837+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 >> 172.21.15.61:0/152039685 conn(0x564d36de8720 msgr2=0x564d36e6eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.838+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 shutdown_connections 2023-10-18T09:21:36.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.838+0000 7f80deb1cb80 1 -- 172.21.15.61:0/152039685 wait complete. 2023-10-18T09:21:36.845 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.839+0000 7f80deb1cb80 20 client.26113 trim_cache size 0 max 16384 2023-10-18T09:21:36.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.876+0000 7f80deb1cb80 1 Processor -- start 2023-10-18T09:21:36.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.876+0000 7f80deb1cb80 1 -- start start 2023-10-18T09:21:36.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 0x564d36e78450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36c47920 con 0x564d36e75c30 2023-10-18T09:21:36.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36e6f5d0 con 0x564d36e752d0 2023-10-18T09:21:36.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36e86c80 con 0x564d36e745d0 2023-10-18T09:21:36.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.877+0000 7f80c979c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 0x564d36e78450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:58364/0 (socket says 172.21.15.61:58364) 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39394/0 (socket says 172.21.15.61:39394) 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c3fff700 1 -- 172.21.15.61:0/1416578393 learned_addr learned my addr 172.21.15.61:0/1416578393 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 -- 172.21.15.61:0/1416578393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 msgr2=0x564d36e74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 -- 172.21.15.61:0/1416578393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 msgr2=0x564d36e84f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 -- 172.21.15.61:0/1416578393 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d36bdac60 con 0x564d36e75c30 2023-10-18T09:21:36.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c3fff700 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:21:36.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.878+0000 7f80c979c700 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 0x564d36e78450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80bc020e70 tx=0x7f80bc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f7e4a1a7aa18ca server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.880+0000 7f80af7fe700 1 -- 172.21.15.61:0/1416578393 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00b040 con 0x564d36e75c30 2023-10-18T09:21:36.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.880+0000 7f80af7fe700 1 -- 172.21.15.61:0/1416578393 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc011040 con 0x564d36e75c30 2023-10-18T09:21:36.884 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.880+0000 7f80af7fe700 1 -- 172.21.15.61:0/1416578393 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc020660 con 0x564d36e75c30 2023-10-18T09:21:36.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.883+0000 7f80deb1cb80 1 -- 172.21.15.61:0/1416578393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 msgr2=0x564d36e78450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.883+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 0x564d36e78450 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80bc020e70 tx=0x7f80bc009c20 comp rx=0 tx=0).stop 2023-10-18T09:21:36.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.884+0000 7f80deb1cb80 1 -- 172.21.15.61:0/1416578393 shutdown_connections 2023-10-18T09:21:36.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.884+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e75c30 0x564d36e78450 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.884+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e745d0 0x564d36e74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.884+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/1416578393 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e752d0 0x564d36e84f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.884+0000 7f80deb1cb80 1 -- 172.21.15.61:0/1416578393 >> 172.21.15.61:0/1416578393 conn(0x564d36de8720 msgr2=0x564d36e7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.885+0000 7f80deb1cb80 1 -- 172.21.15.61:0/1416578393 shutdown_connections 2023-10-18T09:21:36.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.885+0000 7f80deb1cb80 1 -- 172.21.15.61:0/1416578393 wait complete. 2023-10-18T09:21:36.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.885+0000 7f80deb1cb80 1 -- start start 2023-10-18T09:21:36.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.885+0000 7f80deb1cb80 1 Processor -- start 2023-10-18T09:21:36.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36edd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36eddd00 con 0x564d36e752d0 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36edde40 con 0x564d36e745d0 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d36ede000 con 0x564d36e75c30 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80deb1cb80 10 client.0 _ll_register_callbacks cb 0x564d36cada80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80c979c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36edd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80c8f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:58380/0 (socket says 172.21.15.61:58380) 2023-10-18T09:21:36.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80c8f9b700 1 -- 172.21.15.61:0/599707994 learned_addr learned my addr 172.21.15.61:0/599707994 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:21:36.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.886+0000 7f80c979c700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36edd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39406/0 (socket says 172.21.15.61:39406) 2023-10-18T09:21:36.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.887+0000 7f80c8f9b700 1 -- 172.21.15.61:0/599707994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 msgr2=0x564d36edd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.887+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36edd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.887+0000 7f80c8f9b700 1 -- 172.21.15.61:0/599707994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 msgr2=0x564d36e824c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:21:36.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80c3fff700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e824c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80c8f9b700 1 -- 172.21.15.61:0/599707994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d36d843f0 con 0x564d36e745d0 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80c3fff700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80c8f9b700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b4d0 tx=0x7f80b00b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338274a8256868b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.888+0000 7f80a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a1d70 con 0x564d36e745d0 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b801e040 con 0x564d36e745d0 2023-10-18T09:21:36.894 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f80b00b21a0 con 0x564d36e745d0 2023-10-18T09:21:36.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80b0099c40 con 0x564d36e745d0 2023-10-18T09:21:36.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 10 client.26141 fetch_fsmap learned FSMap version 34 2023-10-18T09:21:36.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564d36bf1de0 con 0x564d36e745d0 2023-10-18T09:21:36.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b0065030 con 0x564d36e745d0 2023-10-18T09:21:36.895 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 34) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f80b0098f60 con 0x564d36e745d0 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 10 client.26141 fetch_fsmap finished waiting for FSMap version 34 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 10 client.26141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.889+0000 7f80deb1cb80 20 client.26141 populate_metadata read hostname 'smithi061' 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36edfae0 0x564d36ee1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80deb1cb80 4 client.26141 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:21:36.896 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d36dbfe50 con 0x564d36edfae0 2023-10-18T09:21:36.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80c3fff700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36edfae0 0x564d36ee1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:21:36.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80a3fff700 10 client.26141 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:36.897 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.890+0000 7f80c3fff700 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36edfae0 0x564d36ee1f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492332f7a742b3a9 server_cookie=c9419c74c00a361a in_seq=0 out_seq=0 2023-10-18T09:21:36.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80a3fff700 1 -- 172.21.15.61:0/599707994 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564d36dbfe50 con 0x564d36edfae0 2023-10-18T09:21:36.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80a3fff700 10 client.26141 handle_command_reply: tid=0 2023-10-18T09:21:36.898 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 client.26141 shutdown 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 msgr2=0x564d36e81f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b4d0 tx=0x7f80b00b9040 comp rx=0 tx=0).stop 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 shutdown_connections 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564d36edfae0 0x564d36ee1f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d36e75c30 0x564d36edd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.899 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564d36e745d0 0x564d36e81f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 --2- 172.21.15.61:0/599707994 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564d36e752d0 0x564d36e824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 >> 172.21.15.61:0/599707994 conn(0x564d36de8720 msgr2=0x564d36e7d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80a3fff700 0 client.26141 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:21:36.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 shutdown_connections 2023-10-18T09:21:36.900 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.891+0000 7f80deb1cb80 1 -- 172.21.15.61:0/599707994 wait complete. 2023-10-18T09:21:36.901 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80deb1cb80 20 client.26141 trim_cache size 0 max 16384 2023-10-18T09:21:36.901 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:21:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:21:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:21:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:21:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:21:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:21:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:21:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:21:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:21:36.903 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:21:36.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 msgr2=0x7f80b40b4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.903 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 0x7f80b40b4950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00b5b0 tx=0x7f80bc009cd0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 msgr2=0x7f80c4194190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80b000b500 tx=0x7f80b000b810 comp rx=0 tx=0).stop 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 shutdown_connections 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.892+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80b40b2490 0x7f80b40b4950 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80c4101220 0x7f80c4190d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80c40ff690 0x7f80c4194190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 --2- 172.21.15.61:0/3477605276 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80c4100880 0x7f80c41946d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:21:36.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 >> 172.21.15.61:0/3477605276 conn(0x7f80c40fac00 msgr2=0x7f80c4067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:21:36.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 shutdown_connections 2023-10-18T09:21:36.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:21:36.893+0000 7f80cb1ff700 1 -- 172.21.15.61:0/3477605276 wait complete. 2023-10-18T09:21:36.923 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:21:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:37 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:37.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:37 smithi170 ceph-mon[135150]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 618 op/s 2023-10-18T09:21:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:37 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:37.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:37 smithi110 ceph-mon[134818]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 618 op/s 2023-10-18T09:21:38.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:37 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:21:38.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:37 smithi061 ceph-mon[98573]: pgmap v450: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.9 MiB/s wr, 618 op/s 2023-10-18T09:21:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 145.0 s, 1347.2 tps, lat 75.124 ms stddev 94.814 2023-10-18T09:21:39.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:39 smithi170 ceph-mon[135150]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 584 op/s 2023-10-18T09:21:39.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:39 smithi110 ceph-mon[134818]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 584 op/s 2023-10-18T09:21:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:39 smithi061 ceph-mon[98573]: pgmap v451: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.3 MiB/s wr, 584 op/s 2023-10-18T09:21:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:41 smithi170 ceph-mon[135150]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 736 op/s 2023-10-18T09:21:41.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:41 smithi110 ceph-mon[134818]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 736 op/s 2023-10-18T09:21:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:41 smithi061 ceph-mon[98573]: pgmap v452: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 736 op/s 2023-10-18T09:21:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:43 smithi170 ceph-mon[135150]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2023-10-18T09:21:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:43 smithi110 ceph-mon[134818]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2023-10-18T09:21:43.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:43 smithi061 ceph-mon[98573]: pgmap v453: 129 pgs: 129 active+clean; 1.5 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 MiB/s wr, 615 op/s 2023-10-18T09:21:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 150.0 s, 1801.7 tps, lat 55.368 ms stddev 64.712 2023-10-18T09:21:45.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:45 smithi170 ceph-mon[135150]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 691 op/s 2023-10-18T09:21:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:45 smithi110 ceph-mon[134818]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 691 op/s 2023-10-18T09:21:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:45 smithi061 ceph-mon[98573]: pgmap v454: 129 pgs: 129 active+clean; 1.5 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 MiB/s wr, 691 op/s 2023-10-18T09:21:47.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:47 smithi110 ceph-mon[134818]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 660 op/s 2023-10-18T09:21:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:47 smithi061 ceph-mon[98573]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 660 op/s 2023-10-18T09:21:48.149 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:47 smithi170 ceph-mon[135150]: pgmap v455: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 660 op/s 2023-10-18T09:21:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 155.0 s, 1394.9 tps, lat 70.697 ms stddev 80.065 2023-10-18T09:21:49.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:49 smithi170 ceph-mon[135150]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2023-10-18T09:21:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:49 smithi110 ceph-mon[134818]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2023-10-18T09:21:49.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:49 smithi061 ceph-mon[98573]: pgmap v456: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 614 op/s 2023-10-18T09:21:51.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:51 smithi170 ceph-mon[135150]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 775 op/s 2023-10-18T09:21:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:51 smithi110 ceph-mon[134818]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 775 op/s 2023-10-18T09:21:51.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:51 smithi061 ceph-mon[98573]: pgmap v457: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 MiB/s wr, 775 op/s 2023-10-18T09:21:53.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:53 smithi170 ceph-mon[135150]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 MiB/s wr, 658 op/s 2023-10-18T09:21:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:53 smithi110 ceph-mon[134818]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 MiB/s wr, 658 op/s 2023-10-18T09:21:53.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:53 smithi061 ceph-mon[98573]: pgmap v458: 129 pgs: 129 active+clean; 1.5 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.9 MiB/s wr, 658 op/s 2023-10-18T09:21:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 160.0 s, 833.8 tps, lat 119.868 ms stddev 139.962 2023-10-18T09:21:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:55 smithi170 ceph-mon[135150]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 750 op/s 2023-10-18T09:21:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:55 smithi110 ceph-mon[134818]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 750 op/s 2023-10-18T09:21:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:55 smithi061 ceph-mon[98573]: pgmap v459: 129 pgs: 129 active+clean; 1.5 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 11 MiB/s wr, 750 op/s 2023-10-18T09:21:56.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:56 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:21:56.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:56 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:21:56.512 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:56 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:21:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:57 smithi110 ceph-mon[134818]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.5 MiB/s wr, 711 op/s 2023-10-18T09:21:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:21:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:21:57.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:57 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:21:58.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:57 smithi061 ceph-mon[98573]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.5 MiB/s wr, 711 op/s 2023-10-18T09:21:58.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:21:58.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:21:58.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:57 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:21:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:57 smithi170 ceph-mon[135150]: pgmap v460: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.5 MiB/s wr, 711 op/s 2023-10-18T09:21:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:21:58.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:21:58.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:57 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:21:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 165.0 s, 926.0 tps, lat 107.884 ms stddev 118.734 2023-10-18T09:21:59.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:21:59 smithi170 ceph-mon[135150]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-18T09:21:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:21:59 smithi110 ceph-mon[134818]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-18T09:21:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:21:59 smithi061 ceph-mon[98573]: pgmap v461: 129 pgs: 129 active+clean; 1.5 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 11 MiB/s wr, 709 op/s 2023-10-18T09:22:01.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:01 smithi170 ceph-mon[135150]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 856 op/s 2023-10-18T09:22:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:01 smithi110 ceph-mon[134818]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 856 op/s 2023-10-18T09:22:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:01 smithi061 ceph-mon[98573]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 14 MiB/s wr, 856 op/s 2023-10-18T09:22:03.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:03 smithi170 ceph-mon[135150]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 711 op/s 2023-10-18T09:22:03.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:03 smithi110 ceph-mon[134818]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 711 op/s 2023-10-18T09:22:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:03 smithi061 ceph-mon[98573]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 13 MiB/s wr, 711 op/s 2023-10-18T09:22:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 170.0 s, 1188.6 tps, lat 83.808 ms stddev 102.734 2023-10-18T09:22:05.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:05 smithi170 ceph-mon[135150]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 727 op/s 2023-10-18T09:22:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:05 smithi110 ceph-mon[134818]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 727 op/s 2023-10-18T09:22:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:05 smithi061 ceph-mon[98573]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 727 op/s 2023-10-18T09:22:06.924 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:22:07.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 -- 172.21.15.61:0/541530432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc1007a0 msgr2=0x7f1dfc100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 --2- 172.21.15.61:0/541530432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc100bc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099c0 tx=0x7f1dec009cd0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 -- 172.21.15.61:0/541530432 shutdown_connections 2023-10-18T09:22:07.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 --2- 172.21.15.61:0/541530432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc102330 0x7f1dfc108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 --2- 172.21.15.61:0/541530432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc100bc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 --2- 172.21.15.61:0/541530432 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc101990 0x7f1dfc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 -- 172.21.15.61:0/541530432 >> 172.21.15.61:0/541530432 conn(0x7f1dfc0fbcf0 msgr2=0x7f1dfc0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 -- 172.21.15.61:0/541530432 shutdown_connections 2023-10-18T09:22:07.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.463+0000 7f1e009ff700 1 -- 172.21.15.61:0/541530432 wait complete. 2023-10-18T09:22:07.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.464+0000 7f1e009ff700 1 Processor -- start 2023-10-18T09:22:07.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.465+0000 7f1e009ff700 1 -- start start 2023-10-18T09:22:07.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.465+0000 7f1e009ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1e009ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc101990 0x7f1dfc072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1e009ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 0x7f1dfc073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1e009ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0738c0 con 0x7f1dfc102330 2023-10-18T09:22:07.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1e009ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc073a30 con 0x7f1dfc101990 2023-10-18T09:22:07.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1e009ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1a07b0 con 0x7f1dfc1007a0 2023-10-18T09:22:07.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc101990 0x7f1dfc072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:42368/0 (socket says 172.21.15.61:42368) 2023-10-18T09:22:07.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1dfa59c700 1 -- 172.21.15.61:0/4233336722 learned_addr learned my addr 172.21.15.61:0/4233336722 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:07.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 0x7f1dfc073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.466+0000 7f1dfa59c700 1 -- 172.21.15.61:0/4233336722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc101990 msgr2=0x7f1dfc072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc101990 0x7f1dfc072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfa59c700 1 -- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 msgr2=0x7f1dfc073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 0x7f1dfc073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfa59c700 1 -- 172.21.15.61:0/4233336722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0095f0 con 0x7f1dfc1007a0 2023-10-18T09:22:07.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 0x7f1dfc073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:22:07.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.467+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1dec003730 tx=0x7f1dec00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.468+0000 7f1deb7fe700 1 -- 172.21.15.61:0/4233336722 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec01d070 con 0x7f1dfc1007a0 2023-10-18T09:22:07.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.468+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1a09e0 con 0x7f1dfc1007a0 2023-10-18T09:22:07.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.468+0000 7f1deb7fe700 1 -- 172.21.15.61:0/4233336722 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dec005510 con 0x7f1dfc1007a0 2023-10-18T09:22:07.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.468+0000 7f1deb7fe700 1 -- 172.21.15.61:0/4233336722 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec004ba0 con 0x7f1dfc1007a0 2023-10-18T09:22:07.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.468+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1a0e00 con 0x7f1dfc1007a0 2023-10-18T09:22:07.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.471+0000 7f1deb7fe700 1 -- 172.21.15.61:0/4233336722 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1dec004d00 con 0x7f1dfc1007a0 2023-10-18T09:22:07.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.472+0000 7f1e1437db80 1 Processor -- start 2023-10-18T09:22:07.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 -- start start 2023-10-18T09:22:07.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1deb7fe700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 0x7f1de40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f504090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f504a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f4776e0 con 0x559c0f5045d0 2023-10-18T09:22:07.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f45dc60 con 0x559c0f5052d0 2023-10-18T09:22:07.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.473+0000 7f1e1437db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f45b580 con 0x559c0f505c30 2023-10-18T09:22:07.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 0x7f1de40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1deb7fe700 1 -- 172.21.15.61:0/4233336722 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1dec0cde30 con 0x7f1dfc1007a0 2023-10-18T09:22:07.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1dfad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f504a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f504090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48098/0 (socket says 172.21.15.61:48098) 2023-10-18T09:22:07.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.474+0000 7f1dfa59c700 1 -- 172.21.15.61:0/4062803643 learned_addr learned my addr 172.21.15.61:0/4062803643 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:07.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4062803643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 msgr2=0x559c0f504090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f504090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4062803643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 msgr2=0x559c0f5056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4062803643 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c0f4143f0 con 0x559c0f5045d0 2023-10-18T09:22:07.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.475+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:22:07.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.476+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f504a50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011ee0 tx=0x7f1df4018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29491980576987d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.476+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 0x7f1de40a39c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009fd0 tx=0x7f1df401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.476+0000 7f1de8ff9700 1 -- 172.21.15.61:0/4062803643 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4009ce0 con 0x559c0f5045d0 2023-10-18T09:22:07.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.477+0000 7f1de8ff9700 1 -- 172.21.15.61:0/4062803643 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df400f050 con 0x559c0f5045d0 2023-10-18T09:22:07.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.477+0000 7f1de8ff9700 1 -- 172.21.15.61:0/4062803643 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4007760 con 0x559c0f5045d0 2023-10-18T09:22:07.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 -- 172.21.15.61:0/4062803643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 msgr2=0x559c0f504a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f504a50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011ee0 tx=0x7f1df4018640 comp rx=0 tx=0).stop 2023-10-18T09:22:07.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 -- 172.21.15.61:0/4062803643 shutdown_connections 2023-10-18T09:22:07.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f504a50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f5056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4062803643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f504090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 -- 172.21.15.61:0/4062803643 >> 172.21.15.61:0/4062803643 conn(0x559c0f478720 msgr2=0x559c0f4feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.478+0000 7f1e1437db80 1 -- 172.21.15.61:0/4062803643 shutdown_connections 2023-10-18T09:22:07.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.479+0000 7f1e1437db80 1 -- 172.21.15.61:0/4062803643 wait complete. 2023-10-18T09:22:07.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.479+0000 7f1e1437db80 1 -- start start 2023-10-18T09:22:07.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.479+0000 7f1e1437db80 1 Processor -- start 2023-10-18T09:22:07.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 0x559c0f50e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f456e70 con 0x559c0f5052d0 2023-10-18T09:22:07.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f3f3710 con 0x559c0f5045d0 2023-10-18T09:22:07.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f3f33b0 con 0x559c0f505c30 2023-10-18T09:22:07.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1e1437db80 10 client.0 _ll_register_callbacks cb 0x559c0f33da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:22:07.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 0x559c0f50e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1dfad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48112/0 (socket says 172.21.15.61:48112) 2023-10-18T09:22:07.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1dfad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:42392/0 (socket says 172.21.15.61:42392) 2023-10-18T09:22:07.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.480+0000 7f1dfa59c700 1 -- 172.21.15.61:0/3079667354 learned_addr learned my addr 172.21.15.61:0/3079667354 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:07.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 -- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 msgr2=0x559c0f50dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 -- 172.21.15.61:0/3079667354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 msgr2=0x559c0f50e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 0x559c0f50e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 -- 172.21.15.61:0/3079667354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec09d040 con 0x559c0f505c30 2023-10-18T09:22:07.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1df9d9b700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 0x559c0f50e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:22:07.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:22:07.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011ee0 tx=0x7f1df4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5f841dde3afe3c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:22:07.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4007500 con 0x559c0f505c30 2023-10-18T09:22:07.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c0f45bef0 con 0x559c0f505c30 2023-10-18T09:22:07.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.481+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1df401e070 con 0x559c0f505c30 2023-10-18T09:22:07.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.482+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4007660 con 0x559c0f505c30 2023-10-18T09:22:07.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.482+0000 7f1e1437db80 10 client.26131 fetch_fsmap learned FSMap version 34 2023-10-18T09:22:07.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.482+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f1df4011b10 con 0x559c0f505c30 2023-10-18T09:22:07.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.482+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559c0f45bef0 con 0x559c0f505c30 2023-10-18T09:22:07.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1df4014070 con 0x559c0f505c30 2023-10-18T09:22:07.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 10 client.26131 fetch_fsmap finished waiting for FSMap version 34 2023-10-18T09:22:07.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:22:07.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 20 client.26131 populate_metadata read hostname 'smithi061' 2023-10-18T09:22:07.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f5199e0 0x559c0f51be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 4 client.26131 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:22:07.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c0f4d38b0 con 0x559c0f5199e0 2023-10-18T09:22:07.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.483+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f5199e0 0x559c0f51be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.484+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f5199e0 0x559c0f51be00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ba71f41a37c67 server_cookie=70abc5e977ef6dd0 in_seq=0 out_seq=0 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.484+0000 7f1de1ffb700 10 client.26131 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1de1ffb700 1 -- 172.21.15.61:0/3079667354 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559c0f4d38b0 con 0x559c0f5199e0 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1de1ffb700 10 client.26131 handle_command_reply: tid=0 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 client.26131 shutdown 2023-10-18T09:22:07.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 msgr2=0x559c0f515060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011ee0 tx=0x7f1df4009510 comp rx=0 tx=0).stop 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 shutdown_connections 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f5199e0 0x559c0f51be00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f515060 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5045d0 0x559c0f50dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1de1ffb700 0 client.26131 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:07.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3079667354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5052d0 0x559c0f50e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 >> 172.21.15.61:0/3079667354 conn(0x559c0f478720 msgr2=0x559c0f503080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 shutdown_connections 2023-10-18T09:22:07.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.485+0000 7f1e1437db80 1 -- 172.21.15.61:0/3079667354 wait complete. 2023-10-18T09:22:07.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.486+0000 7f1e1437db80 20 client.26131 trim_cache size 0 max 16384 2023-10-18T09:22:07.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.509+0000 7f1e1437db80 1 Processor -- start 2023-10-18T09:22:07.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 -- start start 2023-10-18T09:22:07.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f50d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f50ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f2d7920 con 0x559c0f5045d0 2023-10-18T09:22:07.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f45bef0 con 0x559c0f5052d0 2023-10-18T09:22:07.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.510+0000 7f1e1437db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f50d670 con 0x559c0f505c30 2023-10-18T09:22:07.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f50d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48120/0 (socket says 172.21.15.61:48120) 2023-10-18T09:22:07.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 -- 172.21.15.61:0/3638311778 learned_addr learned my addr 172.21.15.61:0/3638311778 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:07.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f50ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 -- 172.21.15.61:0/3638311778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 msgr2=0x559c0f50ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f50ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 -- 172.21.15.61:0/3638311778 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 msgr2=0x559c0f50d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f50d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.511+0000 7f1df9d9b700 1 -- 172.21.15.61:0/3638311778 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c0f281de0 con 0x559c0f5052d0 2023-10-18T09:22:07.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.512+0000 7f1df9d9b700 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1df0012580 tx=0x7f1df0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d7040403390169 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.512+0000 7f1de97fa700 1 -- 172.21.15.61:0/3638311778 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0017070 con 0x559c0f5052d0 2023-10-18T09:22:07.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.512+0000 7f1de97fa700 1 -- 172.21.15.61:0/3638311778 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0018040 con 0x559c0f5052d0 2023-10-18T09:22:07.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.512+0000 7f1de97fa700 1 -- 172.21.15.61:0/3638311778 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0003ea0 con 0x559c0f5052d0 2023-10-18T09:22:07.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 -- 172.21.15.61:0/3638311778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 msgr2=0x559c0f515020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1df0012580 tx=0x7f1df0009260 comp rx=0 tx=0).stop 2023-10-18T09:22:07.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 -- 172.21.15.61:0/3638311778 shutdown_connections 2023-10-18T09:22:07.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f505c30 0x559c0f50ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f50d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 --2- 172.21.15.61:0/3638311778 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f5052d0 0x559c0f515020 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 -- 172.21.15.61:0/3638311778 >> 172.21.15.61:0/3638311778 conn(0x559c0f478720 msgr2=0x559c0f50c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.513+0000 7f1e1437db80 1 -- 172.21.15.61:0/3638311778 shutdown_connections 2023-10-18T09:22:07.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.514+0000 7f1e1437db80 1 -- 172.21.15.61:0/3638311778 wait complete. 2023-10-18T09:22:07.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.514+0000 7f1e1437db80 1 -- start start 2023-10-18T09:22:07.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.514+0000 7f1e1437db80 1 Processor -- start 2023-10-18T09:22:07.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f514930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f5052d0 0x559c0f514e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f511a40 con 0x559c0f5045d0 2023-10-18T09:22:07.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f511bb0 con 0x559c0f505c30 2023-10-18T09:22:07.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c0f511d70 con 0x559c0f5052d0 2023-10-18T09:22:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1e1437db80 10 client.0 _ll_register_callbacks cb 0x559c0f33da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:22:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48126/0 (socket says 172.21.15.61:48126) 2023-10-18T09:22:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f5052d0 0x559c0f514e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4139477392 learned_addr learned my addr 172.21.15.61:0/4139477392 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f514930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4139477392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f5052d0 msgr2=0x559c0f514e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f5052d0 0x559c0f514e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4139477392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 msgr2=0x559c0f514930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:07.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f514930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.515+0000 7f1dfad9d700 1 -- 172.21.15.61:0/4139477392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0009de0 con 0x559c0f505c30 2023-10-18T09:22:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1dfad9d700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007fd0 tx=0x7f1df40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d9f30a105e6d193 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:22:07.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df401abe0 con 0x559c0f505c30 2023-10-18T09:22:07.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1df400f050 con 0x559c0f505c30 2023-10-18T09:22:07.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1df401e070 con 0x559c0f505c30 2023-10-18T09:22:07.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4011980 con 0x559c0f505c30 2023-10-18T09:22:07.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1e1437db80 10 client.26159 fetch_fsmap learned FSMap version 34 2023-10-18T09:22:07.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.516+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x559c0f26ac60 con 0x559c0f505c30 2023-10-18T09:22:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df400f050 con 0x559c0f505c30 2023-10-18T09:22:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 34) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1df4014070 con 0x559c0f505c30 2023-10-18T09:22:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 10 client.26159 fetch_fsmap finished waiting for FSMap version 34 2023-10-18T09:22:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 10 client.26159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:22:07.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 20 client.26159 populate_metadata read hostname 'smithi061' 2023-10-18T09:22:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f57fbc0 0x559c0f581fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 4 client.26159 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:22:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c0f44fe50 con 0x559c0f57fbc0 2023-10-18T09:22:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.517+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f57fbc0 0x559c0f581fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:07.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.518+0000 7f1dfa59c700 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f57fbc0 0x559c0f581fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e2b05ea9c7fff3 server_cookie=8fe93a42f23d7f87 in_seq=0 out_seq=0 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.518+0000 7f1de1ffb700 10 client.26159 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.518+0000 7f1de1ffb700 1 -- 172.21.15.61:0/4139477392 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559c0f44fe50 con 0x559c0f57fbc0 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.518+0000 7f1de1ffb700 10 client.26159 handle_command_reply: tid=0 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.518+0000 7f1e1437db80 1 client.26159 shutdown 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 msgr2=0x559c0f511420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007fd0 tx=0x7f1df40117b0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 shutdown_connections 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559c0f57fbc0 0x559c0f581fe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c0f505c30 0x559c0f511420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559c0f5045d0 0x559c0f514930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 --2- 172.21.15.61:0/4139477392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c0f5052d0 0x559c0f514e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 >> 172.21.15.61:0/4139477392 conn(0x559c0f478720 msgr2=0x559c0f50d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.519+0000 7f1de1ffb700 0 client.26159 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:07.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.520+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 shutdown_connections 2023-10-18T09:22:07.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.520+0000 7f1e1437db80 1 -- 172.21.15.61:0/4139477392 wait complete. 2023-10-18T09:22:07.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.520+0000 7f1e1437db80 20 client.26159 trim_cache size 0 max 16384 2023-10-18T09:22:07.533 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:22:07.534 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:22:07.534 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:22:07.534 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:22:07.534 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:22:07.534 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:22:07.535 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:22:07.535 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:22:07.535 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:22:07.535 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:22:07.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 msgr2=0x7f1de40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 0x7f1de40a39c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009fd0 tx=0x7f1df401d000 comp rx=0 tx=0).stop 2023-10-18T09:22:07.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 msgr2=0x7f1dfc074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:07.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1dec003730 tx=0x7f1dec00f740 comp rx=0 tx=0).stop 2023-10-18T09:22:07.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 shutdown_connections 2023-10-18T09:22:07.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1de40a1500 0x7f1de40a39c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1dfc102330 0x7f1dfc073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dfc1007a0 0x7f1dfc074550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 --2- 172.21.15.61:0/4233336722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1dfc101990 0x7f1dfc072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:07.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 >> 172.21.15.61:0/4233336722 conn(0x7f1dfc0fbcf0 msgr2=0x7f1dfc104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:07.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 shutdown_connections 2023-10-18T09:22:07.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:07.521+0000 7f1e009ff700 1 -- 172.21.15.61:0/4233336722 wait complete. 2023-10-18T09:22:07.547 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:22:07.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:07 smithi110 ceph-mon[134818]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-10-18T09:22:08.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:07 smithi061 ceph-mon[98573]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-10-18T09:22:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:07 smithi170 ceph-mon[135150]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 693 op/s 2023-10-18T09:22:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 175.0 s, 1223.6 tps, lat 81.389 ms stddev 98.442 2023-10-18T09:22:09.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:09 smithi170 ceph-mon[135150]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 634 op/s 2023-10-18T09:22:09.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:09 smithi110 ceph-mon[134818]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 634 op/s 2023-10-18T09:22:09.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:09 smithi061 ceph-mon[98573]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 MiB/s wr, 634 op/s 2023-10-18T09:22:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:11 smithi170 ceph-mon[135150]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 756 op/s 2023-10-18T09:22:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:11 smithi110 ceph-mon[134818]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 756 op/s 2023-10-18T09:22:11.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:11 smithi061 ceph-mon[98573]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 MiB/s wr, 756 op/s 2023-10-18T09:22:13.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:13 smithi170 ceph-mon[135150]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 633 op/s 2023-10-18T09:22:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:13 smithi110 ceph-mon[134818]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 633 op/s 2023-10-18T09:22:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:13 smithi061 ceph-mon[98573]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 MiB/s wr, 633 op/s 2023-10-18T09:22:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 180.0 s, 1106.2 tps, lat 90.366 ms stddev 106.006 2023-10-18T09:22:15.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:15 smithi170 ceph-mon[135150]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-10-18T09:22:15.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:15 smithi110 ceph-mon[134818]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-10-18T09:22:15.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:15 smithi061 ceph-mon[98573]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 MiB/s wr, 681 op/s 2023-10-18T09:22:17.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:17 smithi110 ceph-mon[134818]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 666 op/s 2023-10-18T09:22:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:17 smithi061 ceph-mon[98573]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 666 op/s 2023-10-18T09:22:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:17 smithi170 ceph-mon[135150]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 13 MiB/s wr, 666 op/s 2023-10-18T09:22:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 185.0 s, 1016.0 tps, lat 94.662 ms stddev 117.595 2023-10-18T09:22:19.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:19 smithi170 ceph-mon[135150]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-18T09:22:19.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:19 smithi110 ceph-mon[134818]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-18T09:22:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:19 smithi061 ceph-mon[98573]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-18T09:22:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:21 smithi170 ceph-mon[135150]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.17k op/s 2023-10-18T09:22:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:21 smithi110 ceph-mon[134818]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.17k op/s 2023-10-18T09:22:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:21 smithi061 ceph-mon[98573]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 19 MiB/s wr, 1.17k op/s 2023-10-18T09:22:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:23 smithi170 ceph-mon[135150]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-10-18T09:22:23.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:23 smithi110 ceph-mon[134818]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-10-18T09:22:23.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:23 smithi061 ceph-mon[98573]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 21 MiB/s wr, 1.43k op/s 2023-10-18T09:22:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 190.0 s, 610.8 tps, lat 168.740 ms stddev 302.112 2023-10-18T09:22:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:25 smithi170 ceph-mon[135150]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-18T09:22:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:25 smithi110 ceph-mon[134818]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-18T09:22:25.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:25 smithi061 ceph-mon[98573]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-10-18T09:22:27.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:27 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:27.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:27 smithi170 ceph-mon[135150]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-10-18T09:22:27.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:27 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:27.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:27 smithi110 ceph-mon[134818]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-10-18T09:22:27.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:27 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:27.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:27 smithi061 ceph-mon[98573]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-10-18T09:22:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 195.0 s, 1045.6 tps, lat 97.216 ms stddev 122.924 2023-10-18T09:22:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:29 smithi170 ceph-mon[135150]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-10-18T09:22:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:29 smithi110 ceph-mon[134818]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-10-18T09:22:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:29 smithi061 ceph-mon[98573]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 1.59k op/s 2023-10-18T09:22:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:31 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:31 smithi170 ceph-mon[135150]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-18T09:22:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:31 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:31 smithi110 ceph-mon[134818]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-18T09:22:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:31 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:31 smithi061 ceph-mon[98573]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2023-10-18T09:22:33.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:33 smithi170 ceph-mon[135150]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-10-18T09:22:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:33 smithi110 ceph-mon[134818]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-10-18T09:22:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:33 smithi061 ceph-mon[98573]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2023-10-18T09:22:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 200.0 s, 1133.4 tps, lat 88.239 ms stddev 114.784 2023-10-18T09:22:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:35 smithi170 ceph-mon[135150]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-10-18T09:22:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:35 smithi110 ceph-mon[134818]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-10-18T09:22:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:35 smithi061 ceph-mon[98573]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 762 op/s 2023-10-18T09:22:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:37 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:37.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:37 smithi170 ceph-mon[135150]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-18T09:22:37.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:37 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:37.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:37 smithi110 ceph-mon[134818]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-18T09:22:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:37 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:22:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:37 smithi061 ceph-mon[98573]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 669 op/s 2023-10-18T09:22:37.548 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:22:37.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.937+0000 7fa5851ff700 1 -- 172.21.15.61:0/1784727498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580106510 msgr2=0x7fa580108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:37.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.937+0000 7fa5851ff700 1 --2- 172.21.15.61:0/1784727498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580106510 0x7fa580108900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa57800b0a0 tx=0x7fa57800b3b0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 -- 172.21.15.61:0/1784727498 shutdown_connections 2023-10-18T09:22:37.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 --2- 172.21.15.61:0/1784727498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580106510 0x7fa580108900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 --2- 172.21.15.61:0/1784727498 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa5801012b0 0x7fa5801036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 --2- 172.21.15.61:0/1784727498 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa580103be0 0x7fa580105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 -- 172.21.15.61:0/1784727498 >> 172.21.15.61:0/1784727498 conn(0x7fa5800fac00 msgr2=0x7fa5800fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:37.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.938+0000 7fa5851ff700 1 -- 172.21.15.61:0/1784727498 shutdown_connections 2023-10-18T09:22:37.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.939+0000 7fa5851ff700 1 -- 172.21.15.61:0/1784727498 wait complete. 2023-10-18T09:22:37.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.941+0000 7fa5851ff700 1 Processor -- start 2023-10-18T09:22:37.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.942+0000 7fa5851ff700 1 -- start start 2023-10-18T09:22:37.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.942+0000 7fa5851ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.942+0000 7fa5851ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 0x7fa5800ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa5851ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa580106510 0x7fa5800ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa57ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa57ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36582/0 (socket says 172.21.15.61:36582) 2023-10-18T09:22:37.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa57ed9d700 1 -- 172.21.15.61:0/3962036412 learned_addr learned my addr 172.21.15.61:0/3962036412 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:37.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa577fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 0x7fa5800ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa57f59e700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa580106510 0x7fa5800ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58019a6d0 con 0x7fa580106510 2023-10-18T09:22:37.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800ffce0 con 0x7fa5801012b0 2023-10-18T09:22:37.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.943+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800ffea0 con 0x7fa580103be0 2023-10-18T09:22:37.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 -- 172.21.15.61:0/3962036412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 msgr2=0x7fa5800ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:37.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 0x7fa5800ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 -- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa580106510 msgr2=0x7fa5800ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:37.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa580106510 0x7fa5800ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa570009520 con 0x7fa5801012b0 2023-10-18T09:22:37.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa577fff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 0x7fa5800ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:22:37.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eb30 tx=0x7fa57000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:37.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.944+0000 7fa57cd99700 1 -- 172.21.15.61:0/3962036412 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570009880 con 0x7fa5801012b0 2023-10-18T09:22:37.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.945+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa578009d00 con 0x7fa5801012b0 2023-10-18T09:22:37.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.945+0000 7fa57cd99700 1 -- 172.21.15.61:0/3962036412 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5700044e0 con 0x7fa5801012b0 2023-10-18T09:22:37.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.945+0000 7fa57cd99700 1 -- 172.21.15.61:0/3962036412 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570003d80 con 0x7fa5801012b0 2023-10-18T09:22:37.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.945+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5801a5310 con 0x7fa5801012b0 2023-10-18T09:22:37.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.951+0000 7fa57cd99700 1 -- 172.21.15.61:0/3962036412 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa570003f20 con 0x7fa5801012b0 2023-10-18T09:22:37.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.951+0000 7fa598b80b80 1 Processor -- start 2023-10-18T09:22:37.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.951+0000 7fa598b80b80 1 -- start start 2023-10-18T09:22:37.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 0x55cd964e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e6c30 0x55cd964e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964586e0 con 0x55cd964e6c30 2023-10-18T09:22:37.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9643ec60 con 0x55cd964e55d0 2023-10-18T09:22:37.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa598b80b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9643c580 con 0x55cd964e62d0 2023-10-18T09:22:37.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36590/0 (socket says 172.21.15.61:36590) 2023-10-18T09:22:37.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 -- 172.21.15.61:0/4218665612 learned_addr learned my addr 172.21.15.61:0/4218665612 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:37.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa57f59e700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 0x55cd964e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e6c30 0x55cd964e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 -- 172.21.15.61:0/4218665612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 msgr2=0x55cd964e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:37.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 0x55cd964e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 -- 172.21.15.61:0/4218665612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e6c30 msgr2=0x55cd964e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:37.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.952+0000 7fa577fff700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e6c30 0x55cd964e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.953+0000 7fa577fff700 1 -- 172.21.15.61:0/4218665612 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd963f53f0 con 0x55cd964e55d0 2023-10-18T09:22:37.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.953+0000 7fa57f59e700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 0x55cd964e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:22:37.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.953+0000 7fa577fff700 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5680115f0 tx=0x7fa568011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ff0a0e2bdfce32 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:37.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.953+0000 7fa57cd99700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 0x7fa5600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.953+0000 7fa5757fa700 1 -- 172.21.15.61:0/4218665612 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f680 con 0x55cd964e55d0 2023-10-18T09:22:37.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.954+0000 7fa5757fa700 1 -- 172.21.15.61:0/4218665612 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa56800fca0 con 0x55cd964e55d0 2023-10-18T09:22:37.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.954+0000 7fa5757fa700 1 -- 172.21.15.61:0/4218665612 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568019d10 con 0x55cd964e55d0 2023-10-18T09:22:37.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.955+0000 7fa57cd99700 1 -- 172.21.15.61:0/3962036412 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa570014070 con 0x7fa5801012b0 2023-10-18T09:22:37.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.955+0000 7fa598b80b80 1 -- 172.21.15.61:0/4218665612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 msgr2=0x55cd964e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:37.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.955+0000 7fa598b80b80 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa5680115f0 tx=0x7fa568011900 comp rx=0 tx=0).stop 2023-10-18T09:22:37.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa57f59e700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 0x7fa5600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa598b80b80 1 -- 172.21.15.61:0/4218665612 shutdown_connections 2023-10-18T09:22:37.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa598b80b80 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e55d0 0x55cd964e5a50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa598b80b80 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e62d0 0x55cd964e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa598b80b80 1 --2- 172.21.15.61:0/4218665612 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e6c30 0x55cd964e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.956+0000 7fa598b80b80 1 -- 172.21.15.61:0/4218665612 >> 172.21.15.61:0/4218665612 conn(0x55cd96459720 msgr2=0x55cd964dfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:37.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.957+0000 7fa598b80b80 1 -- 172.21.15.61:0/4218665612 shutdown_connections 2023-10-18T09:22:37.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.958+0000 7fa598b80b80 1 -- 172.21.15.61:0/4218665612 wait complete. 2023-10-18T09:22:37.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.958+0000 7fa57f59e700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 0x7fa5600a3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fd0 tx=0x7fa57800b740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:37.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.958+0000 7fa598b80b80 1 -- start start 2023-10-18T09:22:37.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 Processor -- start 2023-10-18T09:22:37.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd96437e70 con 0x55cd964e55d0 2023-10-18T09:22:37.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd963d4710 con 0x55cd964e62d0 2023-10-18T09:22:37.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd963d43b0 con 0x55cd964e6c30 2023-10-18T09:22:37.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.959+0000 7fa598b80b80 10 client.0 _ll_register_callbacks cb 0x55cd9631ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:22:37.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa57f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa57ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa57f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:39964/0 (socket says 172.21.15.61:39964) 2023-10-18T09:22:37.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa57f59e700 1 -- 172.21.15.61:0/3600704070 learned_addr learned my addr 172.21.15.61:0/3600704070 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:37.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 -- 172.21.15.61:0/3600704070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 msgr2=0x55cd964f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:37.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 -- 172.21.15.61:0/3600704070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 msgr2=0x55cd964eeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:37.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964eeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 -- 172.21.15.61:0/3600704070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568009dd0 con 0x55cd964e62d0 2023-10-18T09:22:37.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa577fff700 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964ef0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5680115f0 tx=0x7fa5680196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecde9cb5628df212 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:37.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.960+0000 7fa56e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:22:37.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f680 con 0x55cd964e62d0 2023-10-18T09:22:37.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd9643cef0 con 0x55cd964e62d0 2023-10-18T09:22:37.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa56800f7e0 con 0x55cd964e62d0 2023-10-18T09:22:37.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56800f9a0 con 0x55cd964e62d0 2023-10-18T09:22:37.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa598b80b80 10 client.26171 fetch_fsmap learned FSMap version 37 2023-10-18T09:22:37.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cd96262de0 con 0x55cd964e62d0 2023-10-18T09:22:37.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.961+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cd9643cef0 con 0x55cd964e62d0 2023-10-18T09:22:37.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa568004a50 con 0x55cd964e62d0 2023-10-18T09:22:37.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:22:37.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:22:37.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 20 client.26171 populate_metadata read hostname 'smithi061' 2023-10-18T09:22:37.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd964fa7e0 0x55cd964fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:37.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 4 client.26171 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:22:37.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.962+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd964b48b0 con 0x55cd964fa7e0 2023-10-18T09:22:37.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.963+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd964fa7e0 0x55cd964fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:37.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.964+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd964fa7e0 0x55cd964fcc00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d4642ba0a9993a server_cookie=34fcb13d9fbe6ded in_seq=0 out_seq=0 2023-10-18T09:22:37.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.964+0000 7fa56e7fc700 10 client.26171 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:37.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa56e7fc700 1 -- 172.21.15.61:0/3600704070 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cd964b48b0 con 0x55cd964fa7e0 2023-10-18T09:22:37.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa56e7fc700 10 client.26171 handle_command_reply: tid=0 2023-10-18T09:22:37.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 client.26171 shutdown 2023-10-18T09:22:37.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 msgr2=0x55cd964ef0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:37.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964ef0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa5680115f0 tx=0x7fa5680196f0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 shutdown_connections 2023-10-18T09:22:37.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd964fa7e0 0x55cd964fcc00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964eeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3600704070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964ef0a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:37.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa56e7fc700 0 client.26171 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:37.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.965+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 >> 172.21.15.61:0/3600704070 conn(0x55cd96459720 msgr2=0x55cd964dfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:37.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.966+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 shutdown_connections 2023-10-18T09:22:37.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.966+0000 7fa598b80b80 1 -- 172.21.15.61:0/3600704070 wait complete. 2023-10-18T09:22:37.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:37.966+0000 7fa598b80b80 20 client.26171 trim_cache size 0 max 16384 2023-10-18T09:22:38.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.011+0000 7fa598b80b80 1 Processor -- start 2023-10-18T09:22:38.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.013+0000 7fa598b80b80 1 -- start start 2023-10-18T09:22:38.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd962b8920 con 0x55cd964e55d0 2023-10-18T09:22:38.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964e0120 con 0x55cd964e62d0 2023-10-18T09:22:38.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa598b80b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964e1360 con 0x55cd964e6c30 2023-10-18T09:22:38.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:38.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36606/0 (socket says 172.21.15.61:36606) 2023-10-18T09:22:38.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa577fff700 1 -- 172.21.15.61:0/3833218824 learned_addr learned my addr 172.21.15.61:0/3833218824 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:38.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.014+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:38.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 -- 172.21.15.61:0/3833218824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 msgr2=0x55cd964eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:38.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 -- 172.21.15.61:0/3833218824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 msgr2=0x55cd964e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:38.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 -- 172.21.15.61:0/3833218824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9624bc60 con 0x55cd964e62d0 2023-10-18T09:22:38.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa577fff700 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa568005a80 tx=0x7fa568011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a111aa589b19a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:38.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.015+0000 7fa575ffb700 1 -- 172.21.15.61:0/3833218824 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f480 con 0x55cd964e62d0 2023-10-18T09:22:38.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.016+0000 7fa575ffb700 1 -- 172.21.15.61:0/3833218824 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa56800faa0 con 0x55cd964e62d0 2023-10-18T09:22:38.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.016+0000 7fa575ffb700 1 -- 172.21.15.61:0/3833218824 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568019760 con 0x55cd964e62d0 2023-10-18T09:22:38.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 -- 172.21.15.61:0/3833218824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 msgr2=0x55cd964f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:38.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa568005a80 tx=0x7fa568011900 comp rx=0 tx=0).stop 2023-10-18T09:22:38.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 -- 172.21.15.61:0/3833218824 shutdown_connections 2023-10-18T09:22:38.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 --2- 172.21.15.61:0/3833218824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f60a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.017+0000 7fa598b80b80 1 -- 172.21.15.61:0/3833218824 >> 172.21.15.61:0/3833218824 conn(0x55cd96459720 msgr2=0x55cd964e94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:38.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.018+0000 7fa598b80b80 1 -- 172.21.15.61:0/3833218824 shutdown_connections 2023-10-18T09:22:38.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.018+0000 7fa598b80b80 1 -- 172.21.15.61:0/3833218824 wait complete. 2023-10-18T09:22:38.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.019+0000 7fa598b80b80 1 -- start start 2023-10-18T09:22:38.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.019+0000 7fa598b80b80 1 Processor -- start 2023-10-18T09:22:38.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:38.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964f6100 con 0x55cd964e55d0 2023-10-18T09:22:38.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964f6270 con 0x55cd964e62d0 2023-10-18T09:22:38.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa598b80b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd964f63e0 con 0x55cd964e6c30 2023-10-18T09:22:38.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.021+0000 7fa57ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:38.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa598b80b80 10 client.0 _ll_register_callbacks cb 0x55cd9631ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:22:38.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:35014/0 (socket says 172.21.15.61:35014) 2023-10-18T09:22:38.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 -- 172.21.15.61:0/2616722860 learned_addr learned my addr 172.21.15.61:0/2616722860 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:22:38.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 -- 172.21.15.61:0/2616722860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 msgr2=0x55cd964f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:38.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 -- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 msgr2=0x55cd964f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:22:38.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.022+0000 7fa57ed9d700 1 -- 172.21.15.61:0/2616722860 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd96262de0 con 0x55cd964e55d0 2023-10-18T09:22:38.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.026+0000 7fa57ed9d700 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa5700cad20 tx=0x7fa570003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07e9d88cf3966d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:22:38.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.026+0000 7fa56e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:22:38.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.026+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000cbd0 con 0x55cd964e55d0 2023-10-18T09:22:38.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.026+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa56800fca0 con 0x55cd964e55d0 2023-10-18T09:22:38.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.026+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa57009ac40 con 0x55cd964e55d0 2023-10-18T09:22:38.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.027+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa57009a540 con 0x55cd964e55d0 2023-10-18T09:22:38.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.027+0000 7fa598b80b80 10 client.16032 fetch_fsmap learned FSMap version 37 2023-10-18T09:22:38.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.027+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cd96289250 con 0x55cd964e55d0 2023-10-18T09:22:38.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5700b6080 con 0x55cd964e55d0 2023-10-18T09:22:38.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fa5700cd2f0 con 0x55cd964e55d0 2023-10-18T09:22:38.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:22:38.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 10 client.16032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:22:38.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 20 client.16032 populate_metadata read hostname 'smithi061' 2023-10-18T09:22:38.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd96560ef0 0x55cd96563310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:22:38.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 4 client.16032 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:22:38.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.029+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd96430e50 con 0x55cd96560ef0 2023-10-18T09:22:38.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.031+0000 7fa577fff700 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd96560ef0 0x55cd96563310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:22:38.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.032+0000 7fa577fff700 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd96560ef0 0x55cd96563310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5faebdb413979c6c server_cookie=ae37021401f995ac in_seq=0 out_seq=0 2023-10-18T09:22:38.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.032+0000 7fa56e7fc700 10 client.16032 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:38.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.032+0000 7fa56e7fc700 1 -- 172.21.15.61:0/2616722860 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cd96430e50 con 0x55cd96560ef0 2023-10-18T09:22:38.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.032+0000 7fa56e7fc700 10 client.16032 handle_command_reply: tid=0 2023-10-18T09:22:38.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.032+0000 7fa598b80b80 1 client.16032 shutdown 2023-10-18T09:22:38.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 msgr2=0x55cd964f07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:38.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa5700cad20 tx=0x7fa570003680 comp rx=0 tx=0).stop 2023-10-18T09:22:38.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 shutdown_connections 2023-10-18T09:22:38.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cd96560ef0 0x55cd96563310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd964e6c30 0x55cd964f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cd964e55d0 0x55cd964f07e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 --2- 172.21.15.61:0/2616722860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cd964e62d0 0x55cd964f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 >> 172.21.15.61:0/2616722860 conn(0x55cd96459720 msgr2=0x55cd964ec6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:38.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.033+0000 7fa56e7fc700 0 client.16032 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:22:38.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.034+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 shutdown_connections 2023-10-18T09:22:38.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.036+0000 7fa598b80b80 1 -- 172.21.15.61:0/2616722860 wait complete. 2023-10-18T09:22:38.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.036+0000 7fa598b80b80 20 client.16032 trim_cache size 0 max 16384 2023-10-18T09:22:38.040 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:22:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:22:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:22:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:22:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:22:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:22:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:22:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:22:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:22:38.042 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:22:38.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 msgr2=0x7fa5600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:38.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 0x7fa5600a3940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa578009fd0 tx=0x7fa57800b740 comp rx=0 tx=0).stop 2023-10-18T09:22:38.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 msgr2=0x7fa580100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:22:38.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eb30 tx=0x7fa57000ee40 comp rx=0 tx=0).stop 2023-10-18T09:22:38.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 shutdown_connections 2023-10-18T09:22:38.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa5600a1480 0x7fa5600a3940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa580106510 0x7fa5800ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa5801012b0 0x7fa580100bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 --2- 172.21.15.61:0/3962036412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa580103be0 0x7fa5800ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:22:38.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 >> 172.21.15.61:0/3962036412 conn(0x7fa5800fac00 msgr2=0x7fa580101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:22:38.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 shutdown_connections 2023-10-18T09:22:38.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:22:38.039+0000 7fa5851ff700 1 -- 172.21.15.61:0/3962036412 wait complete. 2023-10-18T09:22:38.074 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:22:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 205.0 s, 1129.8 tps, lat 88.930 ms stddev 105.936 2023-10-18T09:22:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:39 smithi170 ceph-mon[135150]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 558 op/s 2023-10-18T09:22:39.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:39 smithi110 ceph-mon[134818]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 558 op/s 2023-10-18T09:22:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:39 smithi061 ceph-mon[98573]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 9.1 MiB/s wr, 558 op/s 2023-10-18T09:22:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:41 smithi170 ceph-mon[135150]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2023-10-18T09:22:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:41 smithi110 ceph-mon[134818]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2023-10-18T09:22:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:41 smithi061 ceph-mon[98573]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.4 MiB/s wr, 634 op/s 2023-10-18T09:22:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:43 smithi170 ceph-mon[135150]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 637 op/s 2023-10-18T09:22:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:43 smithi110 ceph-mon[134818]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 637 op/s 2023-10-18T09:22:43.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:43 smithi061 ceph-mon[98573]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 637 op/s 2023-10-18T09:22:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 210.0 s, 902.4 tps, lat 110.736 ms stddev 127.368 2023-10-18T09:22:45.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:45 smithi170 ceph-mon[135150]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 625 op/s 2023-10-18T09:22:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:45 smithi110 ceph-mon[134818]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 625 op/s 2023-10-18T09:22:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:45 smithi061 ceph-mon[98573]: pgmap v484: 129 pgs: 129 active+clean; 1.5 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 MiB/s wr, 625 op/s 2023-10-18T09:22:47.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:47 smithi110 ceph-mon[134818]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 655 op/s 2023-10-18T09:22:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:47 smithi061 ceph-mon[98573]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 655 op/s 2023-10-18T09:22:48.145 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:47 smithi170 ceph-mon[135150]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 655 op/s 2023-10-18T09:22:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 215.0 s, 1022.0 tps, lat 98.950 ms stddev 110.564 2023-10-18T09:22:49.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:49 smithi170 ceph-mon[135150]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 MiB/s wr, 594 op/s 2023-10-18T09:22:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:49 smithi110 ceph-mon[134818]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 MiB/s wr, 594 op/s 2023-10-18T09:22:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:49 smithi061 ceph-mon[98573]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 MiB/s wr, 594 op/s 2023-10-18T09:22:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:51 smithi170 ceph-mon[135150]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 641 op/s 2023-10-18T09:22:51.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:51 smithi110 ceph-mon[134818]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 641 op/s 2023-10-18T09:22:51.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:51 smithi061 ceph-mon[98573]: pgmap v487: 129 pgs: 129 active+clean; 1.5 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 MiB/s wr, 641 op/s 2023-10-18T09:22:53.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:53 smithi170 ceph-mon[135150]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 599 op/s 2023-10-18T09:22:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:53 smithi110 ceph-mon[134818]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 599 op/s 2023-10-18T09:22:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:53 smithi061 ceph-mon[98573]: pgmap v488: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 8.1 MiB/s wr, 599 op/s 2023-10-18T09:22:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 220.0 s, 1205.4 tps, lat 81.971 ms stddev 106.019 2023-10-18T09:22:55.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:55 smithi170 ceph-mon[135150]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-18T09:22:55.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:55 smithi110 ceph-mon[134818]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-18T09:22:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:55 smithi061 ceph-mon[98573]: pgmap v489: 129 pgs: 129 active+clean; 1.5 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.3 MiB/s wr, 583 op/s 2023-10-18T09:22:57.966 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:57 smithi110 ceph-mon[134818]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 615 op/s 2023-10-18T09:22:58.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:57 smithi061 ceph-mon[98573]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 615 op/s 2023-10-18T09:22:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:57 smithi170 ceph-mon[135150]: pgmap v490: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.2 MiB/s wr, 615 op/s 2023-10-18T09:22:58.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:58 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:22:59.005 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:58 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:22:59.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:58 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:22:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 225.0 s, 999.2 tps, lat 97.301 ms stddev 133.647 2023-10-18T09:23:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:59 smithi061 ceph-mon[98573]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2023-10-18T09:23:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:23:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:23:00.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:22:59 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:23:00.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:59 smithi170 ceph-mon[135150]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2023-10-18T09:23:00.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:23:00.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:23:00.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:22:59 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:23:00.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:59 smithi110 ceph-mon[134818]: pgmap v491: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 MiB/s wr, 568 op/s 2023-10-18T09:23:00.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:23:00.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:23:00.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:22:59 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:23:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:01 smithi170 ceph-mon[135150]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-10-18T09:23:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:01 smithi110 ceph-mon[134818]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-10-18T09:23:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:01 smithi061 ceph-mon[98573]: pgmap v492: 129 pgs: 129 active+clean; 1.5 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2023-10-18T09:23:03.400 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:03 smithi170 ceph-mon[135150]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.4 MiB/s wr, 630 op/s 2023-10-18T09:23:03.658 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:03 smithi061 ceph-mon[98573]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.4 MiB/s wr, 630 op/s 2023-10-18T09:23:03.658 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:03 smithi110 ceph-mon[134818]: pgmap v493: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 8.4 MiB/s wr, 630 op/s 2023-10-18T09:23:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 230.0 s, 1246.0 tps, lat 82.474 ms stddev 108.083 2023-10-18T09:23:05.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:05 smithi170 ceph-mon[135150]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-10-18T09:23:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:05 smithi110 ceph-mon[134818]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-10-18T09:23:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:05 smithi061 ceph-mon[98573]: pgmap v494: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 604 op/s 2023-10-18T09:23:08.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:07 smithi061 ceph-mon[98573]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 629 op/s 2023-10-18T09:23:08.076 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:23:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:07 smithi170 ceph-mon[135150]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 629 op/s 2023-10-18T09:23:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:07 smithi110 ceph-mon[134818]: pgmap v495: 129 pgs: 129 active+clean; 1.5 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.1 MiB/s wr, 629 op/s 2023-10-18T09:23:08.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 -- 172.21.15.61:0/3843460158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 msgr2=0x7fb3f8104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3843460158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8104020 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80099c0 tx=0x7fb3e8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 -- 172.21.15.61:0/3843460158 shutdown_connections 2023-10-18T09:23:08.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3843460158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3843460158 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3843460158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8104020 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 -- 172.21.15.61:0/3843460158 >> 172.21.15.61:0/3843460158 conn(0x7fb3f80fdf40 msgr2=0x7fb3f8100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.484+0000 7fb4001ff700 1 -- 172.21.15.61:0/3843460158 shutdown_connections 2023-10-18T09:23:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.485+0000 7fb4001ff700 1 -- 172.21.15.61:0/3843460158 wait complete. 2023-10-18T09:23:08.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.487+0000 7fb4001ff700 1 Processor -- start 2023-10-18T09:23:08.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.488+0000 7fb4001ff700 1 -- start start 2023-10-18T09:23:08.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.488+0000 7fb4001ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb4001ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb4001ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb4001ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a0440 con 0x7fb3f81029d0 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb4001ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a0580 con 0x7fb3f8103bc0 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb4001ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f81a0710 con 0x7fb3f8104560 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:59106/0 (socket says 172.21.15.61:59106) 2023-10-18T09:23:08.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3688742910 learned_addr learned my addr 172.21.15.61:0/3688742910 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:08.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.489+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3688742910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 msgr2=0x7fb3f819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 msgr2=0x7fb3f8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3688742910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009520 con 0x7fb3f8103bc0 2023-10-18T09:23:08.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:23:08.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:23:08.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.490+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.491+0000 7fb3eeffd700 1 -- 172.21.15.61:0/3688742910 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e801d070 con 0x7fb3f8103bc0 2023-10-18T09:23:08.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.491+0000 7fb3eeffd700 1 -- 172.21.15.61:0/3688742910 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e80054f0 con 0x7fb3f8103bc0 2023-10-18T09:23:08.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.491+0000 7fb3eeffd700 1 -- 172.21.15.61:0/3688742910 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8004c70 con 0x7fb3f8103bc0 2023-10-18T09:23:08.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.491+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e80095f0 con 0x7fb3f8103bc0 2023-10-18T09:23:08.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.491+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a0bd0 con 0x7fb3f8103bc0 2023-10-18T09:23:08.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.496+0000 7fb3eeffd700 1 -- 172.21.15.61:0/3688742910 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb3e800fbb0 con 0x7fb3f8103bc0 2023-10-18T09:23:08.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.497+0000 7fb3eeffd700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 0x7fb3e40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.497+0000 7fb3eeffd700 1 -- 172.21.15.61:0/3688742910 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb3e80cdde0 con 0x7fb3f8103bc0 2023-10-18T09:23:08.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.497+0000 7fb413c0fb80 1 Processor -- start 2023-10-18T09:23:08.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 -- start start 2023-10-18T09:23:08.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c38fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 0x7fb3e40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c391210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c2fa6e0 con 0x55a43c3903d0 2023-10-18T09:23:08.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c2e0c60 con 0x55a43c38fa90 2023-10-18T09:23:08.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.498+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c2de580 con 0x55a43c390d90 2023-10-18T09:23:08.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fdf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c38fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56392/0 (socket says 172.21.15.61:56392) 2023-10-18T09:23:08.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 -- 172.21.15.61:0/2692871227 learned_addr learned my addr 172.21.15.61:0/2692871227 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:08.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c391210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 -- 172.21.15.61:0/2692871227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 msgr2=0x55a43c391210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c391210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 -- 172.21.15.61:0/2692871227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 msgr2=0x55a43c38fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c38fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 -- 172.21.15.61:0/2692871227 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43c2973f0 con 0x55a43c3903d0 2023-10-18T09:23:08.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.499+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8097850 tx=0x7fb3e809aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17a97a8890eda0a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.500+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 0x7fb3e40a39c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8103a20 tx=0x7fb3f4009d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.500+0000 7fb3e3fff700 1 -- 172.21.15.61:0/2692871227 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80a1070 con 0x55a43c3903d0 2023-10-18T09:23:08.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.500+0000 7fb3e3fff700 1 -- 172.21.15.61:0/2692871227 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e809b4e0 con 0x55a43c3903d0 2023-10-18T09:23:08.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.500+0000 7fb3e3fff700 1 -- 172.21.15.61:0/2692871227 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e809a530 con 0x55a43c3903d0 2023-10-18T09:23:08.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2692871227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 msgr2=0x55a43c390850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8097850 tx=0x7fb3e809aa20 comp rx=0 tx=0).stop 2023-10-18T09:23:08.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2692871227 shutdown_connections 2023-10-18T09:23:08.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c391210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c38fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2692871227 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c3903d0 0x55a43c390850 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.501+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2692871227 >> 172.21.15.61:0/2692871227 conn(0x55a43c2fb720 msgr2=0x55a43c381b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.502+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2692871227 shutdown_connections 2023-10-18T09:23:08.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.502+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2692871227 wait complete. 2023-10-18T09:23:08.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.503+0000 7fb413c0fb80 1 -- start start 2023-10-18T09:23:08.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 Processor -- start 2023-10-18T09:23:08.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c38fa90 0x55a43c399000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c3a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c2d9e70 con 0x55a43c38fa90 2023-10-18T09:23:08.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c276710 con 0x55a43c3903d0 2023-10-18T09:23:08.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c2763b0 con 0x55a43c390d90 2023-10-18T09:23:08.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.505+0000 7fb413c0fb80 10 client.0 _ll_register_callbacks cb 0x55a43c1c0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:23:08.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:59136/0 (socket says 172.21.15.61:59136) 2023-10-18T09:23:08.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3903764940 learned_addr learned my addr 172.21.15.61:0/3903764940 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:08.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c3a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c38fa90 0x55a43c399000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3903764940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 msgr2=0x55a43c3a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c3a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3903764940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c38fa90 msgr2=0x55a43c399000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c38fa90 0x55a43c399000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.506+0000 7fb3fd79a700 1 -- 172.21.15.61:0/3903764940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f81a0bd0 con 0x55a43c3903d0 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb3fd79a700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3e809ad20 tx=0x7fb3e809a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e46b9acb17c5f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb3e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e809ade0 con 0x55a43c3903d0 2023-10-18T09:23:08.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb3e809ef30 con 0x55a43c3903d0 2023-10-18T09:23:08.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a43c2deef0 con 0x55a43c3903d0 2023-10-18T09:23:08.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.507+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3e80aeb30 con 0x55a43c3903d0 2023-10-18T09:23:08.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 10 client.26189 fetch_fsmap learned FSMap version 37 2023-10-18T09:23:08.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a43c104de0 con 0x55a43c3903d0 2023-10-18T09:23:08.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a43c2deef0 con 0x55a43c3903d0 2023-10-18T09:23:08.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb3e80b3320 con 0x55a43c3903d0 2023-10-18T09:23:08.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 10 client.26189 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:23:08.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:23:08.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 20 client.26189 populate_metadata read hostname 'smithi061' 2023-10-18T09:23:08.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3a49a0 0x55a43c3a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 4 client.26189 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:23:08.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.508+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a43c3568b0 con 0x55a43c3a49a0 2023-10-18T09:23:08.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.509+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3a49a0 0x55a43c3a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.509+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3a49a0 0x55a43c3a6dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deb1b91cd11cce4d server_cookie=64d425ce34ad0737 in_seq=0 out_seq=0 2023-10-18T09:23:08.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.509+0000 7fb3e17fa700 10 client.26189 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:08.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.510+0000 7fb3e17fa700 1 -- 172.21.15.61:0/3903764940 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a43c3568b0 con 0x55a43c3a49a0 2023-10-18T09:23:08.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.510+0000 7fb3e17fa700 10 client.26189 handle_command_reply: tid=0 2023-10-18T09:23:08.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.510+0000 7fb413c0fb80 1 client.26189 shutdown 2023-10-18T09:23:08.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 msgr2=0x55a43c399540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb3e809ad20 tx=0x7fb3e809a640 comp rx=0 tx=0).stop 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 shutdown_connections 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3a49a0 0x55a43c3a6dc0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c390d90 0x55a43c3a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c38fa90 0x55a43c399000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/3903764940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c399540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 >> 172.21.15.61:0/3903764940 conn(0x55a43c2fb720 msgr2=0x55a43c386080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.511+0000 7fb3e17fa700 0 client.26189 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:08.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.512+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 shutdown_connections 2023-10-18T09:23:08.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.512+0000 7fb413c0fb80 1 -- 172.21.15.61:0/3903764940 wait complete. 2023-10-18T09:23:08.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.512+0000 7fb413c0fb80 20 client.26189 trim_cache size 0 max 16384 2023-10-18T09:23:08.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.545+0000 7fb413c0fb80 1 Processor -- start 2023-10-18T09:23:08.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 -- start start 2023-10-18T09:23:08.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c398510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c3903d0 0x55a43c3a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c15a920 con 0x55a43c390d90 2023-10-18T09:23:08.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c383640 con 0x55a43c38fa90 2023-10-18T09:23:08.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.548+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c382450 con 0x55a43c3903d0 2023-10-18T09:23:08.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.552+0000 7fb3fd79a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c3903d0 0x55a43c3a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.553+0000 7fb3fe79c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.553+0000 7fb3fe79c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56406/0 (socket says 172.21.15.61:56406) 2023-10-18T09:23:08.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.553+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2614630025 learned_addr learned my addr 172.21.15.61:0/2614630025 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:08.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2614630025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c3903d0 msgr2=0x55a43c3a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c3903d0 0x55a43c3a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2614630025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 msgr2=0x55a43c398510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c398510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2614630025 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43c0edc60 con 0x55a43c390d90 2023-10-18T09:23:08.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.555+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007630 tx=0x7fb3f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cfaf708ad20ead server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.556+0000 7fb3ecff9700 1 -- 172.21.15.61:0/2614630025 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004bb0 con 0x55a43c390d90 2023-10-18T09:23:08.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.556+0000 7fb3ecff9700 1 -- 172.21.15.61:0/2614630025 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3f000f030 con 0x55a43c390d90 2023-10-18T09:23:08.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.556+0000 7fb3ecff9700 1 -- 172.21.15.61:0/2614630025 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00051a0 con 0x55a43c390d90 2023-10-18T09:23:08.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.557+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2614630025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 msgr2=0x55a43c397330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.557+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007630 tx=0x7fb3f000b040 comp rx=0 tx=0).stop 2023-10-18T09:23:08.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.558+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2614630025 shutdown_connections 2023-10-18T09:23:08.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.558+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c397330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.558+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c38fa90 0x55a43c398510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.558+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2614630025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c3903d0 0x55a43c3a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.558+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2614630025 >> 172.21.15.61:0/2614630025 conn(0x55a43c2fb720 msgr2=0x55a43c397d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.559+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2614630025 shutdown_connections 2023-10-18T09:23:08.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.559+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2614630025 wait complete. 2023-10-18T09:23:08.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.559+0000 7fb413c0fb80 1 -- start start 2023-10-18T09:23:08.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.560+0000 7fb413c0fb80 1 Processor -- start 2023-10-18T09:23:08.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c38fa90 0x55a43c3a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c39e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c3a0940 con 0x55a43c390d90 2023-10-18T09:23:08.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c39f2f0 con 0x55a43c3903d0 2023-10-18T09:23:08.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c39f4b0 con 0x55a43c38fa90 2023-10-18T09:23:08.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb3fe79c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.563+0000 7fb413c0fb80 10 client.0 _ll_register_callbacks cb 0x55a43c1c0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:23:08.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.564+0000 7fb3fe79c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56418/0 (socket says 172.21.15.61:56418) 2023-10-18T09:23:08.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.564+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2419898295 learned_addr learned my addr 172.21.15.61:0/2419898295 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:08.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.564+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c38fa90 0x55a43c3a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.565+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2419898295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c38fa90 msgr2=0x55a43c3a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.565+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c38fa90 0x55a43c3a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.565+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 msgr2=0x55a43c39e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:08.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.565+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c39e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.565+0000 7fb3fe79c700 1 -- 172.21.15.61:0/2419898295 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f00092c0 con 0x55a43c390d90 2023-10-18T09:23:08.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.568+0000 7fb3fe79c700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0000f80 tx=0x7fb3f0017960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffc652766bbea29 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:08.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.568+0000 7fb3e17fa700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:23:08.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.568+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005330 con 0x55a43c390d90 2023-10-18T09:23:08.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.568+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb3f0005490 con 0x55a43c390d90 2023-10-18T09:23:08.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.568+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3e809b4e0 con 0x55a43c390d90 2023-10-18T09:23:08.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.569+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3f0005650 con 0x55a43c390d90 2023-10-18T09:23:08.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.569+0000 7fb413c0fb80 10 client.16047 fetch_fsmap learned FSMap version 37 2023-10-18T09:23:08.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.569+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a43c12b250 con 0x55a43c390d90 2023-10-18T09:23:08.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.570+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3e809b4e0 con 0x55a43c390d90 2023-10-18T09:23:08.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.570+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb3f00203e0 con 0x55a43c390d90 2023-10-18T09:23:08.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.570+0000 7fb413c0fb80 10 client.16047 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:23:08.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.570+0000 7fb413c0fb80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:23:08.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.571+0000 7fb413c0fb80 20 client.16047 populate_metadata read hostname 'smithi061' 2023-10-18T09:23:08.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.571+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3f9ba0 0x55a43c3fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:08.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.571+0000 7fb413c0fb80 4 client.16047 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:23:08.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.571+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a43c2d2e50 con 0x55a43c3f9ba0 2023-10-18T09:23:08.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.571+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3f9ba0 0x55a43c3fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:08.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb3fdf9b700 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3f9ba0 0x55a43c3fbfc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec9b9f7487987a03 server_cookie=c5776f89329cb3a4 in_seq=0 out_seq=0 2023-10-18T09:23:08.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb3e17fa700 10 client.16047 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:08.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb3e17fa700 1 -- 172.21.15.61:0/2419898295 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a43c2d2e50 con 0x55a43c3f9ba0 2023-10-18T09:23:08.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb3e17fa700 10 client.16047 handle_command_reply: tid=0 2023-10-18T09:23:08.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb413c0fb80 1 client.16047 shutdown 2023-10-18T09:23:08.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 msgr2=0x55a43c39ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.572+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0000f80 tx=0x7fb3f0017960 comp rx=0 tx=0).stop 2023-10-18T09:23:08.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 shutdown_connections 2023-10-18T09:23:08.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a43c3f9ba0 0x55a43c3fbfc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a43c390d90 0x55a43c39ed80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a43c38fa90 0x55a43c3a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb3e17fa700 0 client.16047 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:08.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 --2- 172.21.15.61:0/2419898295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a43c3903d0 0x55a43c39e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 >> 172.21.15.61:0/2419898295 conn(0x55a43c2fb720 msgr2=0x55a43c397330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.573+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 shutdown_connections 2023-10-18T09:23:08.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.575+0000 7fb413c0fb80 1 -- 172.21.15.61:0/2419898295 wait complete. 2023-10-18T09:23:08.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.575+0000 7fb413c0fb80 20 client.16047 trim_cache size 0 max 16384 2023-10-18T09:23:08.580 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:23:08.580 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:23:08.581 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:23:08.582 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:23:08.582 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:23:08.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 msgr2=0x7fb3e40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 0x7fb3e40a39c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8103a20 tx=0x7fb3f4009d30 comp rx=0 tx=0).stop 2023-10-18T09:23:08.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 msgr2=0x7fb3f8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:08.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006010 tx=0x7fb3e800f740 comp rx=0 tx=0).stop 2023-10-18T09:23:08.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 shutdown_connections 2023-10-18T09:23:08.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb3e40a1500 0x7fb3e40a39c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f8104560 0x7fb3f819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb3f81029d0 0x7fb3f8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 --2- 172.21.15.61:0/3688742910 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3f8103bc0 0x7fb3f8198e50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:08.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.576+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 >> 172.21.15.61:0/3688742910 conn(0x7fb3f80fdf40 msgr2=0x7fb3f8106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:08.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.581+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 shutdown_connections 2023-10-18T09:23:08.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:08.581+0000 7fb4001ff700 1 -- 172.21.15.61:0/3688742910 wait complete. 2023-10-18T09:23:08.617 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:23:09.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 235.0 s, 1023.4 tps, lat 97.113 ms stddev 122.385 2023-10-18T09:23:09.402 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:09 smithi170 ceph-mon[135150]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 580 op/s 2023-10-18T09:23:09.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:09 smithi110 ceph-mon[134818]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 580 op/s 2023-10-18T09:23:09.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:09 smithi061 ceph-mon[98573]: pgmap v496: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.4 MiB/s wr, 580 op/s 2023-10-18T09:23:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:11 smithi170 ceph-mon[135150]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 617 op/s 2023-10-18T09:23:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:11 smithi110 ceph-mon[134818]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 617 op/s 2023-10-18T09:23:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:11 smithi061 ceph-mon[98573]: pgmap v497: 129 pgs: 129 active+clean; 1.5 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 7.5 MiB/s wr, 617 op/s 2023-10-18T09:23:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:13 smithi170 ceph-mon[135150]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2023-10-18T09:23:13.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:13 smithi110 ceph-mon[134818]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2023-10-18T09:23:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:13 smithi061 ceph-mon[98573]: pgmap v498: 129 pgs: 129 active+clean; 1.5 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2023-10-18T09:23:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 240.0 s, 1095.8 tps, lat 92.104 ms stddev 119.363 2023-10-18T09:23:15.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:15 smithi170 ceph-mon[135150]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 579 op/s 2023-10-18T09:23:15.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:15 smithi110 ceph-mon[134818]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 579 op/s 2023-10-18T09:23:15.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:15 smithi061 ceph-mon[98573]: pgmap v499: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 MiB/s wr, 579 op/s 2023-10-18T09:23:18.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:17 smithi061 ceph-mon[98573]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 637 op/s 2023-10-18T09:23:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:17 smithi170 ceph-mon[135150]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 637 op/s 2023-10-18T09:23:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:17 smithi110 ceph-mon[134818]: pgmap v500: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.5 MiB/s wr, 637 op/s 2023-10-18T09:23:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 245.0 s, 1066.6 tps, lat 91.597 ms stddev 101.368 2023-10-18T09:23:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:19 smithi170 ceph-mon[135150]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 572 op/s 2023-10-18T09:23:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:19 smithi110 ceph-mon[134818]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 572 op/s 2023-10-18T09:23:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:19 smithi061 ceph-mon[98573]: pgmap v501: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.8 MiB/s wr, 572 op/s 2023-10-18T09:23:21.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:21 smithi170 ceph-mon[135150]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 622 op/s 2023-10-18T09:23:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:21 smithi110 ceph-mon[134818]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 622 op/s 2023-10-18T09:23:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:21 smithi061 ceph-mon[98573]: pgmap v502: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 622 op/s 2023-10-18T09:23:23.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:23 smithi170 ceph-mon[135150]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.4 MiB/s wr, 615 op/s 2023-10-18T09:23:23.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:23 smithi110 ceph-mon[134818]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.4 MiB/s wr, 615 op/s 2023-10-18T09:23:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:23 smithi061 ceph-mon[98573]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 7.4 MiB/s wr, 615 op/s 2023-10-18T09:23:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 250.0 s, 750.6 tps, lat 132.894 ms stddev 153.746 2023-10-18T09:23:25.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:25 smithi170 ceph-mon[135150]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-10-18T09:23:25.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:25 smithi110 ceph-mon[134818]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-10-18T09:23:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:25 smithi061 ceph-mon[98573]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2023-10-18T09:23:28.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:27 smithi061 ceph-mon[98573]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 640 op/s 2023-10-18T09:23:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:27 smithi170 ceph-mon[135150]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 640 op/s 2023-10-18T09:23:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:27 smithi110 ceph-mon[134818]: pgmap v505: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.1 MiB/s wr, 640 op/s 2023-10-18T09:23:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 255.0 s, 569.8 tps, lat 178.756 ms stddev 218.365 2023-10-18T09:23:29.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:29 smithi170 ceph-mon[135150]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 594 op/s 2023-10-18T09:23:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:29 smithi110 ceph-mon[134818]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 594 op/s 2023-10-18T09:23:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:29 smithi061 ceph-mon[98573]: pgmap v506: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 7.3 MiB/s wr, 594 op/s 2023-10-18T09:23:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:31 smithi170 ceph-mon[135150]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2023-10-18T09:23:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:31 smithi110 ceph-mon[134818]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2023-10-18T09:23:31.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:31 smithi061 ceph-mon[98573]: pgmap v507: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 MiB/s wr, 692 op/s 2023-10-18T09:23:33.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:33 smithi170 ceph-mon[135150]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.7 MiB/s wr, 671 op/s 2023-10-18T09:23:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:33 smithi110 ceph-mon[134818]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.7 MiB/s wr, 671 op/s 2023-10-18T09:23:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:33 smithi061 ceph-mon[98573]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 9.7 MiB/s wr, 671 op/s 2023-10-18T09:23:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 260.0 s, 751.4 tps, lat 133.732 ms stddev 165.936 2023-10-18T09:23:35.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:35 smithi170 ceph-mon[135150]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 MiB/s wr, 656 op/s 2023-10-18T09:23:35.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:35 smithi110 ceph-mon[134818]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 MiB/s wr, 656 op/s 2023-10-18T09:23:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:35 smithi061 ceph-mon[98573]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.5 MiB/s wr, 656 op/s 2023-10-18T09:23:38.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:37 smithi061 ceph-mon[98573]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 742 op/s 2023-10-18T09:23:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:37 smithi170 ceph-mon[135150]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 742 op/s 2023-10-18T09:23:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:37 smithi110 ceph-mon[134818]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 MiB/s wr, 742 op/s 2023-10-18T09:23:38.618 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:23:38.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 -- 172.21.15.61:0/565172298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c104340 msgr2=0x7fae6c1047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:38.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 --2- 172.21.15.61:0/565172298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c104340 0x7fae6c1047c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0098f0 tx=0x7fae5c009c00 comp rx=0 tx=0).stop 2023-10-18T09:23:38.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 -- 172.21.15.61:0/565172298 shutdown_connections 2023-10-18T09:23:38.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 --2- 172.21.15.61:0/565172298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c104340 0x7fae6c1047c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 --2- 172.21.15.61:0/565172298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c0ff000 0x7fae6c101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 --2- 172.21.15.61:0/565172298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 -- 172.21.15.61:0/565172298 >> 172.21.15.61:0/565172298 conn(0x7fae6c0fac00 msgr2=0x7fae6c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:38.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 -- 172.21.15.61:0/565172298 shutdown_connections 2023-10-18T09:23:38.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.971+0000 7fae721ff700 1 -- 172.21.15.61:0/565172298 wait complete. 2023-10-18T09:23:38.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.972+0000 7fae721ff700 1 Processor -- start 2023-10-18T09:23:38.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.972+0000 7fae721ff700 1 -- start start 2023-10-18T09:23:38.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 0x7fae6c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c19d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 0x7fae6c1a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c1a4880 con 0x7fae6c101970 2023-10-18T09:23:38.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c1a49f0 con 0x7fae6c104340 2023-10-18T09:23:38.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae721ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c1a4bb0 con 0x7fae6c0ff000 2023-10-18T09:23:38.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae6b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 0x7fae6c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae6b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 0x7fae6c19cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44006/0 (socket says 172.21.15.61:44006) 2023-10-18T09:23:38.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae6b7fe700 1 -- 172.21.15.61:0/3276092476 learned_addr learned my addr 172.21.15.61:0/3276092476 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:38.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 0x7fae6c1a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.973+0000 7fae6affd700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c19d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6bfff700 1 -- 172.21.15.61:0/3276092476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 msgr2=0x7fae6c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:38.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 0x7fae6c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6bfff700 1 -- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 msgr2=0x7fae6c19d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:38.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c19d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6bfff700 1 -- 172.21.15.61:0/3276092476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640095f0 con 0x7fae6c104340 2023-10-18T09:23:38.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.974+0000 7fae6affd700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:23:38.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.975+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 0x7fae6c1a42b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00fb70 tx=0x7fae5c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:38.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.975+0000 7fae68ff9700 1 -- 172.21.15.61:0/3276092476 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c01c070 con 0x7fae6c104340 2023-10-18T09:23:38.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.976+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c009520 con 0x7fae6c104340 2023-10-18T09:23:38.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.976+0000 7fae68ff9700 1 -- 172.21.15.61:0/3276092476 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5c00f7d0 con 0x7fae6c104340 2023-10-18T09:23:38.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.976+0000 7fae68ff9700 1 -- 172.21.15.61:0/3276092476 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c003950 con 0x7fae6c104340 2023-10-18T09:23:38.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.976+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c1a5120 con 0x7fae6c104340 2023-10-18T09:23:38.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.980+0000 7fae68ff9700 1 -- 172.21.15.61:0/3276092476 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fae5c020070 con 0x7fae6c104340 2023-10-18T09:23:38.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.981+0000 7fae85af7b80 1 Processor -- start 2023-10-18T09:23:38.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.981+0000 7fae85af7b80 1 -- start start 2023-10-18T09:23:38.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a418c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a418ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a40fe6e0 con 0x5635a418b5d0 2023-10-18T09:23:38.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a40e4c60 con 0x5635a418cc30 2023-10-18T09:23:38.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.982+0000 7fae85af7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a40e2580 con 0x5635a418c2d0 2023-10-18T09:23:38.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae68ff9700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 0x7fae580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48866/0 (socket says 172.21.15.61:48866) 2023-10-18T09:23:38.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 -- 172.21.15.61:0/2816204172 learned_addr learned my addr 172.21.15.61:0/2816204172 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:38.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae68ff9700 1 -- 172.21.15.61:0/3276092476 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae5c0cd9d0 con 0x7fae6c104340 2023-10-18T09:23:38.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 0x7fae580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6affd700 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a418ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 -- 172.21.15.61:0/2816204172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 msgr2=0x5635a418c6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:23:38.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a418c6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 -- 172.21.15.61:0/2816204172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 msgr2=0x5635a418ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:38.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a418ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.983+0000 7fae6b7fe700 1 -- 172.21.15.61:0/2816204172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a409b3f0 con 0x5635a418cc30 2023-10-18T09:23:38.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.984+0000 7fae6affd700 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a418ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:23:38.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.985+0000 7fae6bfff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 0x7fae580a39c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0cb180 tx=0x7fae5c009c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:38.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.986+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae64009fd0 tx=0x7fae640117a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63eb06c22666e8d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:38.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.987+0000 7fae61ffb700 1 -- 172.21.15.61:0/2816204172 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64019630 con 0x5635a418cc30 2023-10-18T09:23:38.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.987+0000 7fae61ffb700 1 -- 172.21.15.61:0/2816204172 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae64019c50 con 0x5635a418cc30 2023-10-18T09:23:38.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.988+0000 7fae61ffb700 1 -- 172.21.15.61:0/2816204172 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae64019630 con 0x5635a418cc30 2023-10-18T09:23:38.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.988+0000 7fae85af7b80 1 -- 172.21.15.61:0/2816204172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 msgr2=0x5635a418b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:38.991 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.988+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae64009fd0 tx=0x7fae640117a0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 -- 172.21.15.61:0/2816204172 shutdown_connections 2023-10-18T09:23:38.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a418ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a418c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2816204172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a418b090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 -- 172.21.15.61:0/2816204172 >> 172.21.15.61:0/2816204172 conn(0x5635a40ff720 msgr2=0x5635a4185b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:38.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 -- 172.21.15.61:0/2816204172 shutdown_connections 2023-10-18T09:23:38.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 -- 172.21.15.61:0/2816204172 wait complete. 2023-10-18T09:23:38.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.989+0000 7fae85af7b80 1 -- start start 2023-10-18T09:23:38.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.990+0000 7fae85af7b80 1 Processor -- start 2023-10-18T09:23:38.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.990+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.990+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a41950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:38.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae85af7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a40dde70 con 0x5635a418b5d0 2023-10-18T09:23:38.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae85af7b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a407a710 con 0x5635a418cc30 2023-10-18T09:23:38.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae85af7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a407a3b0 con 0x5635a418c2d0 2023-10-18T09:23:38.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae85af7b80 10 client.0 _ll_register_callbacks cb 0x5635a3fc4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:23:38.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae6affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae6affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:48876/0 (socket says 172.21.15.61:48876) 2023-10-18T09:23:38.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.991+0000 7fae6affd700 1 -- 172.21.15.61:0/895451172 learned_addr learned my addr 172.21.15.61:0/895451172 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:38.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:38.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 -- 172.21.15.61:0/895451172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 msgr2=0x5635a41950e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:23:38.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a41950e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 -- 172.21.15.61:0/895451172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 msgr2=0x5635a4194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:38.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:38.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.992+0000 7fae6affd700 1 -- 172.21.15.61:0/895451172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae64009cb0 con 0x5635a418cc30 2023-10-18T09:23:38.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.996+0000 7fae6affd700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae5400afc0 tx=0x7fae5400d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e34f3cb9411a6ac server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:38.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:38.996+0000 7fae4affd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:23:39.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401a5f0 con 0x5635a418cc30 2023-10-18T09:23:39.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae54006e40 con 0x5635a418cc30 2023-10-18T09:23:39.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635a40e2ef0 con 0x5635a418cc30 2023-10-18T09:23:39.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5401a750 con 0x5635a418cc30 2023-10-18T09:23:39.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae85af7b80 10 client.26201 fetch_fsmap learned FSMap version 37 2023-10-18T09:23:39.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.004+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5635a3f08de0 con 0x5635a418cc30 2023-10-18T09:23:39.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.005+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635a40e2ef0 con 0x5635a418cc30 2023-10-18T09:23:39.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fae54008070 con 0x5635a418cc30 2023-10-18T09:23:39.009 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 10 client.26201 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:23:39.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:23:39.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 20 client.26201 populate_metadata read hostname 'smithi061' 2023-10-18T09:23:39.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a41a08f0 0x5635a41a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.010 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 4 client.26201 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:23:39.011 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.006+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635a415a8b0 con 0x5635a41a08f0 2023-10-18T09:23:39.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.015+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a41a08f0 0x5635a41a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.020+0000 7fae4affd700 10 client.26201 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:39.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.020+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a41a08f0 0x5635a41a2d10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cbbe8921655a7fb server_cookie=f4e7e66af5f52947 in_seq=0 out_seq=0 2023-10-18T09:23:39.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.036+0000 7fae4affd700 1 -- 172.21.15.61:0/895451172 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5635a415a8b0 con 0x5635a41a08f0 2023-10-18T09:23:39.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.036+0000 7fae4affd700 10 client.26201 handle_command_reply: tid=0 2023-10-18T09:23:39.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.036+0000 7fae85af7b80 1 client.26201 shutdown 2023-10-18T09:23:39.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.036+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 msgr2=0x5635a419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:39.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.036+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae5400afc0 tx=0x7fae5400d040 comp rx=0 tx=0).stop 2023-10-18T09:23:39.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 shutdown_connections 2023-10-18T09:23:39.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a41a08f0 0x5635a41a2d10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a419c0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae4affd700 0 client.26201 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:39.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 --2- 172.21.15.61:0/895451172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418c2d0 0x5635a41950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 >> 172.21.15.61:0/895451172 conn(0x5635a40ff720 msgr2=0x5635a418a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:39.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 shutdown_connections 2023-10-18T09:23:39.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 1 -- 172.21.15.61:0/895451172 wait complete. 2023-10-18T09:23:39.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.037+0000 7fae85af7b80 20 client.26201 trim_cache size 0 max 16384 2023-10-18T09:23:39.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.075+0000 7fae85af7b80 1 Processor -- start 2023-10-18T09:23:39.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.075+0000 7fae85af7b80 1 -- start start 2023-10-18T09:23:39.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a419c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a41950e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a3f5e920 con 0x5635a418c2d0 2023-10-18T09:23:39.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a4186930 con 0x5635a418cc30 2023-10-18T09:23:39.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae85af7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a4195b80 con 0x5635a418b5d0 2023-10-18T09:23:39.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae6affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae6affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44034/0 (socket says 172.21.15.61:44034) 2023-10-18T09:23:39.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.076+0000 7fae6affd700 1 -- 172.21.15.61:0/3759182168 learned_addr learned my addr 172.21.15.61:0/3759182168 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:39.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.077+0000 7fae6affd700 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a41950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.077+0000 7fae6affd700 1 -- 172.21.15.61:0/3759182168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 msgr2=0x5635a4194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:39.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.077+0000 7fae6affd700 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.078+0000 7fae6affd700 1 -- 172.21.15.61:0/3759182168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 msgr2=0x5635a419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:39.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.078+0000 7fae6affd700 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.078+0000 7fae6affd700 1 -- 172.21.15.61:0/3759182168 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635a3ef1c60 con 0x5635a418cc30 2023-10-18T09:23:39.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.078+0000 7fae6affd700 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a41950e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae54009ee0 tx=0x7fae5400a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9fdf890b99fdf7b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:39.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.080+0000 7fae627fc700 1 -- 172.21.15.61:0/3759182168 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5402c680 con 0x5635a418cc30 2023-10-18T09:23:39.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.080+0000 7fae627fc700 1 -- 172.21.15.61:0/3759182168 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae54011040 con 0x5635a418cc30 2023-10-18T09:23:39.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.080+0000 7fae627fc700 1 -- 172.21.15.61:0/3759182168 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401a730 con 0x5635a418cc30 2023-10-18T09:23:39.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 -- 172.21.15.61:0/3759182168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 msgr2=0x5635a41950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:39.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a41950e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fae54009ee0 tx=0x7fae5400a730 comp rx=0 tx=0).stop 2023-10-18T09:23:39.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 -- 172.21.15.61:0/3759182168 shutdown_connections 2023-10-18T09:23:39.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a418cc30 0x5635a41950e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 --2- 172.21.15.61:0/3759182168 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 -- 172.21.15.61:0/3759182168 >> 172.21.15.61:0/3759182168 conn(0x5635a40ff720 msgr2=0x5635a41940b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:39.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 -- 172.21.15.61:0/3759182168 shutdown_connections 2023-10-18T09:23:39.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.081+0000 7fae85af7b80 1 -- 172.21.15.61:0/3759182168 wait complete. 2023-10-18T09:23:39.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.082+0000 7fae85af7b80 1 -- start start 2023-10-18T09:23:39.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.082+0000 7fae85af7b80 1 Processor -- start 2023-10-18T09:23:39.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a4199330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a4203710 0x5635a4205b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a4199900 con 0x5635a418c2d0 2023-10-18T09:23:39.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a4199a70 con 0x5635a4203710 2023-10-18T09:23:39.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635a42060c0 con 0x5635a418b5d0 2023-10-18T09:23:39.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae85af7b80 10 client.0 _ll_register_callbacks cb 0x5635a3fc4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:23:39.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae6affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae6affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:44050/0 (socket says 172.21.15.61:44050) 2023-10-18T09:23:39.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae6affd700 1 -- 172.21.15.61:0/2044890354 learned_addr learned my addr 172.21.15.61:0/2044890354 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:23:39.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.083+0000 7fae6affd700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a4203710 0x5635a4205b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.084+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.084+0000 7fae6affd700 1 -- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a4203710 msgr2=0x5635a4205b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:39.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.084+0000 7fae6affd700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a4203710 0x5635a4205b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.084+0000 7fae6affd700 1 -- 172.21.15.61:0/2044890354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 msgr2=0x5635a4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:23:39.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.084+0000 7fae6affd700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a4199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.085+0000 7fae6affd700 1 -- 172.21.15.61:0/2044890354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635a3f08de0 con 0x5635a418b5d0 2023-10-18T09:23:39.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.085+0000 7fae6affd700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae54018670 tx=0x7fae54009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad70ae7ecc09b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:23:39.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.085+0000 7fae4affd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:23:39.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.086+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5402c890 con 0x5635a418b5d0 2023-10-18T09:23:39.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.086+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae64019c50 con 0x5635a418b5d0 2023-10-18T09:23:39.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.086+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae5402c9f0 con 0x5635a418b5d0 2023-10-18T09:23:39.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5402cbb0 con 0x5635a418b5d0 2023-10-18T09:23:39.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 10 client.26176 fetch_fsmap learned FSMap version 37 2023-10-18T09:23:39.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5635a3f2f250 con 0x5635a418b5d0 2023-10-18T09:23:39.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae64019c50 con 0x5635a418b5d0 2023-10-18T09:23:39.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 37) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fae54008050 con 0x5635a418b5d0 2023-10-18T09:23:39.098 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 10 client.26176 fetch_fsmap finished waiting for FSMap version 37 2023-10-18T09:23:39.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 10 client.26176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:23:39.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 20 client.26176 populate_metadata read hostname 'smithi061' 2023-10-18T09:23:39.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a42076f0 0x5635a4209b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:23:39.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 4 client.26176 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:23:39.099 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.095+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635a40d6e50 con 0x5635a42076f0 2023-10-18T09:23:39.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.103+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a42076f0 0x5635a4209b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:23:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.104+0000 7fae4affd700 10 client.26176 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.104+0000 7fae6b7fe700 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a42076f0 0x5635a4209b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdc61c49bbf9879 server_cookie=8adec67023f3065d in_seq=0 out_seq=0 2023-10-18T09:23:39.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.105+0000 7fae4affd700 1 -- 172.21.15.61:0/2044890354 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5635a40d6e50 con 0x5635a42076f0 2023-10-18T09:23:39.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.105+0000 7fae4affd700 10 client.26176 handle_command_reply: tid=0 2023-10-18T09:23:39.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.105+0000 7fae85af7b80 1 client.26176 shutdown 2023-10-18T09:23:39.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.106+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 msgr2=0x5635a4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:39.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.106+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae54018670 tx=0x7fae54009460 comp rx=0 tx=0).stop 2023-10-18T09:23:39.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 shutdown_connections 2023-10-18T09:23:39.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5635a42076f0 0x5635a4209b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5635a4203710 0x5635a4205b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635a418b5d0 0x5635a4198df0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae4affd700 0 client.26176 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:23:39.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 --2- 172.21.15.61:0/2044890354 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5635a418c2d0 0x5635a4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 >> 172.21.15.61:0/2044890354 conn(0x5635a40ff720 msgr2=0x5635a4194b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:39.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 shutdown_connections 2023-10-18T09:23:39.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 1 -- 172.21.15.61:0/2044890354 wait complete. 2023-10-18T09:23:39.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.107+0000 7fae85af7b80 20 client.26176 trim_cache size 0 max 16384 2023-10-18T09:23:39.112 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:23:39.112 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:23:39.112 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:23:39.112 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:23:39.112 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:23:39.113 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:23:39.113 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:23:39.113 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:23:39.113 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:23:39.113 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:23:39.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.108+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 msgr2=0x7fae580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:39.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.108+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 0x7fae580a39c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0cb180 tx=0x7fae5c009c00 comp rx=0 tx=0).stop 2023-10-18T09:23:39.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.108+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 msgr2=0x7fae6c1a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:23:39.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.108+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 0x7fae6c1a42b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00fb70 tx=0x7fae5c00b790 comp rx=0 tx=0).stop 2023-10-18T09:23:39.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 shutdown_connections 2023-10-18T09:23:39.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fae580a1500 0x7fae580a39c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae6c104340 0x7fae6c1a42b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae6c0ff000 0x7fae6c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 --2- 172.21.15.61:0/3276092476 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fae6c101970 0x7fae6c19d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:23:39.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 >> 172.21.15.61:0/3276092476 conn(0x7fae6c0fac00 msgr2=0x7fae6c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:23:39.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 shutdown_connections 2023-10-18T09:23:39.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:23:39.109+0000 7fae721ff700 1 -- 172.21.15.61:0/3276092476 wait complete. 2023-10-18T09:23:39.154 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:23:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 265.0 s, 753.4 tps, lat 132.068 ms stddev 151.224 2023-10-18T09:23:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:39 smithi170 ceph-mon[135150]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-18T09:23:39.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:39 smithi110 ceph-mon[134818]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-18T09:23:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:39 smithi061 ceph-mon[98573]: pgmap v511: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-18T09:23:41.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:41 smithi170 ceph-mon[135150]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:23:41.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:41 smithi110 ceph-mon[134818]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:23:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:41 smithi061 ceph-mon[98573]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 691 op/s 2023-10-18T09:23:43.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:43 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:23:43.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:43 smithi170 ceph-mon[135150]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-18T09:23:43.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:43 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:23:43.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:43 smithi110 ceph-mon[134818]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-18T09:23:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:43 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:23:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:43 smithi061 ceph-mon[98573]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 10 MiB/s wr, 605 op/s 2023-10-18T09:23:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 270.0 s, 829.8 tps, lat 120.438 ms stddev 165.908 2023-10-18T09:23:45.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:45 smithi170 ceph-mon[135150]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-10-18T09:23:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:45 smithi110 ceph-mon[134818]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-10-18T09:23:45.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:45 smithi061 ceph-mon[98573]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 11 MiB/s wr, 660 op/s 2023-10-18T09:23:48.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:47 smithi061 ceph-mon[98573]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:23:48.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:47 smithi170 ceph-mon[135150]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:23:48.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:47 smithi110 ceph-mon[134818]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:23:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 275.0 s, 959.0 tps, lat 104.235 ms stddev 116.822 2023-10-18T09:23:49.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:49 smithi170 ceph-mon[135150]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 593 op/s 2023-10-18T09:23:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:49 smithi110 ceph-mon[134818]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 593 op/s 2023-10-18T09:23:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:49 smithi061 ceph-mon[98573]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.1 MiB/s wr, 593 op/s 2023-10-18T09:23:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:51 smithi170 ceph-mon[135150]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 661 op/s 2023-10-18T09:23:51.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:51 smithi110 ceph-mon[134818]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 661 op/s 2023-10-18T09:23:51.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:51 smithi061 ceph-mon[98573]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 MiB/s wr, 661 op/s 2023-10-18T09:23:53.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:53 smithi170 ceph-mon[135150]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 603 op/s 2023-10-18T09:23:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:53 smithi110 ceph-mon[134818]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 603 op/s 2023-10-18T09:23:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:53 smithi061 ceph-mon[98573]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 603 op/s 2023-10-18T09:23:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 280.0 s, 817.6 tps, lat 121.158 ms stddev 156.861 2023-10-18T09:23:55.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:55 smithi170 ceph-mon[135150]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 627 op/s 2023-10-18T09:23:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:55 smithi110 ceph-mon[134818]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 627 op/s 2023-10-18T09:23:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:55 smithi061 ceph-mon[98573]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 MiB/s wr, 627 op/s 2023-10-18T09:23:58.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:57 smithi170 ceph-mon[135150]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 596 op/s 2023-10-18T09:23:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:57 smithi110 ceph-mon[134818]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 596 op/s 2023-10-18T09:23:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:57 smithi061 ceph-mon[98573]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 MiB/s wr, 596 op/s 2023-10-18T09:23:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 285.0 s, 747.8 tps, lat 136.820 ms stddev 153.822 2023-10-18T09:23:59.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:23:59 smithi170 ceph-mon[135150]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 552 op/s 2023-10-18T09:23:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:23:59 smithi110 ceph-mon[134818]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 552 op/s 2023-10-18T09:23:59.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:23:59 smithi061 ceph-mon[98573]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 MiB/s wr, 552 op/s 2023-10-18T09:24:00.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:00 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:24:00.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:00 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:24:00.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:00 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:24:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:01 smithi110 ceph-mon[134818]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 630 op/s 2023-10-18T09:24:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:01 smithi061 ceph-mon[98573]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 630 op/s 2023-10-18T09:24:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:01 smithi170 ceph-mon[135150]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.9 MiB/s wr, 630 op/s 2023-10-18T09:24:02.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:24:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:24:02.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:24:02.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:24:02.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:24:02.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:24:02.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:24:02.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:24:02.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:24:03.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:03 smithi110 ceph-mon[134818]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-18T09:24:03.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:03 smithi061 ceph-mon[98573]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-18T09:24:03.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:03 smithi170 ceph-mon[135150]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 8.5 MiB/s wr, 567 op/s 2023-10-18T09:24:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 290.0 s, 486.8 tps, lat 162.967 ms stddev 220.084 2023-10-18T09:24:05.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:05 smithi170 ceph-mon[135150]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 611 op/s 2023-10-18T09:24:05.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:05 smithi110 ceph-mon[134818]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 611 op/s 2023-10-18T09:24:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:05 smithi061 ceph-mon[98573]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.9 MiB/s wr, 611 op/s 2023-10-18T09:24:08.151 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:07 smithi170 ceph-mon[135150]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-18T09:24:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:07 smithi110 ceph-mon[134818]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-18T09:24:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:07 smithi061 ceph-mon[98573]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-10-18T09:24:09.155 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:24:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 295.0 s, 655.0 tps, lat 181.027 ms stddev 297.120 2023-10-18T09:24:09.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:09 smithi170 ceph-mon[135150]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-18T09:24:09.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:09 smithi110 ceph-mon[134818]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-18T09:24:09.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:09 smithi061 ceph-mon[98573]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2023-10-18T09:24:09.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.517+0000 7f404f9ff700 1 -- 172.21.15.61:0/4197264524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481007d0 msgr2=0x7f4048100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.517+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4197264524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481007d0 0x7f4048100bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f40440099c0 tx=0x7f4044009cd0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.518+0000 7f404f9ff700 1 -- 172.21.15.61:0/4197264524 shutdown_connections 2023-10-18T09:24:09.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.518+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4197264524 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4048102360 0x7f4048108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.518+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4197264524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481007d0 0x7f4048100bf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.518+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4197264524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481019c0 0x7f4048101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.518+0000 7f404f9ff700 1 -- 172.21.15.61:0/4197264524 >> 172.21.15.61:0/4197264524 conn(0x7f40480fbd40 msgr2=0x7f40480fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.519+0000 7f404f9ff700 1 -- 172.21.15.61:0/4197264524 shutdown_connections 2023-10-18T09:24:09.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.519+0000 7f404f9ff700 1 -- 172.21.15.61:0/4197264524 wait complete. 2023-10-18T09:24:09.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.520+0000 7f404f9ff700 1 Processor -- start 2023-10-18T09:24:09.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.520+0000 7f404f9ff700 1 -- start start 2023-10-18T09:24:09.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481019c0 0x7f4048198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4048102360 0x7f404819ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40481a04f0 con 0x7f4048102360 2023-10-18T09:24:09.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40481a0630 con 0x7f40481019c0 2023-10-18T09:24:09.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404f9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40481a07c0 con 0x7f40481007d0 2023-10-18T09:24:09.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404cf9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481019c0 0x7f4048198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:47196/0 (socket says 172.21.15.61:47196) 2023-10-18T09:24:09.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 -- 172.21.15.61:0/4187507637 learned_addr learned my addr 172.21.15.61:0/4187507637 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:09.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 -- 172.21.15.61:0/4187507637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481019c0 msgr2=0x7f4048198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481019c0 0x7f4048198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.521+0000 7f404d79b700 1 -- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4048102360 msgr2=0x7f404819ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.522+0000 7f404d79b700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4048102360 0x7f404819ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.522+0000 7f404d79b700 1 -- 172.21.15.61:0/4187507637 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440095f0 con 0x7f40481007d0 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.522+0000 7f404d79b700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f404400b580 tx=0x7f4044003dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.527 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.522+0000 7f403e7fc700 1 -- 172.21.15.61:0/4187507637 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404401d070 con 0x7f40481007d0 2023-10-18T09:24:09.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.523+0000 7f403e7fc700 1 -- 172.21.15.61:0/4187507637 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f404400bd10 con 0x7f40481007d0 2023-10-18T09:24:09.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.523+0000 7f403e7fc700 1 -- 172.21.15.61:0/4187507637 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404400fd60 con 0x7f40481007d0 2023-10-18T09:24:09.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.523+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40481a0a40 con 0x7f40481007d0 2023-10-18T09:24:09.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.523+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40481a0e10 con 0x7f40481007d0 2023-10-18T09:24:09.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.527+0000 7f403e7fc700 1 -- 172.21.15.61:0/4187507637 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f404400f460 con 0x7f40481007d0 2023-10-18T09:24:09.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.530+0000 7f403e7fc700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 0x7f40340a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.531+0000 7f403e7fc700 1 -- 172.21.15.61:0/4187507637 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f40440ce0b0 con 0x7f40481007d0 2023-10-18T09:24:09.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.532+0000 7f404cf9a700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 0x7f40340a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.533+0000 7f40632d6b80 1 Processor -- start 2023-10-18T09:24:09.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.533+0000 7f40632d6b80 1 -- start start 2023-10-18T09:24:09.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.533+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e7ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f40632d6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e7536e0 con 0x559b5e7e9d90 2023-10-18T09:24:09.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f40632d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e739c60 con 0x559b5e7e8a90 2023-10-18T09:24:09.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f40632d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e737580 con 0x559b5e7e93d0 2023-10-18T09:24:09.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f404d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f404d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:49742/0 (socket says 172.21.15.61:49742) 2023-10-18T09:24:09.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f404d79b700 1 -- 172.21.15.61:0/2554459078 learned_addr learned my addr 172.21.15.61:0/2554459078 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:09.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.534+0000 7f404cf9a700 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404d79b700 1 -- 172.21.15.61:0/2554459078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 msgr2=0x559b5e7e9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404d79b700 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404d79b700 1 -- 172.21.15.61:0/2554459078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 msgr2=0x559b5e7ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404d79b700 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e7ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404d79b700 1 -- 172.21.15.61:0/2554459078 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5e6f03f0 con 0x559b5e7e8a90 2023-10-18T09:24:09.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.535+0000 7f404cf9a700 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:24:09.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.536+0000 7f404d79b700 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40440975b0 tx=0x7f4044003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af54f01c256e7d21 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.536+0000 7f40337fe700 1 -- 172.21.15.61:0/2554459078 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404409aa10 con 0x559b5e7e8a90 2023-10-18T09:24:09.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.536+0000 7f40337fe700 1 -- 172.21.15.61:0/2554459078 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4044020e50 con 0x559b5e7e8a90 2023-10-18T09:24:09.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.536+0000 7f40337fe700 1 -- 172.21.15.61:0/2554459078 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044099280 con 0x559b5e7e8a90 2023-10-18T09:24:09.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f404cf9a700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 0x7f40340a3a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e20 tx=0x7f4038009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 -- 172.21.15.61:0/2554459078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 msgr2=0x559b5e7e8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40440975b0 tx=0x7f4044003f90 comp rx=0 tx=0).stop 2023-10-18T09:24:09.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 -- 172.21.15.61:0/2554459078 shutdown_connections 2023-10-18T09:24:09.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e7ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e8a90 0x559b5e7e8e90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 --2- 172.21.15.61:0/2554459078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.537+0000 7f40632d6b80 1 -- 172.21.15.61:0/2554459078 >> 172.21.15.61:0/2554459078 conn(0x559b5e754720 msgr2=0x559b5e7dab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.538+0000 7f40632d6b80 1 -- 172.21.15.61:0/2554459078 shutdown_connections 2023-10-18T09:24:09.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.538+0000 7f40632d6b80 1 -- 172.21.15.61:0/2554459078 wait complete. 2023-10-18T09:24:09.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.538+0000 7f40632d6b80 1 -- start start 2023-10-18T09:24:09.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.539+0000 7f40632d6b80 1 Processor -- start 2023-10-18T09:24:09.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.541+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 0x559b5e7e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.541+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 0x559b5e7e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f40632d6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e732e70 con 0x559b5e7e8a90 2023-10-18T09:24:09.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f40632d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e6cf710 con 0x559b5e7e9d90 2023-10-18T09:24:09.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f40632d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e6cf3b0 con 0x559b5e7e93d0 2023-10-18T09:24:09.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f40632d6b80 10 client.0 _ll_register_callbacks cb 0x559b5e619a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:24:09.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f404cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f404cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:47220/0 (socket says 172.21.15.61:47220) 2023-10-18T09:24:09.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f404cf9a700 1 -- 172.21.15.61:0/1382541870 learned_addr learned my addr 172.21.15.61:0/1382541870 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:09.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f404d79b700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 0x559b5e7e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.542+0000 7f404df9c700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 0x559b5e7e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 -- 172.21.15.61:0/1382541870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 msgr2=0x559b5e7e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 -- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 msgr2=0x559b5e7e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 0x559b5e7e6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 -- 172.21.15.61:0/1382541870 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40481a0e10 con 0x559b5e7e8a90 2023-10-18T09:24:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404cf9a700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:24:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404df9c700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 0x559b5e7e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:24:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f404d79b700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 0x559b5e7e1570 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4044096e30 tx=0x7f404400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fad12e338ca1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f4030ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:24:09.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.543+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4044005d40 con 0x559b5e7e8a90 2023-10-18T09:24:09.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4044099260 con 0x559b5e7e8a90 2023-10-18T09:24:09.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b5e737ef0 con 0x559b5e7e8a90 2023-10-18T09:24:09.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40440b92d0 con 0x559b5e7e8a90 2023-10-18T09:24:09.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f40632d6b80 10 client.16056 fetch_fsmap learned FSMap version 38 2023-10-18T09:24:09.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x559b5e55dde0 con 0x559b5e7e8a90 2023-10-18T09:24:09.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b5e737ef0 con 0x559b5e7e8a90 2023-10-18T09:24:09.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.544+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f40440a2f20 con 0x559b5e7e8a90 2023-10-18T09:24:09.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 38 2023-10-18T09:24:09.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:24:09.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 20 client.16056 populate_metadata read hostname 'smithi061' 2023-10-18T09:24:09.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e806010 0x559b5e808430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 4 client.16056 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:24:09.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.545+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b5e7af8b0 con 0x559b5e806010 2023-10-18T09:24:09.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.546+0000 7f404cf9a700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e806010 0x559b5e808430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.546+0000 7f404cf9a700 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e806010 0x559b5e808430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40c465ad38fad2e server_cookie=b9aaacc001befb23 in_seq=0 out_seq=0 2023-10-18T09:24:09.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.546+0000 7f4030ff9700 10 client.16056 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:09.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.548+0000 7f4030ff9700 1 -- 172.21.15.61:0/1382541870 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559b5e7af8b0 con 0x559b5e806010 2023-10-18T09:24:09.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.548+0000 7f4030ff9700 10 client.16056 handle_command_reply: tid=0 2023-10-18T09:24:09.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.548+0000 7f40632d6b80 1 client.16056 shutdown 2023-10-18T09:24:09.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.550+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 msgr2=0x559b5e7e1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.550+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 0x559b5e7e1570 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4044096e30 tx=0x7f404400ff70 comp rx=0 tx=0).stop 2023-10-18T09:24:09.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 shutdown_connections 2023-10-18T09:24:09.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e806010 0x559b5e808430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e9d90 0x559b5e7e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e8a90 0x559b5e7e1570 secure :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4044096e30 tx=0x7f404400ff70 comp rx=0 tx=0).stop 2023-10-18T09:24:09.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 --2- 172.21.15.61:0/1382541870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e93d0 0x559b5e7e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 >> 172.21.15.61:0/1382541870 conn(0x559b5e754720 msgr2=0x559b5e7df080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.553+0000 7f4030ff9700 0 client.16056 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:09.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.556+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 shutdown_connections 2023-10-18T09:24:09.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.556+0000 7f40632d6b80 1 -- 172.21.15.61:0/1382541870 wait complete. 2023-10-18T09:24:09.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.556+0000 7f40632d6b80 20 client.16056 trim_cache size 0 max 16384 2023-10-18T09:24:09.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.599+0000 7f40632d6b80 1 Processor -- start 2023-10-18T09:24:09.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 -- start start 2023-10-18T09:24:09.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e7d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e815c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e5b3920 con 0x559b5e7e9d90 2023-10-18T09:24:09.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e805e20 con 0x559b5e7e93d0 2023-10-18T09:24:09.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f40632d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e7db0c0 con 0x559b5e7e02b0 2023-10-18T09:24:09.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f404df9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f404df9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:47224/0 (socket says 172.21.15.61:47224) 2023-10-18T09:24:09.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f404df9c700 1 -- 172.21.15.61:0/754726971 learned_addr learned my addr 172.21.15.61:0/754726971 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:09.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.600+0000 7f404cf9a700 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e815c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404df9c700 1 -- 172.21.15.61:0/754726971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 msgr2=0x559b5e7d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404df9c700 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e7d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.605 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404df9c700 1 -- 172.21.15.61:0/754726971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 msgr2=0x559b5e815c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404df9c700 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e815c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404df9c700 1 -- 172.21.15.61:0/754726971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5e546c60 con 0x559b5e7e02b0 2023-10-18T09:24:09.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.601+0000 7f404cf9a700 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e815c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:24:09.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.605+0000 7f404df9c700 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4040009d30 tx=0x7f404000b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1f015c5ebb80 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.605+0000 7f4033fff700 1 -- 172.21.15.61:0/754726971 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040010070 con 0x559b5e7e02b0 2023-10-18T09:24:09.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.605+0000 7f4033fff700 1 -- 172.21.15.61:0/754726971 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4040009e80 con 0x559b5e7e02b0 2023-10-18T09:24:09.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.605+0000 7f4033fff700 1 -- 172.21.15.61:0/754726971 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040014580 con 0x559b5e7e02b0 2023-10-18T09:24:09.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.606+0000 7f40632d6b80 1 -- 172.21.15.61:0/754726971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 msgr2=0x559b5e7f0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.606+0000 7f40632d6b80 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4040009d30 tx=0x7f404000b6d0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 -- 172.21.15.61:0/754726971 shutdown_connections 2023-10-18T09:24:09.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e7f0250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e7d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 --2- 172.21.15.61:0/754726971 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e815c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 -- 172.21.15.61:0/754726971 >> 172.21.15.61:0/754726971 conn(0x559b5e754720 msgr2=0x559b5e7ea190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 -- 172.21.15.61:0/754726971 shutdown_connections 2023-10-18T09:24:09.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.607+0000 7f40632d6b80 1 -- 172.21.15.61:0/754726971 wait complete. 2023-10-18T09:24:09.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.608+0000 7f40632d6b80 1 -- start start 2023-10-18T09:24:09.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.610+0000 7f40632d6b80 1 Processor -- start 2023-10-18T09:24:09.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.611+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e8527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.611+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e852d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.611+0000 7f40632d6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e859a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.611+0000 7f40632d6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e859f80 con 0x559b5e7e9d90 2023-10-18T09:24:09.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.611+0000 7f40632d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e85a0c0 con 0x559b5e7e93d0 2023-10-18T09:24:09.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.612+0000 7f40632d6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5e85a280 con 0x559b5e7e02b0 2023-10-18T09:24:09.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.612+0000 7f40632d6b80 10 client.0 _ll_register_callbacks cb 0x559b5e619a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:24:09.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.612+0000 7f404df9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e859a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.612+0000 7f404df9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e859a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:42244/0 (socket says 172.21.15.61:42244) 2023-10-18T09:24:09.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.612+0000 7f404df9c700 1 -- 172.21.15.61:0/3918987008 learned_addr learned my addr 172.21.15.61:0/3918987008 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:09.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.613+0000 7f404cf9a700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e852d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.613+0000 7f404cf9a700 1 -- 172.21.15.61:0/3918987008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 msgr2=0x559b5e8527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.613+0000 7f404cf9a700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e8527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.613+0000 7f404cf9a700 1 -- 172.21.15.61:0/3918987008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 msgr2=0x559b5e859a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:09.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.613+0000 7f404cf9a700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e859a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.617 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.614+0000 7f404cf9a700 1 -- 172.21.15.61:0/3918987008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403801e4a0 con 0x559b5e7e93d0 2023-10-18T09:24:09.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.620+0000 7f404cf9a700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e852d00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559b5e8063f0 tx=0x7f4038004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09d2ce7bb6cd7e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:09.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.620+0000 7f4030ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:24:09.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.622+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038023040 con 0x559b5e7e93d0 2023-10-18T09:24:09.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.622+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f403801ec60 con 0x559b5e7e93d0 2023-10-18T09:24:09.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.623+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b5e72be50 con 0x559b5e7e93d0 2023-10-18T09:24:09.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.623+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4038029e90 con 0x559b5e7e93d0 2023-10-18T09:24:09.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.623+0000 7f40632d6b80 10 client.26222 fetch_fsmap learned FSMap version 38 2023-10-18T09:24:09.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f40400099e0 con 0x559b5e7e93d0 2023-10-18T09:24:09.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b5e72be50 con 0x559b5e7e93d0 2023-10-18T09:24:09.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 38) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f403803c420 con 0x559b5e7e93d0 2023-10-18T09:24:09.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 10 client.26222 fetch_fsmap finished waiting for FSMap version 38 2023-10-18T09:24:09.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:24:09.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 20 client.26222 populate_metadata read hostname 'smithi061' 2023-10-18T09:24:09.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e85bd90 0x559b5e85e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:09.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 4 client.26222 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:24:09.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.625+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b5e72bb70 con 0x559b5e85bd90 2023-10-18T09:24:09.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.626+0000 7f404d79b700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e85bd90 0x559b5e85e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:09.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.629+0000 7f4030ff9700 10 client.26222 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:09.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.629+0000 7f404d79b700 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e85bd90 0x559b5e85e1b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceff0a84581e3c9e server_cookie=8e9aa0d0d8ffcd3c in_seq=0 out_seq=0 2023-10-18T09:24:09.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f4030ff9700 1 -- 172.21.15.61:0/3918987008 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559b5e72bb70 con 0x559b5e85bd90 2023-10-18T09:24:09.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f4030ff9700 10 client.26222 handle_command_reply: tid=0 2023-10-18T09:24:09.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f40632d6b80 1 client.26222 shutdown 2023-10-18T09:24:09.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 msgr2=0x559b5e852d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e852d00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x559b5e8063f0 tx=0x7f4038004750 comp rx=0 tx=0).stop 2023-10-18T09:24:09.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.630+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 shutdown_connections 2023-10-18T09:24:09.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x559b5e85bd90 0x559b5e85e1b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x559b5e7e9d90 0x559b5e859a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f4030ff9700 0 client.26222 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:09.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559b5e7e02b0 0x559b5e8527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 --2- 172.21.15.61:0/3918987008 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559b5e7e93d0 0x559b5e852d00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 >> 172.21.15.61:0/3918987008 conn(0x559b5e754720 msgr2=0x559b5e7df080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 shutdown_connections 2023-10-18T09:24:09.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.631+0000 7f40632d6b80 1 -- 172.21.15.61:0/3918987008 wait complete. 2023-10-18T09:24:09.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.633+0000 7f40632d6b80 20 client.26222 trim_cache size 0 max 16384 2023-10-18T09:24:09.637 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:24:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:24:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:24:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:24:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:24:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:24:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:24:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:24:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:24:09.639 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:24:09.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.634+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 msgr2=0x7f40340a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.634+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 0x7f40340a3a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4038009e20 tx=0x7f4038009ba0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.634+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 msgr2=0x7f40481989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:09.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.634+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f404400b580 tx=0x7f4044003dc0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 shutdown_connections 2023-10-18T09:24:09.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f40340a15d0 0x7f40340a3a90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f4048102360 0x7f404819ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40481007d0 0x7f40481989c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 --2- 172.21.15.61:0/4187507637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40481019c0 0x7f4048198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:09.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 >> 172.21.15.61:0/4187507637 conn(0x7f40480fbd40 msgr2=0x7f4048107160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:09.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 shutdown_connections 2023-10-18T09:24:09.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:09.635+0000 7f404f9ff700 1 -- 172.21.15.61:0/4187507637 wait complete. 2023-10-18T09:24:09.669 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:24:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:11 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:11 smithi170 ceph-mon[135150]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-10-18T09:24:11.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:11 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:11.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:11 smithi110 ceph-mon[134818]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-10-18T09:24:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:11 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:11 smithi061 ceph-mon[98573]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2023-10-18T09:24:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:13 smithi170 ceph-mon[135150]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:13 smithi110 ceph-mon[134818]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:13 smithi061 ceph-mon[98573]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 300.0 s, 776.0 tps, lat 130.690 ms stddev 156.764 2023-10-18T09:24:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:15 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:15 smithi110 ceph-mon[134818]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:15 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:15 smithi061 ceph-mon[98573]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:15.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:15 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:24:15.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:15 smithi170 ceph-mon[135150]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-10-18T09:24:18.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:17 smithi170 ceph-mon[135150]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-18T09:24:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:17 smithi110 ceph-mon[134818]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-18T09:24:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:17 smithi061 ceph-mon[98573]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 17 MiB/s wr, 1.42k op/s 2023-10-18T09:24:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 305.0 s, 791.2 tps, lat 124.366 ms stddev 152.447 2023-10-18T09:24:19.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:19 smithi170 ceph-mon[135150]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-18T09:24:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:19 smithi110 ceph-mon[134818]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-18T09:24:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:19 smithi061 ceph-mon[98573]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 749 op/s 2023-10-18T09:24:21.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:21 smithi170 ceph-mon[135150]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 622 op/s 2023-10-18T09:24:21.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:21 smithi110 ceph-mon[134818]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 622 op/s 2023-10-18T09:24:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:21 smithi061 ceph-mon[98573]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 622 op/s 2023-10-18T09:24:23.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:23 smithi170 ceph-mon[135150]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 559 op/s 2023-10-18T09:24:23.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:23 smithi110 ceph-mon[134818]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 559 op/s 2023-10-18T09:24:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:23 smithi061 ceph-mon[98573]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.9 MiB/s wr, 559 op/s 2023-10-18T09:24:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 310.0 s, 628.4 tps, lat 148.728 ms stddev 164.772 2023-10-18T09:24:25.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:25 smithi170 ceph-mon[135150]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 568 op/s 2023-10-18T09:24:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:25 smithi110 ceph-mon[134818]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 568 op/s 2023-10-18T09:24:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:25 smithi061 ceph-mon[98573]: pgmap v534: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.4 MiB/s wr, 568 op/s 2023-10-18T09:24:28.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:27 smithi170 ceph-mon[135150]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 662 op/s 2023-10-18T09:24:28.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:27 smithi110 ceph-mon[134818]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 662 op/s 2023-10-18T09:24:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:27 smithi061 ceph-mon[98573]: pgmap v535: 129 pgs: 129 active+clean; 1.5 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 662 op/s 2023-10-18T09:24:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 315.0 s, 616.8 tps, lat 174.211 ms stddev 215.108 2023-10-18T09:24:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:29 smithi170 ceph-mon[135150]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2023-10-18T09:24:29.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:29 smithi110 ceph-mon[134818]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2023-10-18T09:24:29.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:29 smithi061 ceph-mon[98573]: pgmap v536: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.4 MiB/s wr, 512 op/s 2023-10-18T09:24:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:31 smithi110 ceph-mon[134818]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-10-18T09:24:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:31 smithi061 ceph-mon[98573]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-10-18T09:24:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:31 smithi170 ceph-mon[135150]: pgmap v537: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.5 MiB/s wr, 562 op/s 2023-10-18T09:24:33.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:33 smithi170 ceph-mon[135150]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2023-10-18T09:24:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:33 smithi110 ceph-mon[134818]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2023-10-18T09:24:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:33 smithi061 ceph-mon[98573]: pgmap v538: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 506 op/s 2023-10-18T09:24:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 320.0 s, 747.8 tps, lat 135.161 ms stddev 154.902 2023-10-18T09:24:35.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:35 smithi170 ceph-mon[135150]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2023-10-18T09:24:35.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:35 smithi110 ceph-mon[134818]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2023-10-18T09:24:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:35 smithi061 ceph-mon[98573]: pgmap v539: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 6.9 MiB/s wr, 501 op/s 2023-10-18T09:24:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:37 smithi170 ceph-mon[135150]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 552 op/s 2023-10-18T09:24:38.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:37 smithi110 ceph-mon[134818]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 552 op/s 2023-10-18T09:24:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:37 smithi061 ceph-mon[98573]: pgmap v540: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 552 op/s 2023-10-18T09:24:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 325.0 s, 898.2 tps, lat 110.191 ms stddev 134.879 2023-10-18T09:24:39.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:39 smithi170 ceph-mon[135150]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 504 op/s 2023-10-18T09:24:39.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:39 smithi110 ceph-mon[134818]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 504 op/s 2023-10-18T09:24:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:39 smithi061 ceph-mon[98573]: pgmap v541: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.0 MiB/s wr, 504 op/s 2023-10-18T09:24:39.670 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:24:40.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.028+0000 7f921ffff700 1 -- 172.21.15.61:0/2414686330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9218101220 msgr2=0x7f9218107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.028+0000 7f921ffff700 1 --2- 172.21.15.61:0/2414686330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9218101220 0x7f9218107ad0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f921000b0d0 tx=0x7f921000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.031+0000 7f921ffff700 1 -- 172.21.15.61:0/2414686330 shutdown_connections 2023-10-18T09:24:40.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.031+0000 7f921ffff700 1 --2- 172.21.15.61:0/2414686330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9218101220 0x7f9218107ad0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.031+0000 7f921ffff700 1 --2- 172.21.15.61:0/2414686330 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f92180ff690 0x7f92180ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.031+0000 7f921ffff700 1 --2- 172.21.15.61:0/2414686330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 0x7f9218100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.031+0000 7f921ffff700 1 -- 172.21.15.61:0/2414686330 >> 172.21.15.61:0/2414686330 conn(0x7f92180fac00 msgr2=0x7f92180fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.032+0000 7f921ffff700 1 -- 172.21.15.61:0/2414686330 shutdown_connections 2023-10-18T09:24:40.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.032+0000 7f921ffff700 1 -- 172.21.15.61:0/2414686330 wait complete. 2023-10-18T09:24:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 Processor -- start 2023-10-18T09:24:40.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 -- start start 2023-10-18T09:24:40.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 0x7f9218198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9218101220 0x7f921819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a05c0 con 0x7f9218101220 2023-10-18T09:24:40.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a0700 con 0x7f92180ff690 2023-10-18T09:24:40.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.034+0000 7f921ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92181a0890 con 0x7f9218100880 2023-10-18T09:24:40.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:33576/0 (socket says 172.21.15.61:33576) 2023-10-18T09:24:40.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 -- 172.21.15.61:0/1816686119 learned_addr learned my addr 172.21.15.61:0/1816686119 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:40.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921d59a700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 0x7f9218198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921e59c700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9218101220 0x7f921819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 -- 172.21.15.61:0/1816686119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 msgr2=0x7f9218198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 0x7f9218198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 -- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9218101220 msgr2=0x7f921819fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9218101220 0x7f921819fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 -- 172.21.15.61:0/1816686119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210009d00 con 0x7f92180ff690 2023-10-18T09:24:40.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.035+0000 7f921dd9b700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f921400b3c0 tx=0x7f9214009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.036+0000 7f920effd700 1 -- 172.21.15.61:0/1816686119 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214018070 con 0x7f92180ff690 2023-10-18T09:24:40.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.036+0000 7f920effd700 1 -- 172.21.15.61:0/1816686119 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92140044e0 con 0x7f92180ff690 2023-10-18T09:24:40.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.036+0000 7f920effd700 1 -- 172.21.15.61:0/1816686119 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214010040 con 0x7f92180ff690 2023-10-18T09:24:40.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.036+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9218101bc0 con 0x7f92180ff690 2023-10-18T09:24:40.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.036+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9218102080 con 0x7f92180ff690 2023-10-18T09:24:40.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.041+0000 7f920effd700 1 -- 172.21.15.61:0/1816686119 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f92140038a0 con 0x7f92180ff690 2023-10-18T09:24:40.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.041+0000 7f923394fb80 1 Processor -- start 2023-10-18T09:24:40.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 -- start start 2023-10-18T09:24:40.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca3890a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca38036e0 con 0x558ca38912d0 2023-10-18T09:24:40.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca37e9c60 con 0x558ca3891c30 2023-10-18T09:24:40.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f923394fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca37e7580 con 0x558ca38905d0 2023-10-18T09:24:40.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f921dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f921d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca3890a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f921d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca3890a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:52222/0 (socket says 172.21.15.61:52222) 2023-10-18T09:24:40.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f921d59a700 1 -- 172.21.15.61:0/2953555297 learned_addr learned my addr 172.21.15.61:0/2953555297 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:40.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.042+0000 7f921dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:33586/0 (socket says 172.21.15.61:33586) 2023-10-18T09:24:40.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921e59c700 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 -- 172.21.15.61:0/2953555297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 msgr2=0x558ca3890a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca3890a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 -- 172.21.15.61:0/2953555297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 msgr2=0x558ca38916f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38916f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 -- 172.21.15.61:0/2953555297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca37a03f0 con 0x558ca3891c30 2023-10-18T09:24:40.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.043+0000 7f921dd9b700 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f92140c9dc0 tx=0x7f92140d2d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d111d9801178c5f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f920effd700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 0x7f92040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f9203fff700 1 -- 172.21.15.61:0/2953555297 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140d4800 con 0x558ca3891c30 2023-10-18T09:24:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f9203fff700 1 -- 172.21.15.61:0/2953555297 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92140d1030 con 0x558ca3891c30 2023-10-18T09:24:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f9203fff700 1 -- 172.21.15.61:0/2953555297 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214098f70 con 0x558ca3891c30 2023-10-18T09:24:40.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f920effd700 1 -- 172.21.15.61:0/1816686119 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f92140cc640 con 0x7f92180ff690 2023-10-18T09:24:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.044+0000 7f921e59c700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 0x7f92040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 -- 172.21.15.61:0/2953555297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 msgr2=0x558ca3890090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f92140c9dc0 tx=0x7f92140d2d60 comp rx=0 tx=0).stop 2023-10-18T09:24:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 -- 172.21.15.61:0/2953555297 shutdown_connections 2023-10-18T09:24:40.051 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca3890a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 --2- 172.21.15.61:0/2953555297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca3890090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 -- 172.21.15.61:0/2953555297 >> 172.21.15.61:0/2953555297 conn(0x558ca3804720 msgr2=0x558ca388ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.045+0000 7f923394fb80 1 -- 172.21.15.61:0/2953555297 shutdown_connections 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.046+0000 7f921e59c700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 0x7f92040a39c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.046+0000 7f923394fb80 1 -- 172.21.15.61:0/2953555297 wait complete. 2023-10-18T09:24:40.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.046+0000 7f923394fb80 1 -- start start 2023-10-18T09:24:40.053 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.047+0000 7f923394fb80 1 Processor -- start 2023-10-18T09:24:40.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f923394fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f923394fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38912d0 0x558ca3899fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f923394fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.055 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f923394fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca37e2e70 con 0x558ca38905d0 2023-10-18T09:24:40.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f921e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f921e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:33600/0 (socket says 172.21.15.61:33600) 2023-10-18T09:24:40.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f921e59c700 1 -- 172.21.15.61:0/498339788 learned_addr learned my addr 172.21.15.61:0/498339788 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:40.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f923394fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca377f710 con 0x558ca3891c30 2023-10-18T09:24:40.056 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.049+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca377f3b0 con 0x558ca38912d0 2023-10-18T09:24:40.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.048+0000 7f921d59a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38912d0 0x558ca3899fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.049+0000 7f921dd9b700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.049+0000 7f923394fb80 10 client.0 _ll_register_callbacks cb 0x558ca36c9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:24:40.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 -- 172.21.15.61:0/498339788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38912d0 msgr2=0x558ca3899fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.057 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38912d0 0x558ca3899fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 -- 172.21.15.61:0/498339788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38905d0 msgr2=0x558ca3899a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 -- 172.21.15.61:0/498339788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9218102080 con 0x558ca3891c30 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f921e59c700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f921000bd20 tx=0x7f921000b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e58d28bb16a934a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.050+0000 7f92017fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:24:40.058 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000e040 con 0x558ca3891c30 2023-10-18T09:24:40.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f921000bd50 con 0x558ca3891c30 2023-10-18T09:24:40.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ca37e7ef0 con 0x558ca3891c30 2023-10-18T09:24:40.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92100092e0 con 0x558ca3891c30 2023-10-18T09:24:40.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f923394fb80 10 client.26234 fetch_fsmap learned FSMap version 40 2023-10-18T09:24:40.059 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.051+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558ca360dde0 con 0x558ca3891c30 2023-10-18T09:24:40.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9210036030 con 0x558ca3891c30 2023-10-18T09:24:40.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9210029a00 con 0x558ca3891c30 2023-10-18T09:24:40.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 10 client.26234 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:24:40.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 10 client.26234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:24:40.060 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 20 client.26234 populate_metadata read hostname 'smithi061' 2023-10-18T09:24:40.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38a5940 0x558ca38a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 4 client.26234 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:24:40.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.052+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ca385f8b0 con 0x558ca38a5940 2023-10-18T09:24:40.061 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.053+0000 7f921dd9b700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38a5940 0x558ca38a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.054+0000 7f92017fa700 10 client.26234 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:40.062 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.054+0000 7f921dd9b700 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38a5940 0x558ca38a7d60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b25322b416cce823 server_cookie=58c934611ff86a5b in_seq=0 out_seq=0 2023-10-18T09:24:40.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f92017fa700 1 -- 172.21.15.61:0/498339788 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558ca385f8b0 con 0x558ca38a5940 2023-10-18T09:24:40.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f92017fa700 10 client.26234 handle_command_reply: tid=0 2023-10-18T09:24:40.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 client.26234 shutdown 2023-10-18T09:24:40.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 msgr2=0x558ca38a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.063 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f921000bd20 tx=0x7f921000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 shutdown_connections 2023-10-18T09:24:40.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38a5940 0x558ca38a7d60 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca38a0fe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.064 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 --2- 172.21.15.61:0/498339788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38912d0 0x558ca3899fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 >> 172.21.15.61:0/498339788 conn(0x558ca3804720 msgr2=0x558ca388f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f92017fa700 0 client.26234 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:40.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.055+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 shutdown_connections 2023-10-18T09:24:40.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.056+0000 7f923394fb80 1 -- 172.21.15.61:0/498339788 wait complete. 2023-10-18T09:24:40.066 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.061+0000 7f923394fb80 20 client.26234 trim_cache size 0 max 16384 2023-10-18T09:24:40.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.109+0000 7f923394fb80 1 Processor -- start 2023-10-18T09:24:40.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.109+0000 7f923394fb80 1 -- start start 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.109+0000 7f923394fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f923394fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f923394fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 0x558ca3899fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f923394fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca3663920 con 0x558ca38912d0 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f923394fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca3887550 con 0x558ca38905d0 2023-10-18T09:24:40.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f923394fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca3898b10 con 0x558ca3891c30 2023-10-18T09:24:40.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f921e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 0x558ca3899fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f921e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 0x558ca3899fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:52242/0 (socket says 172.21.15.61:52242) 2023-10-18T09:24:40.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.110+0000 7f921e59c700 1 -- 172.21.15.61:0/3308655160 learned_addr learned my addr 172.21.15.61:0/3308655160 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:40.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.112+0000 7f921dd9b700 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.112+0000 7f921d59a700 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 -- 172.21.15.61:0/3308655160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 msgr2=0x558ca3899fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 0x558ca3899fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 -- 172.21.15.61:0/3308655160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca38905d0 msgr2=0x558ca3899a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 -- 172.21.15.61:0/3308655160 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ca35f6c60 con 0x558ca38912d0 2023-10-18T09:24:40.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.113+0000 7f921d59a700 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38a0fa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9208005c50 tx=0x7f9208009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab26de725d4bca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.114+0000 7f920cff9700 1 -- 172.21.15.61:0/3308655160 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208017050 con 0x558ca38912d0 2023-10-18T09:24:40.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.115+0000 7f920cff9700 1 -- 172.21.15.61:0/3308655160 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9208018040 con 0x558ca38912d0 2023-10-18T09:24:40.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.115+0000 7f920cff9700 1 -- 172.21.15.61:0/3308655160 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208010e40 con 0x558ca38912d0 2023-10-18T09:24:40.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.116+0000 7f923394fb80 1 -- 172.21.15.61:0/3308655160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 msgr2=0x558ca38a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.116+0000 7f923394fb80 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38a0fa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9208005c50 tx=0x7f9208009260 comp rx=0 tx=0).stop 2023-10-18T09:24:40.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.117+0000 7f923394fb80 1 -- 172.21.15.61:0/3308655160 shutdown_connections 2023-10-18T09:24:40.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.117+0000 7f923394fb80 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca3891c30 0x558ca3899fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.117+0000 7f923394fb80 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca38905d0 0x558ca3899a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.117+0000 7f923394fb80 1 --2- 172.21.15.61:0/3308655160 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca38a0fa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.117+0000 7f923394fb80 1 -- 172.21.15.61:0/3308655160 >> 172.21.15.61:0/3308655160 conn(0x558ca3804720 msgr2=0x558ca3898cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.119+0000 7f923394fb80 1 -- 172.21.15.61:0/3308655160 shutdown_connections 2023-10-18T09:24:40.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.120+0000 7f923394fb80 1 -- 172.21.15.61:0/3308655160 wait complete. 2023-10-18T09:24:40.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.121+0000 7f923394fb80 1 -- start start 2023-10-18T09:24:40.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.121+0000 7f923394fb80 1 Processor -- start 2023-10-18T09:24:40.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.122+0000 7f923394fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca38a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.122+0000 7f923394fb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca389f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.122+0000 7f923394fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.122+0000 7f923394fb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca38a0400 con 0x558ca38912d0 2023-10-18T09:24:40.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:33604/0 (socket says 172.21.15.61:33604) 2023-10-18T09:24:40.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 -- 172.21.15.61:0/2172501384 learned_addr learned my addr 172.21.15.61:0/2172501384 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:24:40.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca38f9720 con 0x558ca3891c30 2023-10-18T09:24:40.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ca38f9860 con 0x558ca38905d0 2023-10-18T09:24:40.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921d59a700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca389f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f923394fb80 10 client.0 _ll_register_callbacks cb 0x558ca36c9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:24:40.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 -- 172.21.15.61:0/2172501384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 msgr2=0x558ca38a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca38a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 -- 172.21.15.61:0/2172501384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 msgr2=0x558ca389f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:24:40.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca389f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.123+0000 7f921e59c700 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9208009b20 con 0x558ca3891c30 2023-10-18T09:24:40.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.124+0000 7f921e59c700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f921001c720 tx=0x7f9210006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfc1ef41ab80c0f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:24:40.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.124+0000 7f92017fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:24:40.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.127+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000e040 con 0x558ca3891c30 2023-10-18T09:24:40.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.127+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9210029640 con 0x558ca3891c30 2023-10-18T09:24:40.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.128+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92140d1030 con 0x558ca3891c30 2023-10-18T09:24:40.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.130+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9210029800 con 0x558ca3891c30 2023-10-18T09:24:40.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.130+0000 7f923394fb80 10 client.26237 fetch_fsmap learned FSMap version 40 2023-10-18T09:24:40.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.130+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558ca3634250 con 0x558ca3891c30 2023-10-18T09:24:40.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92140d1030 con 0x558ca3891c30 2023-10-18T09:24:40.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f9210046400 con 0x558ca3891c30 2023-10-18T09:24:40.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 10 client.26237 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:24:40.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:24:40.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 20 client.26237 populate_metadata read hostname 'smithi061' 2023-10-18T09:24:40.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38fad90 0x558ca38fd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:24:40.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 4 client.26237 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:24:40.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.131+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ca37dbe50 con 0x558ca38fad90 2023-10-18T09:24:40.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.132+0000 7f921dd9b700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38fad90 0x558ca38fd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:24:40.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.133+0000 7f92017fa700 10 client.26237 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:40.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.133+0000 7f921dd9b700 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38fad90 0x558ca38fd1d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c124ba9fa03422e server_cookie=a3f22e54d1b95748 in_seq=0 out_seq=0 2023-10-18T09:24:40.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.133+0000 7f92017fa700 1 -- 172.21.15.61:0/2172501384 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558ca37dbe50 con 0x558ca38fad90 2023-10-18T09:24:40.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.133+0000 7f92017fa700 10 client.26237 handle_command_reply: tid=0 2023-10-18T09:24:40.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.133+0000 7f923394fb80 1 client.26237 shutdown 2023-10-18T09:24:40.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.134+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 msgr2=0x558ca389fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f921001c720 tx=0x7f9210006230 comp rx=0 tx=0).stop 2023-10-18T09:24:40.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 shutdown_connections 2023-10-18T09:24:40.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558ca38fad90 0x558ca38fd1d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558ca3891c30 0x558ca389fcc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558ca38905d0 0x558ca38a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 --2- 172.21.15.61:0/2172501384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558ca38912d0 0x558ca389f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f92017fa700 0 client.26237 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:24:40.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 >> 172.21.15.61:0/2172501384 conn(0x558ca3804720 msgr2=0x558ca38942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.137+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 shutdown_connections 2023-10-18T09:24:40.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.139+0000 7f923394fb80 1 -- 172.21.15.61:0/2172501384 wait complete. 2023-10-18T09:24:40.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.143+0000 7f923394fb80 20 client.26237 trim_cache size 0 max 16384 2023-10-18T09:24:40.146 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:24:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:24:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:24:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:24:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:24:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:24:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:24:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:24:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:24:40.148 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:24:40.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 msgr2=0x7f92040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 0x7f92040a39c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000b040 comp rx=0 tx=0).stop 2023-10-18T09:24:40.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 msgr2=0x7f9218198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:24:40.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f921400b3c0 tx=0x7f9214009ab0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 shutdown_connections 2023-10-18T09:24:40.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f92040a1500 0x7f92040a39c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f9218101220 0x7f921819fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92180ff690 0x7f9218198a90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 --2- 172.21.15.61:0/1816686119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9218100880 0x7f9218198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:24:40.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 >> 172.21.15.61:0/1816686119 conn(0x7f92180fac00 msgr2=0x7f921818f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:24:40.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 shutdown_connections 2023-10-18T09:24:40.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:24:40.144+0000 7f921ffff700 1 -- 172.21.15.61:0/1816686119 wait complete. 2023-10-18T09:24:40.179 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:24:41.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:41 smithi170 ceph-mon[135150]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2023-10-18T09:24:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:41 smithi110 ceph-mon[134818]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2023-10-18T09:24:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:41 smithi061 ceph-mon[98573]: pgmap v542: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 MiB/s wr, 573 op/s 2023-10-18T09:24:43.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:43 smithi170 ceph-mon[135150]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 514 op/s 2023-10-18T09:24:43.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:43 smithi110 ceph-mon[134818]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 514 op/s 2023-10-18T09:24:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:43 smithi061 ceph-mon[98573]: pgmap v543: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.6 MiB/s wr, 514 op/s 2023-10-18T09:24:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 330.0 s, 697.6 tps, lat 142.234 ms stddev 167.411 2023-10-18T09:24:45.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:45 smithi170 ceph-mon[135150]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 529 op/s 2023-10-18T09:24:45.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:45 smithi110 ceph-mon[134818]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 529 op/s 2023-10-18T09:24:45.509 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:45 smithi061 ceph-mon[98573]: pgmap v544: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 529 op/s 2023-10-18T09:24:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:47 smithi170 ceph-mon[135150]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 595 op/s 2023-10-18T09:24:48.213 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:47 smithi110 ceph-mon[134818]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 595 op/s 2023-10-18T09:24:48.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:47 smithi061 ceph-mon[98573]: pgmap v545: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.6 MiB/s wr, 595 op/s 2023-10-18T09:24:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 335.0 s, 705.0 tps, lat 142.666 ms stddev 158.717 2023-10-18T09:24:49.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:49 smithi170 ceph-mon[135150]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-18T09:24:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:49 smithi110 ceph-mon[134818]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-18T09:24:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:49 smithi061 ceph-mon[98573]: pgmap v546: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2023-10-18T09:24:51.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:51 smithi170 ceph-mon[135150]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 592 op/s 2023-10-18T09:24:51.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:51 smithi110 ceph-mon[134818]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 592 op/s 2023-10-18T09:24:51.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:51 smithi061 ceph-mon[98573]: pgmap v547: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 592 op/s 2023-10-18T09:24:53.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:53 smithi110 ceph-mon[134818]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 521 op/s 2023-10-18T09:24:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:53 smithi061 ceph-mon[98573]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 521 op/s 2023-10-18T09:24:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:53 smithi170 ceph-mon[135150]: pgmap v548: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 MiB/s wr, 521 op/s 2023-10-18T09:24:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 340.0 s, 474.0 tps, lat 206.597 ms stddev 215.347 2023-10-18T09:24:55.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:55 smithi170 ceph-mon[135150]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 MiB/s wr, 535 op/s 2023-10-18T09:24:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:55 smithi110 ceph-mon[134818]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 MiB/s wr, 535 op/s 2023-10-18T09:24:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:55 smithi061 ceph-mon[98573]: pgmap v549: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.4 MiB/s wr, 535 op/s 2023-10-18T09:24:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:57 smithi170 ceph-mon[135150]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 593 op/s 2023-10-18T09:24:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:57 smithi110 ceph-mon[134818]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 593 op/s 2023-10-18T09:24:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:57 smithi061 ceph-mon[98573]: pgmap v550: 129 pgs: 129 active+clean; 1.5 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 MiB/s wr, 593 op/s 2023-10-18T09:24:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 345.0 s, 646.6 tps, lat 155.003 ms stddev 188.182 2023-10-18T09:24:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:24:59 smithi110 ceph-mon[134818]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 MiB/s wr, 583 op/s 2023-10-18T09:24:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:24:59 smithi061 ceph-mon[98573]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 MiB/s wr, 583 op/s 2023-10-18T09:24:59.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:24:59 smithi170 ceph-mon[135150]: pgmap v551: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 7.2 MiB/s wr, 583 op/s 2023-10-18T09:25:01.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:01 smithi170 ceph-mon[135150]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 613 op/s 2023-10-18T09:25:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:01 smithi110 ceph-mon[134818]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 613 op/s 2023-10-18T09:25:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:01 smithi061 ceph-mon[98573]: pgmap v552: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 MiB/s wr, 613 op/s 2023-10-18T09:25:02.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:02 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:25:02.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:02 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:25:02.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:02 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:25:03.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:03 smithi110 ceph-mon[134818]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-10-18T09:25:03.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:03 smithi061 ceph-mon[98573]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-10-18T09:25:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:03 smithi170 ceph-mon[135150]: pgmap v553: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.8 MiB/s wr, 551 op/s 2023-10-18T09:25:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 350.0 s, 718.2 tps, lat 141.197 ms stddev 167.534 2023-10-18T09:25:04.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:25:04.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:25:04.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:25:04.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:25:04.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:25:04.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:25:04.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:25:04.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:25:04.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:25:04.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:25:04.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:25:04.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:25:04.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:25:04.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:25:04.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:25:04.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:25:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:05 smithi110 ceph-mon[134818]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 550 op/s 2023-10-18T09:25:05.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:05 smithi061 ceph-mon[98573]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 550 op/s 2023-10-18T09:25:05.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:05 smithi170 ceph-mon[135150]: pgmap v554: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.5 MiB/s wr, 550 op/s 2023-10-18T09:25:08.146 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:07 smithi170 ceph-mon[135150]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2023-10-18T09:25:08.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:07 smithi110 ceph-mon[134818]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2023-10-18T09:25:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:07 smithi061 ceph-mon[98573]: pgmap v555: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.7 MiB/s wr, 594 op/s 2023-10-18T09:25:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 355.0 s, 810.6 tps, lat 124.344 ms stddev 141.036 2023-10-18T09:25:09.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:09 smithi110 ceph-mon[134818]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 575 op/s 2023-10-18T09:25:09.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:09 smithi061 ceph-mon[98573]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 575 op/s 2023-10-18T09:25:09.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:09 smithi170 ceph-mon[135150]: pgmap v556: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 575 op/s 2023-10-18T09:25:10.180 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:25:10.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.571+0000 7f54ce59d700 1 -- 172.21.15.61:0/1924533140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c80ff800 msgr2=0x7f54c80ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.571+0000 7f54ce59d700 1 --2- 172.21.15.61:0/1924533140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c80ff800 0x7f54c80ffc80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54c40098f0 tx=0x7f54c4009c00 comp rx=0 tx=0).stop 2023-10-18T09:25:10.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 -- 172.21.15.61:0/1924533140 shutdown_connections 2023-10-18T09:25:10.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 --2- 172.21.15.61:0/1924533140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54c81001c0 0x7f54c8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 --2- 172.21.15.61:0/1924533140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c80ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 --2- 172.21.15.61:0/1924533140 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c80ff800 0x7f54c80ffc80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 -- 172.21.15.61:0/1924533140 >> 172.21.15.61:0/1924533140 conn(0x7f54c80faa80 msgr2=0x7f54c80fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 -- 172.21.15.61:0/1924533140 shutdown_connections 2023-10-18T09:25:10.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.572+0000 7f54ce59d700 1 -- 172.21.15.61:0/1924533140 wait complete. 2023-10-18T09:25:10.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.576+0000 7f54ce59d700 1 Processor -- start 2023-10-18T09:25:10.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.576+0000 7f54ce59d700 1 -- start start 2023-10-18T09:25:10.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54c80ff800 0x7f54c810fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c81001c0 0x7f54c81acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81ad030 con 0x7f54c81001c0 2023-10-18T09:25:10.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81ad170 con 0x7f54c80feea0 2023-10-18T09:25:10.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54ce59d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c81ad2b0 con 0x7f54c80ff800 2023-10-18T09:25:10.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.577+0000 7f54cd59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:52228/0 (socket says 172.21.15.61:52228) 2023-10-18T09:25:10.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 -- 172.21.15.61:0/3750031475 learned_addr learned my addr 172.21.15.61:0/3750031475 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:10.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 -- 172.21.15.61:0/3750031475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54c80ff800 msgr2=0x7f54c810fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54c80ff800 0x7f54c810fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 -- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c81001c0 msgr2=0x7f54c81acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c81001c0 0x7f54c81acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 -- 172.21.15.61:0/3750031475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0095f0 con 0x7f54c80feea0 2023-10-18T09:25:10.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.578+0000 7f54cd59b700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007f40 tx=0x7f54c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.579+0000 7f54ba7fc700 1 -- 172.21.15.61:0/3750031475 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c000de60 con 0x7f54c80feea0 2023-10-18T09:25:10.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.579+0000 7f54ba7fc700 1 -- 172.21.15.61:0/3750031475 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54c000f040 con 0x7f54c80feea0 2023-10-18T09:25:10.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.579+0000 7f54ba7fc700 1 -- 172.21.15.61:0/3750031475 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c0015400 con 0x7f54c80feea0 2023-10-18T09:25:10.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.579+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c4009520 con 0x7f54c80feea0 2023-10-18T09:25:10.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.579+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c81ad810 con 0x7f54c80feea0 2023-10-18T09:25:10.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.587+0000 7f54ba7fc700 1 -- 172.21.15.61:0/3750031475 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f54c0003a00 con 0x7f54c80feea0 2023-10-18T09:25:10.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.590+0000 7f54e3059b80 1 Processor -- start 2023-10-18T09:25:10.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.590+0000 7f54e3059b80 1 -- start start 2023-10-18T09:25:10.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.591+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.592+0000 7f54ba7fc700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 0x7f54b40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.592+0000 7f54ba7fc700 1 -- 172.21.15.61:0/3750031475 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f54c00cca40 con 0x7f54c80feea0 2023-10-18T09:25:10.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.593+0000 7f54cd59b700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 0x7f54b40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:52242/0 (socket says 172.21.15.61:52242) 2023-10-18T09:25:10.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d0c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.597 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54e3059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0c7e6e0 con 0x561ca0d0c7f0 2023-10-18T09:25:10.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54e3059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0c64c60 con 0x561ca0d0b4d0 2023-10-18T09:25:10.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54e3059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0c62580 con 0x561ca0d0be30 2023-10-18T09:25:10.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.594+0000 7f54cdd9c700 1 -- 172.21.15.61:0/566664885 learned_addr learned my addr 172.21.15.61:0/566664885 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:10.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d0c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 -- 172.21.15.61:0/566664885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 msgr2=0x561ca0d0c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d0c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 -- 172.21.15.61:0/566664885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 msgr2=0x561ca0d0cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 -- 172.21.15.61:0/566664885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ca0c1b3f0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d0c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:25:10.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.595+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fb0 tx=0x7f54bc010650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1a93e68d5df3d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.596+0000 7f54b37fe700 1 -- 172.21.15.61:0/566664885 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004030 con 0x561ca0d0b4d0 2023-10-18T09:25:10.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.596+0000 7f54b37fe700 1 -- 172.21.15.61:0/566664885 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc003680 con 0x561ca0d0b4d0 2023-10-18T09:25:10.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.596+0000 7f54b37fe700 1 -- 172.21.15.61:0/566664885 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc0126e0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 -- 172.21.15.61:0/566664885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 msgr2=0x561ca0d0b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.601 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009fb0 tx=0x7f54bc010650 comp rx=0 tx=0).stop 2023-10-18T09:25:10.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 -- 172.21.15.61:0/566664885 shutdown_connections 2023-10-18T09:25:10.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d0cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.602 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0b4d0 0x561ca0d0b8f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 --2- 172.21.15.61:0/566664885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d0c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54e3059b80 1 -- 172.21.15.61:0/566664885 >> 172.21.15.61:0/566664885 conn(0x561ca0c7f720 msgr2=0x561ca0d05d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.597+0000 7f54cd59b700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 0x7f54b40a39c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f54c00999a0 tx=0x7f54c0084040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.603 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.598+0000 7f54e3059b80 1 -- 172.21.15.61:0/566664885 shutdown_connections 2023-10-18T09:25:10.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.598+0000 7f54e3059b80 1 -- 172.21.15.61:0/566664885 wait complete. 2023-10-18T09:25:10.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.598+0000 7f54e3059b80 1 -- start start 2023-10-18T09:25:10.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.599+0000 7f54e3059b80 1 Processor -- start 2023-10-18T09:25:10.604 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.600+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.600+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.606 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54e3059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0c5de70 con 0x561ca0d0b4d0 2023-10-18T09:25:10.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54e3059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0bfa710 con 0x561ca0d0c7f0 2023-10-18T09:25:10.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54e3059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0bfa3b0 con 0x561ca0d0be30 2023-10-18T09:25:10.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54e3059b80 10 client.0 _ll_register_callbacks cb 0x561ca0b44a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:25:10.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.607 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49580/0 (socket says 172.21.15.61:49580) 2023-10-18T09:25:10.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54cdd9c700 1 -- 172.21.15.61:0/3212261470 learned_addr learned my addr 172.21.15.61:0/3212261470 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:10.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.601+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 -- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 msgr2=0x561ca0d1c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.608 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 -- 172.21.15.61:0/3212261470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 msgr2=0x561ca0d14d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 -- 172.21.15.61:0/3212261470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc009c60 con 0x561ca0d0be30 2023-10-18T09:25:10.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.602+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d15280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009230 tx=0x7f54c400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58d1c13902bd5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.609 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:25:10.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c401c070 con 0x561ca0d0be30 2023-10-18T09:25:10.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ca0c62ef0 con 0x561ca0d0be30 2023-10-18T09:25:10.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f54c400ba10 con 0x561ca0d0be30 2023-10-18T09:25:10.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c400bbd0 con 0x561ca0d0be30 2023-10-18T09:25:10.610 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54e3059b80 10 client.26209 fetch_fsmap learned FSMap version 40 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.603+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ca0a88de0 con 0x561ca0d0be30 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561ca0c62ef0 con 0x561ca0d0be30 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f54c4015020 con 0x561ca0d0be30 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:25:10.611 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 20 client.26209 populate_metadata read hostname 'smithi061' 2023-10-18T09:25:10.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d20b40 0x561ca0d22f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 4 client.26209 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:25:10.612 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ca0cda8b0 con 0x561ca0d20b40 2023-10-18T09:25:10.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.604+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d20b40 0x561ca0d22f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.613 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.605+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d20b40 0x561ca0d22f60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846c1d821a5c60f4 server_cookie=7c912365c7133581 in_seq=0 out_seq=0 2023-10-18T09:25:10.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.605+0000 7f54b0ff9700 10 client.26209 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:10.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.606+0000 7f54b0ff9700 1 -- 172.21.15.61:0/3212261470 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561ca0cda8b0 con 0x561ca0d20b40 2023-10-18T09:25:10.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.606+0000 7f54b0ff9700 10 client.26209 handle_command_reply: tid=0 2023-10-18T09:25:10.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.608+0000 7f54e3059b80 1 client.26209 shutdown 2023-10-18T09:25:10.614 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.608+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 msgr2=0x561ca0d15280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.608+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d15280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009230 tx=0x7f54c400b790 comp rx=0 tx=0).stop 2023-10-18T09:25:10.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 shutdown_connections 2023-10-18T09:25:10.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d20b40 0x561ca0d22f60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.615 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 --2- 172.21.15.61:0/3212261470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0be30 0x561ca0d15280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 >> 172.21.15.61:0/3212261470 conn(0x561ca0c7f720 msgr2=0x561ca0d0a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54b0ff9700 0 client.26209 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.609+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 shutdown_connections 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.610+0000 7f54e3059b80 1 -- 172.21.15.61:0/3212261470 wait complete. 2023-10-18T09:25:10.616 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.610+0000 7f54e3059b80 20 client.26209 trim_cache size 0 max 16384 2023-10-18T09:25:10.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.631+0000 7f54e3059b80 1 Processor -- start 2023-10-18T09:25:10.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 -- start start 2023-10-18T09:25:10.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0ade920 con 0x561ca0d0be30 2023-10-18T09:25:10.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0d15f90 con 0x561ca0d0c7f0 2023-10-18T09:25:10.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54e3059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0d148b0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54ccd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:52262/0 (socket says 172.21.15.61:52262) 2023-10-18T09:25:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54ccd9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:49584/0 (socket says 172.21.15.61:49584) 2023-10-18T09:25:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54cdd9c700 1 -- 172.21.15.61:0/1508354974 learned_addr learned my addr 172.21.15.61:0/1508354974 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.632+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 -- 172.21.15.61:0/1508354974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 msgr2=0x561ca0d14250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 -- 172.21.15.61:0/1508354974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 msgr2=0x561ca0d00820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 -- 172.21.15.61:0/1508354974 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ca0a71c60 con 0x561ca0d0c7f0 2023-10-18T09:25:10.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:25:10.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.633+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003ec0 tx=0x7f54bc003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33bcda31983f1f2d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54b3fff700 1 -- 172.21.15.61:0/1508354974 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc02e5d0 con 0x561ca0d0c7f0 2023-10-18T09:25:10.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54b3fff700 1 -- 172.21.15.61:0/1508354974 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc020040 con 0x561ca0d0c7f0 2023-10-18T09:25:10.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54b3fff700 1 -- 172.21.15.61:0/1508354974 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc003ef0 con 0x561ca0d0c7f0 2023-10-18T09:25:10.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54e3059b80 1 -- 172.21.15.61:0/1508354974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 msgr2=0x561ca0d1c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54e3059b80 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f54bc003ec0 tx=0x7f54bc003cc0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.634+0000 7f54e3059b80 1 -- 172.21.15.61:0/1508354974 shutdown_connections 2023-10-18T09:25:10.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1c1e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d14250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 --2- 172.21.15.61:0/1508354974 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0be30 0x561ca0d00820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 -- 172.21.15.61:0/1508354974 >> 172.21.15.61:0/1508354974 conn(0x561ca0c7f720 msgr2=0x561ca0d0cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 -- 172.21.15.61:0/1508354974 shutdown_connections 2023-10-18T09:25:10.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 -- 172.21.15.61:0/1508354974 wait complete. 2023-10-18T09:25:10.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 -- start start 2023-10-18T09:25:10.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.635+0000 7f54e3059b80 1 Processor -- start 2023-10-18T09:25:10.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0be30 0x561ca0d1c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0d1ad30 con 0x561ca0d0c7f0 2023-10-18T09:25:10.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0d1aea0 con 0x561ca0d0be30 2023-10-18T09:25:10.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ca0d1b060 con 0x561ca0d0b4d0 2023-10-18T09:25:10.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54e3059b80 10 client.0 _ll_register_callbacks cb 0x561ca0b44a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:25:10.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54cdd9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46222/0 (socket says 172.21.15.61:46222) 2023-10-18T09:25:10.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.636+0000 7f54cdd9c700 1 -- 172.21.15.61:0/720962573 learned_addr learned my addr 172.21.15.61:0/720962573 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:10.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0be30 0x561ca0d1c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 -- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0be30 msgr2=0x561ca0d1c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0be30 0x561ca0d1c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.646 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 -- 172.21.15.61:0/720962573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 msgr2=0x561ca0d1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:10.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.637+0000 7f54cdd9c700 1 -- 172.21.15.61:0/720962573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ca0a88de0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.638+0000 7f54cdd9c700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009480 tx=0x7f54bc0098a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e910ef1309457ba server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:10.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.638+0000 7f54b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:25:10.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.638+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01e070 con 0x561ca0d0b4d0 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.638+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f54bc000e40 con 0x561ca0d0b4d0 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.638+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54bc003680 con 0x561ca0d0b4d0 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.639+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54bc004930 con 0x561ca0d0b4d0 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.640+0000 7f54e3059b80 10 client.26215 fetch_fsmap learned FSMap version 40 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.640+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561ca0c1b3f0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.641+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54bc003680 con 0x561ca0d0b4d0 2023-10-18T09:25:10.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.641+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f54bc0043a0 con 0x561ca0d0b4d0 2023-10-18T09:25:10.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 10 client.26215 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:25:10.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 10 client.26215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:25:10.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 20 client.26215 populate_metadata read hostname 'smithi061' 2023-10-18T09:25:10.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d759b0 0x561ca0d77dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:10.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 4 client.26215 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.642+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ca0c56e50 con 0x561ca0d759b0 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.643+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d759b0 0x561ca0d77dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.643+0000 7f54ccd9a700 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d759b0 0x561ca0d77dd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f9e4b93c4813c6 server_cookie=78a453d6f6690524 in_seq=0 out_seq=0 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.643+0000 7f54b0ff9700 10 client.26215 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54b0ff9700 1 -- 172.21.15.61:0/720962573 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561ca0c56e50 con 0x561ca0d759b0 2023-10-18T09:25:10.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54b0ff9700 10 client.26215 handle_command_reply: tid=0 2023-10-18T09:25:10.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 client.26215 shutdown 2023-10-18T09:25:10.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 msgr2=0x561ca0d1bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009480 tx=0x7f54bc0098a0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 shutdown_connections 2023-10-18T09:25:10.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561ca0d759b0 0x561ca0d77dd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561ca0d0c7f0 0x561ca0d1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561ca0d0b4d0 0x561ca0d1bc10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54b0ff9700 0 client.26215 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:10.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 --2- 172.21.15.61:0/720962573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561ca0d0be30 0x561ca0d1c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 >> 172.21.15.61:0/720962573 conn(0x561ca0c7f720 msgr2=0x561ca0d05cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 shutdown_connections 2023-10-18T09:25:10.654 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:25:10.654 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:25:10.654 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:25:10.655 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:25:10.656 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:25:10.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.644+0000 7f54e3059b80 1 -- 172.21.15.61:0/720962573 wait complete. 2023-10-18T09:25:10.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54e3059b80 20 client.26215 trim_cache size 0 max 16384 2023-10-18T09:25:10.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 msgr2=0x7f54b40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 0x7f54b40a39c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f54c00999a0 tx=0x7f54c0084040 comp rx=0 tx=0).stop 2023-10-18T09:25:10.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 msgr2=0x7f54c810f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:10.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c0007f40 tx=0x7f54c000d3b0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 shutdown_connections 2023-10-18T09:25:10.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f54b40a1500 0x7f54b40a39c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f54c81001c0 0x7f54c81acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c80feea0 0x7f54c810f6c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 --2- 172.21.15.61:0/3750031475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54c80ff800 0x7f54c810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:10.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.645+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 >> 172.21.15.61:0/3750031475 conn(0x7f54c80faa80 msgr2=0x7f54c80fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:10.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.646+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 shutdown_connections 2023-10-18T09:25:10.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:10.646+0000 7f54ce59d700 1 -- 172.21.15.61:0/3750031475 wait complete. 2023-10-18T09:25:10.675 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:25:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:11 smithi110 ceph-mon[134818]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-10-18T09:25:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:11 smithi061 ceph-mon[98573]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-10-18T09:25:11.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:11 smithi170 ceph-mon[135150]: pgmap v557: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 583 op/s 2023-10-18T09:25:13.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:13 smithi110 ceph-mon[134818]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 503 op/s 2023-10-18T09:25:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:13 smithi061 ceph-mon[98573]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 503 op/s 2023-10-18T09:25:13.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:13 smithi170 ceph-mon[135150]: pgmap v558: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 503 op/s 2023-10-18T09:25:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 360.0 s, 505.2 tps, lat 189.328 ms stddev 217.255 2023-10-18T09:25:15.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:15 smithi110 ceph-mon[134818]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 530 op/s 2023-10-18T09:25:15.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:15 smithi061 ceph-mon[98573]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 530 op/s 2023-10-18T09:25:15.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:15 smithi170 ceph-mon[135150]: pgmap v559: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 530 op/s 2023-10-18T09:25:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:17 smithi170 ceph-mon[135150]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2023-10-18T09:25:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:17 smithi110 ceph-mon[134818]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2023-10-18T09:25:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:17 smithi061 ceph-mon[98573]: pgmap v560: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.3 MiB/s wr, 628 op/s 2023-10-18T09:25:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 365.0 s, 436.4 tps, lat 230.989 ms stddev 260.059 2023-10-18T09:25:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:19 smithi110 ceph-mon[134818]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-18T09:25:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:19 smithi061 ceph-mon[98573]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-18T09:25:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:19 smithi170 ceph-mon[135150]: pgmap v561: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-18T09:25:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:21 smithi110 ceph-mon[134818]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 700 op/s 2023-10-18T09:25:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:21 smithi061 ceph-mon[98573]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 700 op/s 2023-10-18T09:25:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:21 smithi170 ceph-mon[135150]: pgmap v562: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 MiB/s wr, 700 op/s 2023-10-18T09:25:23.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:23 smithi110 ceph-mon[134818]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-10-18T09:25:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:23 smithi061 ceph-mon[98573]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-10-18T09:25:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:23 smithi170 ceph-mon[135150]: pgmap v563: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 MiB/s wr, 644 op/s 2023-10-18T09:25:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 370.0 s, 529.2 tps, lat 194.612 ms stddev 222.009 2023-10-18T09:25:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:25 smithi110 ceph-mon[134818]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-18T09:25:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:25 smithi061 ceph-mon[98573]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-18T09:25:25.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:25 smithi170 ceph-mon[135150]: pgmap v564: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 MiB/s wr, 651 op/s 2023-10-18T09:25:28.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:27 smithi170 ceph-mon[135150]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:25:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:27 smithi110 ceph-mon[134818]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:25:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:27 smithi061 ceph-mon[98573]: pgmap v565: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 697 op/s 2023-10-18T09:25:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 375.0 s, 670.8 tps, lat 147.474 ms stddev 176.914 2023-10-18T09:25:29.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:29 smithi110 ceph-mon[134818]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 671 op/s 2023-10-18T09:25:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:29 smithi061 ceph-mon[98573]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 671 op/s 2023-10-18T09:25:29.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:29 smithi170 ceph-mon[135150]: pgmap v566: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 MiB/s wr, 671 op/s 2023-10-18T09:25:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:31 smithi110 ceph-mon[134818]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 610 op/s 2023-10-18T09:25:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:31 smithi061 ceph-mon[98573]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 610 op/s 2023-10-18T09:25:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:31 smithi170 ceph-mon[135150]: pgmap v567: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 610 op/s 2023-10-18T09:25:33.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:33 smithi110 ceph-mon[134818]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 MiB/s wr, 562 op/s 2023-10-18T09:25:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:33 smithi061 ceph-mon[98573]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 MiB/s wr, 562 op/s 2023-10-18T09:25:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:33 smithi170 ceph-mon[135150]: pgmap v568: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.8 MiB/s wr, 562 op/s 2023-10-18T09:25:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 380.0 s, 572.8 tps, lat 176.128 ms stddev 210.995 2023-10-18T09:25:35.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:35 smithi110 ceph-mon[134818]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 MiB/s wr, 561 op/s 2023-10-18T09:25:35.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:35 smithi061 ceph-mon[98573]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 MiB/s wr, 561 op/s 2023-10-18T09:25:35.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:35 smithi170 ceph-mon[135150]: pgmap v569: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.6 MiB/s wr, 561 op/s 2023-10-18T09:25:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:37 smithi170 ceph-mon[135150]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-18T09:25:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:37 smithi110 ceph-mon[134818]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-18T09:25:38.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:37 smithi061 ceph-mon[98573]: pgmap v570: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 625 op/s 2023-10-18T09:25:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 385.0 s, 528.4 tps, lat 185.487 ms stddev 228.031 2023-10-18T09:25:39.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:39 smithi110 ceph-mon[134818]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 606 op/s 2023-10-18T09:25:39.515 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:39 smithi061 ceph-mon[98573]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 606 op/s 2023-10-18T09:25:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:39 smithi170 ceph-mon[135150]: pgmap v571: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 MiB/s wr, 606 op/s 2023-10-18T09:25:40.677 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:25:41.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:41 smithi110 ceph-mon[134818]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 572 op/s 2023-10-18T09:25:41.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:41 smithi170 ceph-mon[135150]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 572 op/s 2023-10-18T09:25:41.838 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:41 smithi061 ceph-mon[98573]: pgmap v572: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.3 MiB/s wr, 572 op/s 2023-10-18T09:25:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:43 smithi110 ceph-mon[134818]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-18T09:25:43.554 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:43 smithi061 ceph-mon[98573]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-18T09:25:43.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:43 smithi170 ceph-mon[135150]: pgmap v573: 129 pgs: 129 active+clean; 1.5 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-18T09:25:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 390.0 s, 255.6 tps, lat 360.938 ms stddev 672.713 2023-10-18T09:25:45.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:45 smithi110 ceph-mon[134818]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-10-18T09:25:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:45 smithi061 ceph-mon[98573]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-10-18T09:25:45.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:45 smithi170 ceph-mon[135150]: pgmap v574: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-10-18T09:25:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:47 smithi170 ceph-mon[135150]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-10-18T09:25:48.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:47 smithi110 ceph-mon[134818]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-10-18T09:25:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:47 smithi061 ceph-mon[98573]: pgmap v575: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 MiB/s wr, 1.35k op/s 2023-10-18T09:25:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 395.0 s, 446.4 tps, lat 246.934 ms stddev 447.060 2023-10-18T09:25:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:49 smithi110 ceph-mon[134818]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-10-18T09:25:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:49 smithi061 ceph-mon[98573]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-10-18T09:25:49.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:49 smithi170 ceph-mon[135150]: pgmap v576: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 20 MiB/s wr, 1.47k op/s 2023-10-18T09:25:49.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 -- 172.21.15.61:0/1991645856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 msgr2=0x7f4190100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:49.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 --2- 172.21.15.61:0/1991645856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 0x7f4190100ce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f41840098f0 tx=0x7f4184009c00 comp rx=0 tx=0).stop 2023-10-18T09:25:49.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 -- 172.21.15.61:0/1991645856 shutdown_connections 2023-10-18T09:25:49.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 --2- 172.21.15.61:0/1991645856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 --2- 172.21.15.61:0/1991645856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41900ff690 0x7f41900ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 --2- 172.21.15.61:0/1991645856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 0x7f4190100ce0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.924+0000 7f4195dff700 1 -- 172.21.15.61:0/1991645856 >> 172.21.15.61:0/1991645856 conn(0x7f41900fac00 msgr2=0x7f41900fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:49.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.925+0000 7f4195dff700 1 -- 172.21.15.61:0/1991645856 shutdown_connections 2023-10-18T09:25:49.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.925+0000 7f4195dff700 1 -- 172.21.15.61:0/1991645856 wait complete. 2023-10-18T09:25:49.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.926+0000 7f4195dff700 1 Processor -- start 2023-10-18T09:25:49.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.926+0000 7f4195dff700 1 -- start start 2023-10-18T09:25:49.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41900ff690 0x7f419006a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 0x7f419006ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190072120 con 0x7f41900ff690 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190072290 con 0x7f4190101220 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f4195dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190072450 con 0x7f4190100880 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43544/0 (socket says 172.21.15.61:43544) 2023-10-18T09:25:49.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.927+0000 7f418ffff700 1 -- 172.21.15.61:0/2550460711 learned_addr learned my addr 172.21.15.61:0/2550460711 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:49.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 -- 172.21.15.61:0/2550460711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 msgr2=0x7f419006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:49.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 0x7f419006ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 -- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41900ff690 msgr2=0x7f419006a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:49.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41900ff690 0x7f419006a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 -- 172.21.15.61:0/2550460711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4184009520 con 0x7f4190101220 2023-10-18T09:25:49.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.928+0000 7f418ffff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f418800c110 tx=0x7f418800c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:49.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.929+0000 7f418cff9700 1 -- 172.21.15.61:0/2550460711 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188017070 con 0x7f4190101220 2023-10-18T09:25:49.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.929+0000 7f418cff9700 1 -- 172.21.15.61:0/2550460711 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418800e050 con 0x7f4190101220 2023-10-18T09:25:49.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.929+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190072670 con 0x7f4190101220 2023-10-18T09:25:49.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.929+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4190101c80 con 0x7f4190101220 2023-10-18T09:25:49.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.929+0000 7f418cff9700 1 -- 172.21.15.61:0/2550460711 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188007a00 con 0x7f4190101220 2023-10-18T09:25:49.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.936+0000 7f418cff9700 1 -- 172.21.15.61:0/2550460711 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4188007c80 con 0x7f4190101220 2023-10-18T09:25:49.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.938+0000 7f418cff9700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 0x7f41780a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.939+0000 7f418cff9700 1 -- 172.21.15.61:0/2550460711 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f41880ccb70 con 0x7f4190101220 2023-10-18T09:25:49.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.939+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 0x7f41780a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 Processor -- start 2023-10-18T09:25:49.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 -- start start 2023-10-18T09:25:49.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 0x7f41780a39c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41901006e0 tx=0x7f4180006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:49.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11605440 0x561f116038a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 0x561f11604260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.945 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1156e6e0 con 0x561f11605440 2023-10-18T09:25:49.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f11554c60 con 0x561f11603de0 2023-10-18T09:25:49.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.940+0000 7f41a972eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f11552580 con 0x561f11604ae0 2023-10-18T09:25:49.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.941+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.941+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:45616/0 (socket says 172.21.15.61:45616) 2023-10-18T09:25:49.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.941+0000 7f418f7fe700 1 -- 172.21.15.61:0/3784795441 learned_addr learned my addr 172.21.15.61:0/3784795441 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:49.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.941+0000 7f418ffff700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 0x561f11604260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.941+0000 7f418effd700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11605440 0x561f116038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 -- 172.21.15.61:0/3784795441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 msgr2=0x561f11604260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:49.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 0x561f11604260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 -- 172.21.15.61:0/3784795441 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11605440 msgr2=0x561f116038a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:49.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11605440 0x561f116038a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 -- 172.21.15.61:0/3784795441 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1150b3f0 con 0x561f11604ae0 2023-10-18T09:25:49.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418ffff700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 0x561f11604260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:25:49.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f418f7fe700 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4180015950 tx=0x7f4180006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e78fcdf589645f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:49.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.942+0000 7f4175ffb700 1 -- 172.21.15.61:0/3784795441 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418001a070 con 0x561f11604ae0 2023-10-18T09:25:49.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.943+0000 7f4175ffb700 1 -- 172.21.15.61:0/3784795441 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4180012950 con 0x561f11604ae0 2023-10-18T09:25:49.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.943+0000 7f4175ffb700 1 -- 172.21.15.61:0/3784795441 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180013980 con 0x561f11604ae0 2023-10-18T09:25:49.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.944+0000 7f41a972eb80 1 -- 172.21.15.61:0/3784795441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 msgr2=0x561f11604f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:49.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.944+0000 7f41a972eb80 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4180015950 tx=0x7f4180006f90 comp rx=0 tx=0).stop 2023-10-18T09:25:49.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.946+0000 7f41a972eb80 1 -- 172.21.15.61:0/3784795441 shutdown_connections 2023-10-18T09:25:49.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.946+0000 7f41a972eb80 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11603de0 0x561f11604260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.946+0000 7f41a972eb80 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11604ae0 0x561f11604f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.946+0000 7f41a972eb80 1 --2- 172.21.15.61:0/3784795441 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11605440 0x561f116038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.946+0000 7f41a972eb80 1 -- 172.21.15.61:0/3784795441 >> 172.21.15.61:0/3784795441 conn(0x561f1156f720 msgr2=0x561f115f5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:49.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.947+0000 7f41a972eb80 1 -- 172.21.15.61:0/3784795441 shutdown_connections 2023-10-18T09:25:49.952 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.947+0000 7f41a972eb80 1 -- 172.21.15.61:0/3784795441 wait complete. 2023-10-18T09:25:49.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.947+0000 7f41a972eb80 1 -- start start 2023-10-18T09:25:49.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.948+0000 7f41a972eb80 1 Processor -- start 2023-10-18T09:25:49.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.949+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.950+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f1160d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.950+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11614900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f41a972eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1154de70 con 0x561f11604ae0 2023-10-18T09:25:49.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f41a972eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f114ea710 con 0x561f11605440 2023-10-18T09:25:49.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f41a972eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f114ea3b0 con 0x561f11603de0 2023-10-18T09:25:49.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:45620/0 (socket says 172.21.15.61:45620) 2023-10-18T09:25:49.955 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f418f7fe700 1 -- 172.21.15.61:0/2898803600 learned_addr learned my addr 172.21.15.61:0/2898803600 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:49.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f41a972eb80 10 client.0 _ll_register_callbacks cb 0x561f11434a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:25:49.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.951+0000 7f418ffff700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11614900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.952+0000 7f418f7fe700 1 -- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 msgr2=0x561f11614900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:49.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.952+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11614900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.952+0000 7f418f7fe700 1 -- 172.21.15.61:0/2898803600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 msgr2=0x561f1160d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:49.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.952+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f1160d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.952+0000 7f418f7fe700 1 -- 172.21.15.61:0/2898803600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4190101c80 con 0x561f11603de0 2023-10-18T09:25:49.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.953+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4180012720 tx=0x7f4180012750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9681fda0865ed7d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:49.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.953+0000 7f416effd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:25:49.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.953+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418001a040 con 0x561f11603de0 2023-10-18T09:25:49.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.953+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f11552ef0 con 0x561f11603de0 2023-10-18T09:25:49.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.953+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4180013b10 con 0x561f11603de0 2023-10-18T09:25:49.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.955+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4180013cd0 con 0x561f11603de0 2023-10-18T09:25:49.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.955+0000 7f41a972eb80 10 client.26224 fetch_fsmap learned FSMap version 40 2023-10-18T09:25:49.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.955+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f4180015630 con 0x561f11603de0 2023-10-18T09:25:49.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f11552ef0 con 0x561f11603de0 2023-10-18T09:25:49.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f41800273f0 con 0x561f11603de0 2023-10-18T09:25:49.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 10 client.26224 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:25:49.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:25:49.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 20 client.26224 populate_metadata read hostname 'smithi061' 2023-10-18T09:25:49.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f11619070 0x561f1161b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:49.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 4 client.26224 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:25:49.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.959+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f115ca8b0 con 0x561f11619070 2023-10-18T09:25:49.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.966+0000 7f418effd700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f11619070 0x561f1161b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:49.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.967+0000 7f418effd700 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f11619070 0x561f1161b490 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b034733d46248a36 server_cookie=47441217eef0d2a3 in_seq=0 out_seq=0 2023-10-18T09:25:49.970 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.967+0000 7f416effd700 10 client.26224 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:49.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.969+0000 7f416effd700 1 -- 172.21.15.61:0/2898803600 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561f115ca8b0 con 0x561f11619070 2023-10-18T09:25:49.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.969+0000 7f416effd700 10 client.26224 handle_command_reply: tid=0 2023-10-18T09:25:49.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.969+0000 7f41a972eb80 1 client.26224 shutdown 2023-10-18T09:25:49.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 msgr2=0x561f1160d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:49.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4180012720 tx=0x7f4180012750 comp rx=0 tx=0).stop 2023-10-18T09:25:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 shutdown_connections 2023-10-18T09:25:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f11619070 0x561f1161b490 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11614900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f1160d380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2898803600 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f1160d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:49.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 >> 172.21.15.61:0/2898803600 conn(0x561f1156f720 msgr2=0x561f11602890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:49.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.970+0000 7f416effd700 0 client.26224 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:49.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.972+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 shutdown_connections 2023-10-18T09:25:49.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.972+0000 7f41a972eb80 1 -- 172.21.15.61:0/2898803600 wait complete. 2023-10-18T09:25:49.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:49.972+0000 7f41a972eb80 20 client.26224 trim_cache size 0 max 16384 2023-10-18T09:25:50.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.013+0000 7f41a972eb80 1 Processor -- start 2023-10-18T09:25:50.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.013+0000 7f41a972eb80 1 -- start start 2023-10-18T09:25:50.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f11607ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11614890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f11607ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f418effd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11614890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43554/0 (socket says 172.21.15.61:43554) 2023-10-18T09:25:50.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f418ffff700 1 -- 172.21.15.61:0/1389232039 learned_addr learned my addr 172.21.15.61:0/1389232039 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:50.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.014+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f113ce920 con 0x561f11604ae0 2023-10-18T09:25:50.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.015+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f115ee060 con 0x561f11605440 2023-10-18T09:25:50.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.015+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f116038d0 con 0x561f11603de0 2023-10-18T09:25:50.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.015+0000 7f418ffff700 1 -- 172.21.15.61:0/1389232039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 msgr2=0x561f11607ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:50.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.015+0000 7f418ffff700 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f11607ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.015+0000 7f418ffff700 1 -- 172.21.15.61:0/1389232039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 msgr2=0x561f11614890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:50.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f418ffff700 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11614890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f418ffff700 1 -- 172.21.15.61:0/1389232039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f11378de0 con 0x561f11605440 2023-10-18T09:25:50.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f418ffff700 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f41880ca320 tx=0x7f418801e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a49efb6af30a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:50.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f41767fc700 1 -- 172.21.15.61:0/1389232039 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188099b40 con 0x561f11605440 2023-10-18T09:25:50.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f41767fc700 1 -- 172.21.15.61:0/1389232039 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41880b1070 con 0x561f11605440 2023-10-18T09:25:50.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.016+0000 7f41767fc700 1 -- 172.21.15.61:0/1389232039 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188097fe0 con 0x561f11605440 2023-10-18T09:25:50.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 msgr2=0x561f11608020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:50.023 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f41880ca320 tx=0x7f418801e040 comp rx=0 tx=0).stop 2023-10-18T09:25:50.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 shutdown_connections 2023-10-18T09:25:50.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f11608020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f11607ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 --2- 172.21.15.61:0/1389232039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11614890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 >> 172.21.15.61:0/1389232039 conn(0x561f1156f720 msgr2=0x561f1160c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:50.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 shutdown_connections 2023-10-18T09:25:50.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.017+0000 7f41a972eb80 1 -- 172.21.15.61:0/1389232039 wait complete. 2023-10-18T09:25:50.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.018+0000 7f41a972eb80 1 -- start start 2023-10-18T09:25:50.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.019+0000 7f41a972eb80 1 Processor -- start 2023-10-18T09:25:50.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f116145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11612ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f116131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f11613720 con 0x561f11604ae0 2023-10-18T09:25:50.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f11613890 con 0x561f11605440 2023-10-18T09:25:50.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f11613a50 con 0x561f11603de0 2023-10-18T09:25:50.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f41a972eb80 10 client.0 _ll_register_callbacks cb 0x561f11434a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:25:50.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f116131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f418f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f116145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f418ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f116131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43562/0 (socket says 172.21.15.61:43562) 2023-10-18T09:25:50.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f418ffff700 1 -- 172.21.15.61:0/2454618298 learned_addr learned my addr 172.21.15.61:0/2454618298 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:25:50.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.020+0000 7f418effd700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11612ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.021+0000 7f418effd700 1 -- 172.21.15.61:0/2454618298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 msgr2=0x561f116145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:50.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.021+0000 7f418effd700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f116145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.021+0000 7f418effd700 1 -- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 msgr2=0x561f116131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:25:50.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.021+0000 7f418effd700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f116131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.021+0000 7f418effd700 1 -- 172.21.15.61:0/2454618298 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418809ae70 con 0x561f11604ae0 2023-10-18T09:25:50.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.022+0000 7f418effd700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11612ca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4184009810 tx=0x7f418400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d00b1e43d3dc624 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:25:50.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.022+0000 7f416effd700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:25:50.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.022+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418401c040 con 0x561f11604ae0 2023-10-18T09:25:50.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.022+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4184022970 con 0x561f11604ae0 2023-10-18T09:25:50.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.023+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4180012950 con 0x561f11604ae0 2023-10-18T09:25:50.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.024+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4184022580 con 0x561f11604ae0 2023-10-18T09:25:50.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.024+0000 7f41a972eb80 10 client.16098 fetch_fsmap learned FSMap version 40 2023-10-18T09:25:50.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.024+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x561f11361c60 con 0x561f11604ae0 2023-10-18T09:25:50.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.025+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4180012950 con 0x561f11604ae0 2023-10-18T09:25:50.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.026+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 40) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4184015020 con 0x561f11604ae0 2023-10-18T09:25:50.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 10 client.16098 fetch_fsmap finished waiting for FSMap version 40 2023-10-18T09:25:50.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:25:50.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 20 client.16098 populate_metadata read hostname 'smithi061' 2023-10-18T09:25:50.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f1166e650 0x561f11670a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:25:50.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 4 client.16098 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:25:50.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f11546e50 con 0x561f1166e650 2023-10-18T09:25:50.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.027+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f1166e650 0x561f11670a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:25:50.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.028+0000 7f418f7fe700 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f1166e650 0x561f11670a70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75984e3a6e104d47 server_cookie=4b4c2129b8fb2654 in_seq=0 out_seq=0 2023-10-18T09:25:50.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.028+0000 7f416effd700 10 client.16098 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:50.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.029+0000 7f416effd700 1 -- 172.21.15.61:0/2454618298 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561f11546e50 con 0x561f1166e650 2023-10-18T09:25:50.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.029+0000 7f416effd700 10 client.16098 handle_command_reply: tid=0 2023-10-18T09:25:50.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.029+0000 7f41a972eb80 1 client.16098 shutdown 2023-10-18T09:25:50.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 msgr2=0x561f11612ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:50.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11612ca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4184009810 tx=0x7f418400f790 comp rx=0 tx=0).stop 2023-10-18T09:25:50.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 shutdown_connections 2023-10-18T09:25:50.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x561f1166e650 0x561f11670a70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f11605440 0x561f116131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f416effd700 0 client.16098 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:25:50.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.030+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561f11603de0 0x561f116145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.031+0000 7f41a972eb80 1 --2- 172.21.15.61:0/2454618298 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x561f11604ae0 0x561f11612ca0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.031+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 >> 172.21.15.61:0/2454618298 conn(0x561f1156f720 msgr2=0x561f11607ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:50.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.031+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 shutdown_connections 2023-10-18T09:25:50.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.031+0000 7f41a972eb80 1 -- 172.21.15.61:0/2454618298 wait complete. 2023-10-18T09:25:50.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.032+0000 7f41a972eb80 20 client.16098 trim_cache size 0 max 16384 2023-10-18T09:25:50.040 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:25:50.041 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:25:50.041 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:25:50.041 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:25:50.041 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:25:50.042 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:25:50.042 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:25:50.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:25:50.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:25:50.042 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:25:50.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 msgr2=0x7f41780a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:50.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 0x7f41780a39c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41901006e0 tx=0x7f4180006d20 comp rx=0 tx=0).stop 2023-10-18T09:25:50.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 msgr2=0x7f4190071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:25:50.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f418800c110 tx=0x7f418800c4d0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 shutdown_connections 2023-10-18T09:25:50.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f41780a1500 0x7f41780a39c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4190101220 0x7f4190071b50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f41900ff690 0x7f419006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 --2- 172.21.15.61:0/2550460711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4190100880 0x7f419006ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:25:50.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 >> 172.21.15.61:0/2550460711 conn(0x7f41900fac00 msgr2=0x7f4190067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:25:50.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 shutdown_connections 2023-10-18T09:25:50.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:25:50.033+0000 7f4195dff700 1 -- 172.21.15.61:0/2550460711 wait complete. 2023-10-18T09:25:50.100 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:25:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:51 smithi110 ceph-mon[134818]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-10-18T09:25:51.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:51 smithi061 ceph-mon[98573]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-10-18T09:25:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:51 smithi170 ceph-mon[135150]: pgmap v577: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 19 MiB/s wr, 1.46k op/s 2023-10-18T09:25:53.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:53 smithi110 ceph-mon[134818]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-10-18T09:25:53.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:53 smithi061 ceph-mon[98573]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-10-18T09:25:53.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:53 smithi170 ceph-mon[135150]: pgmap v578: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.44k op/s 2023-10-18T09:25:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 400.0 s, 477.4 tps, lat 207.642 ms stddev 213.530 2023-10-18T09:25:55.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:55 smithi110 ceph-mon[134818]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-10-18T09:25:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:55 smithi061 ceph-mon[98573]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-10-18T09:25:55.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:55 smithi170 ceph-mon[135150]: pgmap v579: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-10-18T09:25:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:57 smithi170 ceph-mon[135150]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 875 op/s 2023-10-18T09:25:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:57 smithi110 ceph-mon[134818]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 875 op/s 2023-10-18T09:25:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:57 smithi061 ceph-mon[98573]: pgmap v580: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 875 op/s 2023-10-18T09:25:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 405.0 s, 604.4 tps, lat 163.370 ms stddev 171.343 2023-10-18T09:25:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:25:59 smithi110 ceph-mon[134818]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 721 op/s 2023-10-18T09:25:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:25:59 smithi061 ceph-mon[98573]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 721 op/s 2023-10-18T09:25:59.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:25:59 smithi170 ceph-mon[135150]: pgmap v581: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 9.9 MiB/s wr, 721 op/s 2023-10-18T09:26:01.473 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:01 smithi110 ceph-mon[134818]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 548 op/s 2023-10-18T09:26:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:01 smithi061 ceph-mon[98573]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 548 op/s 2023-10-18T09:26:01.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:01 smithi170 ceph-mon[135150]: pgmap v582: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 548 op/s 2023-10-18T09:26:03.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:03 smithi110 ceph-mon[134818]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-10-18T09:26:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:03 smithi061 ceph-mon[98573]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-10-18T09:26:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:03 smithi170 ceph-mon[135150]: pgmap v583: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.9 MiB/s wr, 538 op/s 2023-10-18T09:26:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 410.0 s, 526.6 tps, lat 190.217 ms stddev 245.054 2023-10-18T09:26:04.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:04 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:26:04.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:04 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:26:04.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:04 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:26:05.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-10-18T09:26:05.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-18T09:26:05.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:05 smithi110 ceph-mon[134818]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 583 op/s 2023-10-18T09:26:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-10-18T09:26:05.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-18T09:26:05.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:05 smithi061 ceph-mon[98573]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 583 op/s 2023-10-18T09:26:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi170 to 196.2M 2023-10-18T09:26:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: Unable to set osd_memory_target on smithi170 to 205773209: error parsing value: Value '205773209' is below minimum 939524096 2023-10-18T09:26:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:05 smithi170 ceph-mon[135150]: pgmap v584: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 583 op/s 2023-10-18T09:26:06.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:06 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:06.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:06 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:06.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:06 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:07.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi110 to 196.2M 2023-10-18T09:26:07.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: Unable to set osd_memory_target on smithi110 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-18T09:26:07.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:07 smithi110 ceph-mon[134818]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 572 op/s 2023-10-18T09:26:07.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi110 to 196.2M 2023-10-18T09:26:07.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: Unable to set osd_memory_target on smithi110 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-18T09:26:07.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:07 smithi061 ceph-mon[98573]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 572 op/s 2023-10-18T09:26:07.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:26:07.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi110 to 196.2M 2023-10-18T09:26:07.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: Unable to set osd_memory_target on smithi110 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-10-18T09:26:07.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:07 smithi170 ceph-mon[135150]: pgmap v585: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.4 MiB/s wr, 572 op/s 2023-10-18T09:26:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 415.0 s, 510.8 tps, lat 198.017 ms stddev 232.221 2023-10-18T09:26:09.473 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:09 smithi110 ceph-mon[134818]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.9 MiB/s wr, 651 op/s 2023-10-18T09:26:09.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:09 smithi061 ceph-mon[98573]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.9 MiB/s wr, 651 op/s 2023-10-18T09:26:09.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:09 smithi170 ceph-mon[135150]: pgmap v586: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.9 MiB/s wr, 651 op/s 2023-10-18T09:26:10.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:26:10.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:26:10.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:26:10.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:26:10.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:26:10.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:26:10.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:26:10.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:26:10.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:26:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:11 smithi110 ceph-mon[134818]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 559 op/s 2023-10-18T09:26:11.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:11 smithi061 ceph-mon[98573]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 559 op/s 2023-10-18T09:26:11.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:11 smithi170 ceph-mon[135150]: pgmap v587: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.0 MiB/s wr, 559 op/s 2023-10-18T09:26:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:13 smithi110 ceph-mon[134818]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2023-10-18T09:26:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:13 smithi061 ceph-mon[98573]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2023-10-18T09:26:13.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:13 smithi170 ceph-mon[135150]: pgmap v588: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.8 MiB/s wr, 582 op/s 2023-10-18T09:26:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 420.0 s, 612.2 tps, lat 163.098 ms stddev 209.306 2023-10-18T09:26:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:15 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:15 smithi110 ceph-mon[134818]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 579 op/s 2023-10-18T09:26:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:15 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:15 smithi061 ceph-mon[98573]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 579 op/s 2023-10-18T09:26:15.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:15 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:15.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:15 smithi170 ceph-mon[135150]: pgmap v589: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 579 op/s 2023-10-18T09:26:18.147 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:17 smithi170 ceph-mon[135150]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-10-18T09:26:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:17 smithi110 ceph-mon[134818]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-10-18T09:26:18.257 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:17 smithi061 ceph-mon[98573]: pgmap v590: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.0 MiB/s wr, 529 op/s 2023-10-18T09:26:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 425.0 s, 537.6 tps, lat 181.881 ms stddev 205.353 2023-10-18T09:26:19.468 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:19 smithi110 ceph-mon[134818]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 589 op/s 2023-10-18T09:26:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:19 smithi061 ceph-mon[98573]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 589 op/s 2023-10-18T09:26:19.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:19 smithi170 ceph-mon[135150]: pgmap v591: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 589 op/s 2023-10-18T09:26:20.102 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:26:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:21 smithi110 ceph-mon[134818]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 482 op/s 2023-10-18T09:26:21.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:21 smithi061 ceph-mon[98573]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 482 op/s 2023-10-18T09:26:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:21 smithi170 ceph-mon[135150]: pgmap v592: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 MiB/s wr, 482 op/s 2023-10-18T09:26:22.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 -- 172.21.15.61:0/3049450323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d81003c0 msgr2=0x7fb4d8100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/3049450323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d81003c0 0x7fb4d8100840 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009990 tx=0x7fb4d0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 -- 172.21.15.61:0/3049450323 shutdown_connections 2023-10-18T09:26:22.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/3049450323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d81003c0 0x7fb4d8100840 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/3049450323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 0x7fb4d80ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/3049450323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d80ffa00 0x7fb4d80ffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.674+0000 7fb4e07ff700 1 -- 172.21.15.61:0/3049450323 >> 172.21.15.61:0/3049450323 conn(0x7fb4d80fac00 msgr2=0x7fb4d80fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.675+0000 7fb4e07ff700 1 -- 172.21.15.61:0/3049450323 shutdown_connections 2023-10-18T09:26:22.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.675+0000 7fb4e07ff700 1 -- 172.21.15.61:0/3049450323 wait complete. 2023-10-18T09:26:22.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.677+0000 7fb4e07ff700 1 Processor -- start 2023-10-18T09:26:22.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.677+0000 7fb4e07ff700 1 -- start start 2023-10-18T09:26:22.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 0x7fb4d810f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d81003c0 0x7fb4d81acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8110470 con 0x7fb4d80ff0a0 2023-10-18T09:26:22.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4ddd9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d81ad2c0 con 0x7fb4d80ffa00 2023-10-18T09:26:22.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4e07ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d81ad400 con 0x7fb4d81003c0 2023-10-18T09:26:22.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4de59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 0x7fb4d810f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4ddd9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36324/0 (socket says 172.21.15.61:36324) 2023-10-18T09:26:22.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/311004258 learned_addr learned my addr 172.21.15.61:0/311004258 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:22.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.678+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d81003c0 0x7fb4d81acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/311004258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d81003c0 msgr2=0x7fb4d81acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d81003c0 0x7fb4d81acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 msgr2=0x7fb4d810f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 0x7fb4d810f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/311004258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00095f0 con 0x7fb4d80ffa00 2023-10-18T09:26:22.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400b530 tx=0x7fb4c400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4d77fe700 1 -- 172.21.15.61:0/311004258 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011830 con 0x7fb4d80ffa00 2023-10-18T09:26:22.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d81ad690 con 0x7fb4d80ffa00 2023-10-18T09:26:22.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4d77fe700 1 -- 172.21.15.61:0/311004258 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c4011e50 con 0x7fb4d80ffa00 2023-10-18T09:26:22.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.679+0000 7fb4d77fe700 1 -- 172.21.15.61:0/311004258 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4010de0 con 0x7fb4d80ffa00 2023-10-18T09:26:22.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.680+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d81adbf0 con 0x7fb4d80ffa00 2023-10-18T09:26:22.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.687+0000 7fb4d77fe700 1 -- 172.21.15.61:0/311004258 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fb4c401a400 con 0x7fb4d80ffa00 2023-10-18T09:26:22.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.689+0000 7fb4d77fe700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 0x7fb4c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.690+0000 7fb4d77fe700 1 -- 172.21.15.61:0/311004258 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb4c40cda30 con 0x7fb4d80ffa00 2023-10-18T09:26:22.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.693+0000 7fb4de59b700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 0x7fb4c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.724+0000 7fb4de59b700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 0x7fb4c80a3940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00a400 tx=0x7fb4cc00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.724+0000 7fb4f4159b80 1 Processor -- start 2023-10-18T09:26:22.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.725+0000 7fb4f4159b80 1 -- start start 2023-10-18T09:26:22.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.725+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x5573164786f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316477090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316477a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573163ea6e0 con 0x5573164775d0 2023-10-18T09:26:22.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573163d0c60 con 0x557316478c30 2023-10-18T09:26:22.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573163ce580 con 0x5573164782d0 2023-10-18T09:26:22.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x5573164786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4de59b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x5573164786f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:35142/0 (socket says 172.21.15.61:35142) 2023-10-18T09:26:22.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4de59b700 1 -- 172.21.15.61:0/259957705 learned_addr learned my addr 172.21.15.61:0/259957705 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:22.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.726+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316477090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/259957705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 msgr2=0x5573164786f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x5573164786f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/259957705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 msgr2=0x557316477a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316477a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 -- 172.21.15.61:0/259957705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573163873f0 con 0x557316478c30 2023-10-18T09:26:22.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316477090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40cb550 tx=0x7fb4c4004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8f326ec633710c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4d4ff9700 1 -- 172.21.15.61:0/259957705 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c409aa00 con 0x557316478c30 2023-10-18T09:26:22.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4d4ff9700 1 -- 172.21.15.61:0/259957705 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4c409eb50 con 0x557316478c30 2023-10-18T09:26:22.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.727+0000 7fb4d4ff9700 1 -- 172.21.15.61:0/259957705 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40a2900 con 0x557316478c30 2023-10-18T09:26:22.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 -- 172.21.15.61:0/259957705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 msgr2=0x557316477090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316477090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40cb550 tx=0x7fb4c4004860 comp rx=0 tx=0).stop 2023-10-18T09:26:22.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 -- 172.21.15.61:0/259957705 shutdown_connections 2023-10-18T09:26:22.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316477a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x5573164786f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/259957705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316477090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 -- 172.21.15.61:0/259957705 >> 172.21.15.61:0/259957705 conn(0x5573163eb720 msgr2=0x557316471b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 -- 172.21.15.61:0/259957705 shutdown_connections 2023-10-18T09:26:22.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.729+0000 7fb4f4159b80 1 -- 172.21.15.61:0/259957705 wait complete. 2023-10-18T09:26:22.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.730+0000 7fb4f4159b80 1 -- start start 2023-10-18T09:26:22.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.730+0000 7fb4f4159b80 1 Processor -- start 2023-10-18T09:26:22.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316480ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x557316480ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573163c9e70 con 0x5573164775d0 2023-10-18T09:26:22.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557316366710 con 0x557316478c30 2023-10-18T09:26:22.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573163663b0 con 0x5573164782d0 2023-10-18T09:26:22.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4f4159b80 10 client.0 _ll_register_callbacks cb 0x5573162b0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:26:22.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.731+0000 7fb4de59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316480ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36340/0 (socket says 172.21.15.61:36340) 2023-10-18T09:26:22.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4de59b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316480ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57794/0 (socket says 172.21.15.61:57794) 2023-10-18T09:26:22.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4de59b700 1 -- 172.21.15.61:0/1477471070 learned_addr learned my addr 172.21.15.61:0/1477471070 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:22.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ddd9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x557316480ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 -- 172.21.15.61:0/1477471070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 msgr2=0x557316480ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x557316480ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 -- 172.21.15.61:0/1477471070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 msgr2=0x557316480ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316480ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.732+0000 7fb4ded9c700 1 -- 172.21.15.61:0/1477471070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d81adbf0 con 0x557316478c30 2023-10-18T09:26:22.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0006010 tx=0x7fb4d0005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf07364e89076b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:26:22.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001d070 con 0x557316478c30 2023-10-18T09:26:22.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb4d00056b0 con 0x557316478c30 2023-10-18T09:26:22.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573163ceef0 con 0x557316478c30 2023-10-18T09:26:22.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d0004710 con 0x557316478c30 2023-10-18T09:26:22.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4f4159b80 10 client.26285 fetch_fsmap learned FSMap version 42 2023-10-18T09:26:22.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.733+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5573161f4de0 con 0x557316478c30 2023-10-18T09:26:22.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d001e040 con 0x557316478c30 2023-10-18T09:26:22.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb4d000f9c0 con 0x557316478c30 2023-10-18T09:26:22.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 10 client.26285 fetch_fsmap finished waiting for FSMap version 42 2023-10-18T09:26:22.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:26:22.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 20 client.26285 populate_metadata read hostname 'smithi061' 2023-10-18T09:26:22.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55731648c970 0x55731648edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 4 client.26285 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:26:22.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.734+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573163ceef0 con 0x55731648c970 2023-10-18T09:26:22.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.735+0000 7fb4de59b700 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55731648c970 0x55731648edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.735+0000 7fb4de59b700 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55731648c970 0x55731648edb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17293e7ecd13c45 server_cookie=b8cefd40577000e7 in_seq=0 out_seq=0 2023-10-18T09:26:22.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.735+0000 7fb4c1ffb700 10 client.26285 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:22.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.736+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/1477471070 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5573163ceef0 con 0x55731648c970 2023-10-18T09:26:22.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.736+0000 7fb4c1ffb700 10 client.26285 handle_command_reply: tid=0 2023-10-18T09:26:22.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.736+0000 7fb4f4159b80 1 client.26285 shutdown 2023-10-18T09:26:22.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 msgr2=0x557316488030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0006010 tx=0x7fb4d0005c90 comp rx=0 tx=0).stop 2023-10-18T09:26:22.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 shutdown_connections 2023-10-18T09:26:22.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55731648c970 0x55731648edb0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316488030 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164775d0 0x557316480ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4c1ffb700 0 client.26285 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:22.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/1477471070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164782d0 0x557316480ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 >> 172.21.15.61:0/1477471070 conn(0x5573163eb720 msgr2=0x557316476080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.737+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 shutdown_connections 2023-10-18T09:26:22.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.738+0000 7fb4f4159b80 1 -- 172.21.15.61:0/1477471070 wait complete. 2023-10-18T09:26:22.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.738+0000 7fb4f4159b80 20 client.26285 trim_cache size 0 max 16384 2023-10-18T09:26:22.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.777+0000 7fb4f4159b80 1 Processor -- start 2023-10-18T09:26:22.780 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.777+0000 7fb4f4159b80 1 -- start start 2023-10-18T09:26:22.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x55731647ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573164782d0 0x557316487f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.781 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731624a920 con 0x557316478c30 2023-10-18T09:26:22.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557316488490 con 0x5573164782d0 2023-10-18T09:26:22.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.778+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557316473b80 con 0x5573164775d0 2023-10-18T09:26:22.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.782 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:57800/0 (socket says 172.21.15.61:57800) 2023-10-18T09:26:22.783 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 -- 172.21.15.61:0/746376325 learned_addr learned my addr 172.21.15.61:0/746376325 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:22.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 -- 172.21.15.61:0/746376325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 msgr2=0x55731647ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x55731647ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.789 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 -- 172.21.15.61:0/746376325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573164782d0 msgr2=0x557316487f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573164782d0 0x557316487f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.779+0000 7fb4ded9c700 1 -- 172.21.15.61:0/746376325 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573161ddc60 con 0x557316478c30 2023-10-18T09:26:22.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.780+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009990 tx=0x7fb4d0029480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f1276600df3ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.790 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.781+0000 7fb4d57fa700 1 -- 172.21.15.61:0/746376325 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001d040 con 0x557316478c30 2023-10-18T09:26:22.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.781+0000 7fb4d57fa700 1 -- 172.21.15.61:0/746376325 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d002a730 con 0x557316478c30 2023-10-18T09:26:22.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.781+0000 7fb4d57fa700 1 -- 172.21.15.61:0/746376325 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00163a0 con 0x557316478c30 2023-10-18T09:26:22.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.782+0000 7fb4f4159b80 1 -- 172.21.15.61:0/746376325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 msgr2=0x55731647eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.782+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009990 tx=0x7fb4d0029480 comp rx=0 tx=0).stop 2023-10-18T09:26:22.791 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 -- 172.21.15.61:0/746376325 shutdown_connections 2023-10-18T09:26:22.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x557316478c30 0x55731647eac0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x55731647ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/746376325 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573164782d0 0x557316487f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.792 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 -- 172.21.15.61:0/746376325 >> 172.21.15.61:0/746376325 conn(0x5573163eb720 msgr2=0x557316471b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 -- 172.21.15.61:0/746376325 shutdown_connections 2023-10-18T09:26:22.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.783+0000 7fb4f4159b80 1 -- 172.21.15.61:0/746376325 wait complete. 2023-10-18T09:26:22.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.784+0000 7fb4f4159b80 1 -- start start 2023-10-18T09:26:22.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.785+0000 7fb4f4159b80 1 Processor -- start 2023-10-18T09:26:22.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x557316487ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 0x557316486580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573164872c0 con 0x5573164782d0 2023-10-18T09:26:22.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557316487430 con 0x557316478c30 2023-10-18T09:26:22.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573164e0830 con 0x5573164775d0 2023-10-18T09:26:22.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4f4159b80 10 client.0 _ll_register_callbacks cb 0x5573162b0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:26:22.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.786+0000 7fb4ded9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:36350/0 (socket says 172.21.15.61:36350) 2023-10-18T09:26:22.796 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 -- 172.21.15.61:0/2140478348 learned_addr learned my addr 172.21.15.61:0/2140478348 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:22.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 0x557316486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 -- 172.21.15.61:0/2140478348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 msgr2=0x557316487ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x557316487ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 -- 172.21.15.61:0/2140478348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 msgr2=0x557316486580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:22.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 0x557316486580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ded9c700 1 -- 172.21.15.61:0/2140478348 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573161f4de0 con 0x557316478c30 2023-10-18T09:26:22.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.787+0000 7fb4ddd9a700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 0x557316486580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:26:22.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.788+0000 7fb4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:26:22.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.788+0000 7fb4ded9c700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0000c00 tx=0x7fb4d0029650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0051e18dfe3ede9 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:22.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.788+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d001d070 con 0x557316478c30 2023-10-18T09:26:22.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.788+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb4d00163a0 con 0x557316478c30 2023-10-18T09:26:22.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.789+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573164468b0 con 0x557316478c30 2023-10-18T09:26:22.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.789+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d0016560 con 0x557316478c30 2023-10-18T09:26:22.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.789+0000 7fb4f4159b80 10 client.26288 fetch_fsmap learned FSMap version 42 2023-10-18T09:26:22.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.789+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55731621b250 con 0x557316478c30 2023-10-18T09:26:22.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5573164468b0 con 0x557316478c30 2023-10-18T09:26:22.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 42) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb4d000f9c0 con 0x557316478c30 2023-10-18T09:26:22.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 10 client.26288 fetch_fsmap finished waiting for FSMap version 42 2023-10-18T09:26:22.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:26:22.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 20 client.26288 populate_metadata read hostname 'smithi061' 2023-10-18T09:26:22.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5573164e1db0 0x5573164e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:22.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 4 client.26288 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:26:22.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573163c2e50 con 0x5573164e1db0 2023-10-18T09:26:22.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.790+0000 7fb4de59b700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5573164e1db0 0x5573164e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:22.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.795+0000 7fb4de59b700 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5573164e1db0 0x5573164e41d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294a836b10ca6b95 server_cookie=debad2fffbfd72f4 in_seq=0 out_seq=0 2023-10-18T09:26:22.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.795+0000 7fb4c1ffb700 10 client.26288 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:22.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.795+0000 7fb4c1ffb700 1 -- 172.21.15.61:0/2140478348 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5573163c2e50 con 0x5573164e1db0 2023-10-18T09:26:22.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.795+0000 7fb4c1ffb700 10 client.26288 handle_command_reply: tid=0 2023-10-18T09:26:22.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.795+0000 7fb4f4159b80 1 client.26288 shutdown 2023-10-18T09:26:22.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.796+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 msgr2=0x557316486b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.796+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0000c00 tx=0x7fb4d0029650 comp rx=0 tx=0).stop 2023-10-18T09:26:22.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 shutdown_connections 2023-10-18T09:26:22.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5573164e1db0 0x5573164e41d0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557316478c30 0x557316486b60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573164775d0 0x557316487ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 --2- 172.21.15.61:0/2140478348 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5573164782d0 0x557316486580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4c1ffb700 0 client.26288 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:22.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 >> 172.21.15.61:0/2140478348 conn(0x5573163eb720 msgr2=0x55731647ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 shutdown_connections 2023-10-18T09:26:22.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.800+0000 7fb4f4159b80 1 -- 172.21.15.61:0/2140478348 wait complete. 2023-10-18T09:26:22.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.801+0000 7fb4f4159b80 20 client.26288 trim_cache size 0 max 16384 2023-10-18T09:26:22.809 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:26:22.809 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:26:22.809 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:26:22.810 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:26:22.810 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:26:22.810 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:26:22.810 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:26:22.810 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:26:22.811 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:26:22.811 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:26:22.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.801+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 msgr2=0x7fb4c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.801+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 0x7fb4c80a3940 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00a400 tx=0x7fb4cc00d040 comp rx=0 tx=0).stop 2023-10-18T09:26:22.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.801+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 msgr2=0x7fb4d810fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:22.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.801+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb4c400b530 tx=0x7fb4c400b840 comp rx=0 tx=0).stop 2023-10-18T09:26:22.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 shutdown_connections 2023-10-18T09:26:22.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fb4c80a1480 0x7fb4c80a3940 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d81003c0 0x7fb4d81acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fb4d80ff0a0 0x7fb4d810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 --2- 172.21.15.61:0/311004258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4d80ffa00 0x7fb4d810fe10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:22.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 >> 172.21.15.61:0/311004258 conn(0x7fb4d80fac00 msgr2=0x7fb4d80fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:22.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 shutdown_connections 2023-10-18T09:26:22.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:22.802+0000 7fb4e07ff700 1 -- 172.21.15.61:0/311004258 wait complete. 2023-10-18T09:26:22.838 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:26:23.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:23 smithi110 ceph-mon[134818]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 530 op/s 2023-10-18T09:26:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:23 smithi061 ceph-mon[98573]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 530 op/s 2023-10-18T09:26:23.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:23 smithi170 ceph-mon[135150]: pgmap v593: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.6 MiB/s wr, 530 op/s 2023-10-18T09:26:24.340 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 430.0 s, 526.6 tps, lat 192.840 ms stddev 226.984 2023-10-18T09:26:25.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:25 smithi110 ceph-mon[134818]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 555 op/s 2023-10-18T09:26:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:25 smithi061 ceph-mon[98573]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 555 op/s 2023-10-18T09:26:25.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:25 smithi170 ceph-mon[135150]: pgmap v594: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.9 MiB/s wr, 555 op/s 2023-10-18T09:26:28.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:27 smithi170 ceph-mon[135150]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.8 MiB/s wr, 526 op/s 2023-10-18T09:26:28.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:27 smithi110 ceph-mon[134818]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.8 MiB/s wr, 526 op/s 2023-10-18T09:26:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:27 smithi061 ceph-mon[98573]: pgmap v595: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.8 MiB/s wr, 526 op/s 2023-10-18T09:26:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 435.0 s, 563.2 tps, lat 178.689 ms stddev 201.533 2023-10-18T09:26:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:29 smithi110 ceph-mon[134818]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 607 op/s 2023-10-18T09:26:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:29 smithi061 ceph-mon[98573]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 607 op/s 2023-10-18T09:26:29.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:29 smithi170 ceph-mon[135150]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.6 MiB/s wr, 607 op/s 2023-10-18T09:26:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:31 smithi110 ceph-mon[134818]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-10-18T09:26:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:31 smithi061 ceph-mon[98573]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-10-18T09:26:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:31 smithi170 ceph-mon[135150]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2023-10-18T09:26:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:33 smithi110 ceph-mon[134818]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.1 MiB/s wr, 564 op/s 2023-10-18T09:26:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:33 smithi061 ceph-mon[98573]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.1 MiB/s wr, 564 op/s 2023-10-18T09:26:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:33 smithi170 ceph-mon[135150]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.1 MiB/s wr, 564 op/s 2023-10-18T09:26:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 440.0 s, 543.6 tps, lat 180.773 ms stddev 197.786 2023-10-18T09:26:35.470 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:35 smithi110 ceph-mon[134818]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 560 op/s 2023-10-18T09:26:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:35 smithi061 ceph-mon[98573]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 560 op/s 2023-10-18T09:26:35.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:35 smithi170 ceph-mon[135150]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.1 MiB/s wr, 560 op/s 2023-10-18T09:26:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:37 smithi170 ceph-mon[135150]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2023-10-18T09:26:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:37 smithi110 ceph-mon[134818]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2023-10-18T09:26:38.258 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:37 smithi061 ceph-mon[98573]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.2 MiB/s wr, 528 op/s 2023-10-18T09:26:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 445.0 s, 602.2 tps, lat 169.197 ms stddev 203.179 2023-10-18T09:26:39.466 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:39 smithi110 ceph-mon[134818]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2023-10-18T09:26:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:39 smithi061 ceph-mon[98573]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2023-10-18T09:26:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:39 smithi170 ceph-mon[135150]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 598 op/s 2023-10-18T09:26:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:41 smithi061 ceph-mon[98573]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 505 op/s 2023-10-18T09:26:41.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:41 smithi170 ceph-mon[135150]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 505 op/s 2023-10-18T09:26:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:41 smithi110 ceph-mon[134818]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.6 MiB/s wr, 505 op/s 2023-10-18T09:26:43.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:43 smithi110 ceph-mon[134818]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 566 op/s 2023-10-18T09:26:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:43 smithi061 ceph-mon[98573]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 566 op/s 2023-10-18T09:26:43.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:43 smithi170 ceph-mon[135150]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.2 MiB/s wr, 566 op/s 2023-10-18T09:26:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 450.0 s, 534.2 tps, lat 184.701 ms stddev 280.051 2023-10-18T09:26:45.722 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:45 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:45.722 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:45 smithi110 ceph-mon[134818]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 561 op/s 2023-10-18T09:26:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:45 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:45 smithi061 ceph-mon[98573]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 561 op/s 2023-10-18T09:26:45.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:45 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:26:45.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:45 smithi170 ceph-mon[135150]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 561 op/s 2023-10-18T09:26:48.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:47 smithi170 ceph-mon[135150]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-10-18T09:26:48.216 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:47 smithi110 ceph-mon[134818]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-10-18T09:26:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:47 smithi061 ceph-mon[98573]: pgmap v605: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.0 MiB/s wr, 526 op/s 2023-10-18T09:26:49.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 455.0 s, 546.8 tps, lat 184.176 ms stddev 212.695 2023-10-18T09:26:49.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:49 smithi110 ceph-mon[134818]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-10-18T09:26:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:49 smithi061 ceph-mon[98573]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-10-18T09:26:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:49 smithi170 ceph-mon[135150]: pgmap v606: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2023-10-18T09:26:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:51 smithi110 ceph-mon[134818]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2023-10-18T09:26:51.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:51 smithi061 ceph-mon[98573]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2023-10-18T09:26:51.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:51 smithi170 ceph-mon[135150]: pgmap v607: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.7 MiB/s wr, 513 op/s 2023-10-18T09:26:52.840 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:26:53.361 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1913623680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 msgr2=0x7fd708102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1913623680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708102df0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd704009990 tx=0x7fd704009ca0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1913623680 shutdown_connections 2023-10-18T09:26:53.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1913623680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.362 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1913623680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708102df0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1913623680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 0x7fd708104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1913623680 >> 172.21.15.61:0/1913623680 conn(0x7fd7080fdf40 msgr2=0x7fd708100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1913623680 shutdown_connections 2023-10-18T09:26:53.363 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.359+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1913623680 wait complete. 2023-10-18T09:26:53.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.360+0000 7fd70f9ff700 1 Processor -- start 2023-10-18T09:26:53.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.360+0000 7fd70f9ff700 1 -- start start 2023-10-18T09:26:53.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.360+0000 7fd70f9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.360+0000 7fd70f9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 0x7fd708198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70f9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70f9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081a0440 con 0x7fd708103bc0 2023-10-18T09:26:53.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70f9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081a0580 con 0x7fd708104560 2023-10-18T09:26:53.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70f9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081a0710 con 0x7fd7081029d0 2023-10-18T09:26:53.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:59506/0 (socket says 172.21.15.61:59506) 2023-10-18T09:26:53.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70d79b700 1 -- 172.21.15.61:0/1568504748 learned_addr learned my addr 172.21.15.61:0/1568504748 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:53.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 0x7fd708198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.361+0000 7fd70df9c700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 -- 172.21.15.61:0/1568504748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 msgr2=0x7fd70819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 -- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 msgr2=0x7fd708198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 0x7fd708198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 -- 172.21.15.61:0/1568504748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7040095f0 con 0x7fd7081029d0 2023-10-18T09:26:53.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70df9c700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:26:53.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.362+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd704006010 tx=0x7fd704003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.363+0000 7fd6fe7fc700 1 -- 172.21.15.61:0/1568504748 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70401d070 con 0x7fd7081029d0 2023-10-18T09:26:53.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.363+0000 7fd6fe7fc700 1 -- 172.21.15.61:0/1568504748 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd704003e30 con 0x7fd7081029d0 2023-10-18T09:26:53.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.363+0000 7fd6fe7fc700 1 -- 172.21.15.61:0/1568504748 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70400fe70 con 0x7fd7081029d0 2023-10-18T09:26:53.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.363+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7081a0990 con 0x7fd7081029d0 2023-10-18T09:26:53.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.363+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7081a0d60 con 0x7fd7081029d0 2023-10-18T09:26:53.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.368+0000 7fd6fe7fc700 1 -- 172.21.15.61:0/1568504748 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd70400f560 con 0x7fd7081029d0 2023-10-18T09:26:53.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.370+0000 7fd6fe7fc700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 0x7fd6f40a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.370+0000 7fd6fe7fc700 1 -- 172.21.15.61:0/1568504748 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd7040ce220 con 0x7fd7081029d0 2023-10-18T09:26:53.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.370+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 0x7fd6f40a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.371+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 0x7fd6f40a3b10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8005950 tx=0x7fd6f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.380+0000 7fd7232c4b80 1 Processor -- start 2023-10-18T09:26:53.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 -- start start 2023-10-18T09:26:53.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 0x5648478eca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.384 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484785f6e0 con 0x5648478edc30 2023-10-18T09:26:53.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847845c60 con 0x5648478ed2d0 2023-10-18T09:26:53.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.381+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847843580 con 0x5648478ec5d0 2023-10-18T09:26:53.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.382+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.382+0000 7fd70df9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 0x5648478eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.382+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50474/0 (socket says 172.21.15.61:50474) 2023-10-18T09:26:53.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.382+0000 7fd70d79b700 1 -- 172.21.15.61:0/570102471 learned_addr learned my addr 172.21.15.61:0/570102471 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:53.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70d79b700 1 -- 172.21.15.61:0/570102471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 msgr2=0x5648478eca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70d79b700 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 0x5648478eca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70d79b700 1 -- 172.21.15.61:0/570102471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 msgr2=0x5648478ec090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:26:53.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70d79b700 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70d79b700 1 -- 172.21.15.61:0/570102471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648477fc3f0 con 0x5648478ed2d0 2023-10-18T09:26:53.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.383+0000 7fd70df9c700 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 0x5648478eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:26:53.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.384+0000 7fd70d79b700 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd704097c90 tx=0x7fd704003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2c1591d3c8a2d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.384+0000 7fd6f37fe700 1 -- 172.21.15.61:0/570102471 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704020e50 con 0x5648478ed2d0 2023-10-18T09:26:53.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.384+0000 7fd6f37fe700 1 -- 172.21.15.61:0/570102471 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70409b920 con 0x5648478ed2d0 2023-10-18T09:26:53.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.384+0000 7fd6f37fe700 1 -- 172.21.15.61:0/570102471 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704020e50 con 0x5648478ed2d0 2023-10-18T09:26:53.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 -- 172.21.15.61:0/570102471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 msgr2=0x5648478ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd704097c90 tx=0x7fd704003680 comp rx=0 tx=0).stop 2023-10-18T09:26:53.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 -- 172.21.15.61:0/570102471 shutdown_connections 2023-10-18T09:26:53.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ec5d0 0x5648478eca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ed2d0 0x5648478ed6f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/570102471 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 -- 172.21.15.61:0/570102471 >> 172.21.15.61:0/570102471 conn(0x564847860720 msgr2=0x5648478e6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 -- 172.21.15.61:0/570102471 shutdown_connections 2023-10-18T09:26:53.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.385+0000 7fd7232c4b80 1 -- 172.21.15.61:0/570102471 wait complete. 2023-10-18T09:26:53.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.386+0000 7fd7232c4b80 1 -- start start 2023-10-18T09:26:53.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.386+0000 7fd7232c4b80 1 Processor -- start 2023-10-18T09:26:53.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484783ee70 con 0x5648478edc30 2023-10-18T09:26:53.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648477db710 con 0x5648478ec5d0 2023-10-18T09:26:53.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648477db3b0 con 0x5648478ed2d0 2023-10-18T09:26:53.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd7232c4b80 10 client.0 _ll_register_callbacks cb 0x564847725a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:26:53.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50486/0 (socket says 172.21.15.61:50486) 2023-10-18T09:26:53.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70d79b700 1 -- 172.21.15.61:0/1086869349 learned_addr learned my addr 172.21.15.61:0/1086869349 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:53.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70df9c700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70d79b700 1 -- 172.21.15.61:0/1086869349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 msgr2=0x5648478f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.387+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd70d79b700 1 -- 172.21.15.61:0/1086869349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 msgr2=0x5648478fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd70d79b700 1 -- 172.21.15.61:0/1086869349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70000f040 con 0x5648478ec5d0 2023-10-18T09:26:53.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd70df9c700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:26:53.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd70d79b700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd70409af20 tx=0x7fd70400f4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cfb3ba710276b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd6f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:26:53.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b9070 con 0x5648478ec5d0 2023-10-18T09:26:53.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd704020e50 con 0x5648478ec5d0 2023-10-18T09:26:53.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564847843ef0 con 0x5648478ec5d0 2023-10-18T09:26:53.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd70409a920 con 0x5648478ec5d0 2023-10-18T09:26:53.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd7232c4b80 10 client.26297 fetch_fsmap learned FSMap version 43 2023-10-18T09:26:53.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fd7081a0d60 con 0x5648478ec5d0 2023-10-18T09:26:53.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.388+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564847843ef0 con 0x5648478ec5d0 2023-10-18T09:26:53.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd704097010 con 0x5648478ec5d0 2023-10-18T09:26:53.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 10 client.26297 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:26:53.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 10 client.26297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:26:53.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 20 client.26297 populate_metadata read hostname 'smithi061' 2023-10-18T09:26:53.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564847901070 0x564847903490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 4 client.26297 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:26:53.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648478bb8b0 con 0x564847901070 2023-10-18T09:26:53.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.389+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564847901070 0x564847903490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.390+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564847901070 0x564847903490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7596d786309ee2 server_cookie=7302f55cc26b421a in_seq=0 out_seq=0 2023-10-18T09:26:53.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.390+0000 7fd6f0ff9700 10 client.26297 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:53.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.390+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/1086869349 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5648478bb8b0 con 0x564847901070 2023-10-18T09:26:53.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.390+0000 7fd6f0ff9700 10 client.26297 handle_command_reply: tid=0 2023-10-18T09:26:53.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 client.26297 shutdown 2023-10-18T09:26:53.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 msgr2=0x5648478f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd70409af20 tx=0x7fd70400f4a0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 shutdown_connections 2023-10-18T09:26:53.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564847901070 0x564847903490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478edc30 0x5648478fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478ec5d0 0x5648478f5800 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/1086869349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 >> 172.21.15.61:0/1086869349 conn(0x564847860720 msgr2=0x5648478eb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd6f0ff9700 0 client.26297 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:53.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 shutdown_connections 2023-10-18T09:26:53.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 1 -- 172.21.15.61:0/1086869349 wait complete. 2023-10-18T09:26:53.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.391+0000 7fd7232c4b80 20 client.26297 trim_cache size 0 max 16384 2023-10-18T09:26:53.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 Processor -- start 2023-10-18T09:26:53.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 -- start start 2023-10-18T09:26:53.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648476bf920 con 0x5648478ec5d0 2023-10-18T09:26:53.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648478bb8b0 con 0x5648478edc30 2023-10-18T09:26:53.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.426+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564847843ef0 con 0x5648478ed2d0 2023-10-18T09:26:53.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.427+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.427+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56458/0 (socket says 172.21.15.61:56458) 2023-10-18T09:26:53.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.427+0000 7fd70d79b700 1 -- 172.21.15.61:0/169646481 learned_addr learned my addr 172.21.15.61:0/169646481 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:53.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.427+0000 7fd70df9c700 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.428+0000 7fd70d79b700 1 -- 172.21.15.61:0/169646481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 msgr2=0x5648478fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.428+0000 7fd70d79b700 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.428+0000 7fd70d79b700 1 -- 172.21.15.61:0/169646481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 msgr2=0x5648478f08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.428+0000 7fd70d79b700 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478f08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.428+0000 7fd70d79b700 1 -- 172.21.15.61:0/169646481 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564847669de0 con 0x5648478ec5d0 2023-10-18T09:26:53.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.429+0000 7fd70d79b700 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd704097720 tx=0x7fd704003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceabe817b59c26f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.430+0000 7fd6f3fff700 1 -- 172.21.15.61:0/169646481 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b9040 con 0x5648478ec5d0 2023-10-18T09:26:53.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.430+0000 7fd6f3fff700 1 -- 172.21.15.61:0/169646481 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70409f120 con 0x5648478ec5d0 2023-10-18T09:26:53.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.430+0000 7fd6f3fff700 1 -- 172.21.15.61:0/169646481 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b9040 con 0x5648478ec5d0 2023-10-18T09:26:53.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 -- 172.21.15.61:0/169646481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 msgr2=0x5648478ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd704097720 tx=0x7fd704003680 comp rx=0 tx=0).stop 2023-10-18T09:26:53.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 -- 172.21.15.61:0/169646481 shutdown_connections 2023-10-18T09:26:53.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478f08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478ec090 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/169646481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 -- 172.21.15.61:0/169646481 >> 172.21.15.61:0/169646481 conn(0x564847860720 msgr2=0x5648478f3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 -- 172.21.15.61:0/169646481 shutdown_connections 2023-10-18T09:26:53.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.431+0000 7fd7232c4b80 1 -- 172.21.15.61:0/169646481 wait complete. 2023-10-18T09:26:53.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.433+0000 7fd7232c4b80 1 -- start start 2023-10-18T09:26:53.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.433+0000 7fd7232c4b80 1 Processor -- start 2023-10-18T09:26:53.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648478fbe30 con 0x5648478ec5d0 2023-10-18T09:26:53.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648478fbfa0 con 0x5648478edc30 2023-10-18T09:26:53.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56484796e670 con 0x5648478ed2d0 2023-10-18T09:26:53.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.434+0000 7fd7232c4b80 10 client.0 _ll_register_callbacks cb 0x564847725a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:26:53.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:56472/0 (socket says 172.21.15.61:56472) 2023-10-18T09:26:53.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 -- 172.21.15.61:0/2388838036 learned_addr learned my addr 172.21.15.61:0/2388838036 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:26:53.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 -- 172.21.15.61:0/2388838036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 msgr2=0x5648478fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 -- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 msgr2=0x5648478fb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:26:53.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.436+0000 7fd70d79b700 1 -- 172.21.15.61:0/2388838036 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70000f040 con 0x5648478ec5d0 2023-10-18T09:26:53.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.437+0000 7fd6f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:26:53.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.437+0000 7fd70d79b700 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd70400f8b0 tx=0x7fd70400f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31bdac6670343bb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:26:53.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.438+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040b9040 con 0x5648478ec5d0 2023-10-18T09:26:53.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.438+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd704020e50 con 0x5648478ec5d0 2023-10-18T09:26:53.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.439+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd70409b920 con 0x5648478ec5d0 2023-10-18T09:26:53.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.440+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7040b9040 con 0x5648478ec5d0 2023-10-18T09:26:53.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.440+0000 7fd7232c4b80 10 client.16119 fetch_fsmap learned FSMap version 43 2023-10-18T09:26:53.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.440+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x564847652c60 con 0x5648478ec5d0 2023-10-18T09:26:53.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.440+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd70409ad60 con 0x5648478ec5d0 2023-10-18T09:26:53.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd70409a3e0 con 0x5648478ec5d0 2023-10-18T09:26:53.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 10 client.16119 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:26:53.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:26:53.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 20 client.16119 populate_metadata read hostname 'smithi061' 2023-10-18T09:26:53.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56484796f9b0 0x564847971da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:26:53.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 4 client.16119 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:26:53.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.441+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fd70409b920 con 0x56484796f9b0 2023-10-18T09:26:53.448 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.444+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56484796f9b0 0x564847971da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:26:53.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd6f0ff9700 10 client.16119 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:53.449 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd70cf9a700 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56484796f9b0 0x564847971da0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da40842e1a4d714e server_cookie=4889e9830a5cd9df in_seq=0 out_seq=0 2023-10-18T09:26:53.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd6f0ff9700 1 -- 172.21.15.61:0/2388838036 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7fd70409b920 con 0x56484796f9b0 2023-10-18T09:26:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd6f0ff9700 10 client.16119 handle_command_reply: tid=0 2023-10-18T09:26:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd7232c4b80 1 client.16119 shutdown 2023-10-18T09:26:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 msgr2=0x5648478fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.447+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd70400f8b0 tx=0x7fd70400f900 comp rx=0 tx=0).stop 2023-10-18T09:26:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 shutdown_connections 2023-10-18T09:26:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56484796f9b0 0x564847971da0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5648478edc30 0x5648478fb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5648478ec5d0 0x5648478fcbb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd6f0ff9700 0 client.16119 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:26:53.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 --2- 172.21.15.61:0/2388838036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648478ed2d0 0x5648478fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 >> 172.21.15.61:0/2388838036 conn(0x564847860720 msgr2=0x5648478e6810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 shutdown_connections 2023-10-18T09:26:53.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 1 -- 172.21.15.61:0/2388838036 wait complete. 2023-10-18T09:26:53.453 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.448+0000 7fd7232c4b80 20 client.16119 trim_cache size 0 max 16384 2023-10-18T09:26:53.454 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:26:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:26:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:26:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:26:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:26:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:26:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:26:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:26:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:26:53.456 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:26:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 msgr2=0x7fd6f40a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 0x7fd6f40a3b10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8005950 tx=0x7fd6f800b410 comp rx=0 tx=0).stop 2023-10-18T09:26:53.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 msgr2=0x7fd708198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:26:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd704006010 tx=0x7fd704003b90 comp rx=0 tx=0).stop 2023-10-18T09:26:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 shutdown_connections 2023-10-18T09:26:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd6f40a1650 0x7fd6f40a3b10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd708104560 0x7fd70819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7081029d0 0x7fd708198910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 --2- 172.21.15.61:0/1568504748 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd708103bc0 0x7fd708198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:26:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.449+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 >> 172.21.15.61:0/1568504748 conn(0x7fd7080fdf40 msgr2=0x7fd708106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:26:53.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.450+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 shutdown_connections 2023-10-18T09:26:53.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:26:53.450+0000 7fd70f9ff700 1 -- 172.21.15.61:0/1568504748 wait complete. 2023-10-18T09:26:53.484 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:53 smithi061 ceph-mon[98573]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.3 MiB/s wr, 559 op/s 2023-10-18T09:26:53.488 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:26:53.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:53 smithi170 ceph-mon[135150]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.3 MiB/s wr, 559 op/s 2023-10-18T09:26:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:53 smithi110 ceph-mon[134818]: pgmap v608: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 6.3 MiB/s wr, 559 op/s 2023-10-18T09:26:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 460.0 s, 498.4 tps, lat 198.446 ms stddev 212.264 2023-10-18T09:26:55.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:55 smithi110 ceph-mon[134818]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 545 op/s 2023-10-18T09:26:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:55 smithi061 ceph-mon[98573]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 545 op/s 2023-10-18T09:26:55.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:55 smithi170 ceph-mon[135150]: pgmap v609: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 6.2 MiB/s wr, 545 op/s 2023-10-18T09:26:58.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:57 smithi170 ceph-mon[135150]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 515 op/s 2023-10-18T09:26:58.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:57 smithi110 ceph-mon[134818]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 515 op/s 2023-10-18T09:26:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:57 smithi061 ceph-mon[98573]: pgmap v610: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 515 op/s 2023-10-18T09:26:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 465.0 s, 479.2 tps, lat 209.856 ms stddev 246.649 2023-10-18T09:26:59.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:26:59 smithi110 ceph-mon[134818]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-18T09:26:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:26:59 smithi061 ceph-mon[98573]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-18T09:26:59.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:26:59 smithi170 ceph-mon[135150]: pgmap v611: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 6.5 MiB/s wr, 570 op/s 2023-10-18T09:27:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:01 smithi110 ceph-mon[134818]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-18T09:27:01.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:01 smithi061 ceph-mon[98573]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-18T09:27:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:01 smithi170 ceph-mon[135150]: pgmap v612: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.6 MiB/s wr, 496 op/s 2023-10-18T09:27:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:03 smithi061 ceph-mon[98573]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:03.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:03 smithi170 ceph-mon[135150]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:03 smithi110 ceph-mon[134818]: pgmap v613: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 470.0 s, 483.8 tps, lat 209.052 ms stddev 218.163 2023-10-18T09:27:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:05 smithi110 ceph-mon[134818]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2023-10-18T09:27:05.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:05 smithi061 ceph-mon[98573]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2023-10-18T09:27:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:05 smithi170 ceph-mon[135150]: pgmap v614: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.5 MiB/s wr, 498 op/s 2023-10-18T09:27:08.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:07 smithi170 ceph-mon[135150]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2023-10-18T09:27:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:07 smithi110 ceph-mon[134818]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2023-10-18T09:27:08.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:07 smithi061 ceph-mon[98573]: pgmap v615: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 4.8 MiB/s wr, 458 op/s 2023-10-18T09:27:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 475.0 s, 436.8 tps, lat 227.344 ms stddev 258.996 2023-10-18T09:27:09.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:09 smithi110 ceph-mon[134818]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 476 op/s 2023-10-18T09:27:09.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:09 smithi061 ceph-mon[98573]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 476 op/s 2023-10-18T09:27:09.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:09 smithi170 ceph-mon[135150]: pgmap v616: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.1 MiB/s wr, 476 op/s 2023-10-18T09:27:10.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:10 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:27:10.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:10 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:27:10.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:10 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:27:11.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:11 smithi061 ceph-mon[98573]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.7 MiB/s wr, 455 op/s 2023-10-18T09:27:11.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:11 smithi170 ceph-mon[135150]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.7 MiB/s wr, 455 op/s 2023-10-18T09:27:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:11 smithi110 ceph-mon[134818]: pgmap v617: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.7 MiB/s wr, 455 op/s 2023-10-18T09:27:13.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:27:13.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:27:13.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:13 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:27:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:27:13.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:27:13.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:13 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:27:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:27:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:27:13.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:13 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:27:14.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 480.0 s, 560.5 tps, lat 179.824 ms stddev 202.639 2023-10-18T09:27:14.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:14 smithi170 ceph-mon[135150]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:27:14.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:14 smithi110 ceph-mon[134818]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:27:14.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:14 smithi061 ceph-mon[98573]: pgmap v618: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:27:15.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:15 smithi061 ceph-mon[98573]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 5.3 MiB/s wr, 470 op/s 2023-10-18T09:27:15.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:15 smithi170 ceph-mon[135150]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 5.3 MiB/s wr, 470 op/s 2023-10-18T09:27:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:15 smithi110 ceph-mon[134818]: pgmap v619: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 5.3 MiB/s wr, 470 op/s 2023-10-18T09:27:18.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:17 smithi170 ceph-mon[135150]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:17 smithi110 ceph-mon[134818]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:17 smithi061 ceph-mon[98573]: pgmap v620: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 485.0 s, 604.3 tps, lat 167.248 ms stddev 198.601 2023-10-18T09:27:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:19 smithi061 ceph-mon[98573]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:19 smithi170 ceph-mon[135150]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:19.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:19 smithi110 ceph-mon[134818]: pgmap v621: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 5.4 MiB/s wr, 492 op/s 2023-10-18T09:27:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:21 smithi110 ceph-mon[134818]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-10-18T09:27:21.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:21 smithi061 ceph-mon[98573]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-10-18T09:27:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:21 smithi170 ceph-mon[135150]: pgmap v622: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 6.2 MiB/s wr, 567 op/s 2023-10-18T09:27:23.489 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:27:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:23 smithi061 ceph-mon[98573]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:27:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:23 smithi170 ceph-mon[135150]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:27:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:23 smithi110 ceph-mon[134818]: pgmap v623: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:27:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.266+0000 7fc3539ff700 1 -- 172.21.15.61:0/3719024953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c106470 msgr2=0x7fc34c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.266+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3719024953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c106470 0x7fc34c1088e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc344009f90 tx=0x7fc34400b330 comp rx=0 tx=0).stop 2023-10-18T09:27:24.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 -- 172.21.15.61:0/3719024953 shutdown_connections 2023-10-18T09:27:24.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3719024953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c106470 0x7fc34c1088e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3719024953 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c101210 0x7fc34c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3719024953 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c103b40 0x7fc34c105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 -- 172.21.15.61:0/3719024953 >> 172.21.15.61:0/3719024953 conn(0x7fc34c0fac00 msgr2=0x7fc34c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 -- 172.21.15.61:0/3719024953 shutdown_connections 2023-10-18T09:27:24.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.267+0000 7fc3539ff700 1 -- 172.21.15.61:0/3719024953 wait complete. 2023-10-18T09:27:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.268+0000 7fc3539ff700 1 Processor -- start 2023-10-18T09:27:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 -- start start 2023-10-18T09:27:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 0x7fc34c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c106470 0x7fc34c19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c1a0560 con 0x7fc34c103b40 2023-10-18T09:27:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c1a06a0 con 0x7fc34c106470 2023-10-18T09:27:24.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.269+0000 7fc3539ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c1a0830 con 0x7fc34c101210 2023-10-18T09:27:24.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:38860/0 (socket says 172.21.15.61:38860) 2023-10-18T09:27:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 -- 172.21.15.61:0/3427635894 learned_addr learned my addr 172.21.15.61:0/3427635894 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc351f9c700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c106470 0x7fc34c19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc350f9a700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 0x7fc34c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 -- 172.21.15.61:0/3427635894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c106470 msgr2=0x7fc34c19ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c106470 0x7fc34c19ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 -- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 msgr2=0x7fc34c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 0x7fc34c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc35179b700 1 -- 172.21.15.61:0/3427635894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc348009520 con 0x7fc34c101210 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.270+0000 7fc350f9a700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 0x7fc34c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:27:24.276 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.271+0000 7fc35179b700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e9c0 tx=0x7fc33c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.272+0000 7fc3427fc700 1 -- 172.21.15.61:0/3427635894 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00cbd0 con 0x7fc34c101210 2023-10-18T09:27:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.272+0000 7fc3427fc700 1 -- 172.21.15.61:0/3427635894 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc33c00eeb0 con 0x7fc34c101210 2023-10-18T09:27:24.277 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.272+0000 7fc3427fc700 1 -- 172.21.15.61:0/3427635894 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00cbd0 con 0x7fc34c101210 2023-10-18T09:27:24.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.272+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344009c40 con 0x7fc34c101210 2023-10-18T09:27:24.278 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.273+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c1a0e40 con 0x7fc34c101210 2023-10-18T09:27:24.282 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.279+0000 7fc3427fc700 1 -- 172.21.15.61:0/3427635894 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc33c0052e0 con 0x7fc34c101210 2023-10-18T09:27:24.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.282+0000 7fc3427fc700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 0x7fc3380a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.283+0000 7fc3427fc700 1 -- 172.21.15.61:0/3427635894 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc33c014070 con 0x7fc34c101210 2023-10-18T09:27:24.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.283+0000 7fc350f9a700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 0x7fc3380a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.284+0000 7fc350f9a700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 0x7fc3380a3a20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d70 tx=0x7fc348009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.288+0000 7fc367388b80 1 Processor -- start 2023-10-18T09:27:24.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.288+0000 7fc367388b80 1 -- start start 2023-10-18T09:27:24.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb2fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb2fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb2686e0 con 0x55c9cb2feae0 2023-10-18T09:27:24.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb24ec60 con 0x55c9cb2fdde0 2023-10-18T09:27:24.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc367388b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb24c580 con 0x55c9cb2ff440 2023-10-18T09:27:24.292 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb2fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb2fd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:38874/0 (socket says 172.21.15.61:38874) 2023-10-18T09:27:24.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc350f9a700 1 -- 172.21.15.61:0/1065417530 learned_addr learned my addr 172.21.15.61:0/1065417530 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:24.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.289+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 -- 172.21.15.61:0/1065417530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 msgr2=0x55c9cb2fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb2fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 -- 172.21.15.61:0/1065417530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 msgr2=0x55c9cb2fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb2fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 -- 172.21.15.61:0/1065417530 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9cb2053f0 con 0x55c9cb2fdde0 2023-10-18T09:27:24.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fe260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc344007900 tx=0x7fc344009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22485222eb2ac57 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.290+0000 7fc3377fe700 1 -- 172.21.15.61:0/1065417530 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400e040 con 0x55c9cb2fdde0 2023-10-18T09:27:24.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.291+0000 7fc3377fe700 1 -- 172.21.15.61:0/1065417530 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc344007620 con 0x55c9cb2fdde0 2023-10-18T09:27:24.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.291+0000 7fc3377fe700 1 -- 172.21.15.61:0/1065417530 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34400bc50 con 0x55c9cb2fdde0 2023-10-18T09:27:24.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.291+0000 7fc367388b80 1 -- 172.21.15.61:0/1065417530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 msgr2=0x55c9cb2fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.291+0000 7fc367388b80 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fe260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc344007900 tx=0x7fc344009b40 comp rx=0 tx=0).stop 2023-10-18T09:27:24.297 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 -- 172.21.15.61:0/1065417530 shutdown_connections 2023-10-18T09:27:24.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fe260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb2fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 --2- 172.21.15.61:0/1065417530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb2fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 -- 172.21.15.61:0/1065417530 >> 172.21.15.61:0/1065417530 conn(0x55c9cb269720 msgr2=0x55c9cb2efb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.298 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 -- 172.21.15.61:0/1065417530 shutdown_connections 2023-10-18T09:27:24.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.292+0000 7fc367388b80 1 -- 172.21.15.61:0/1065417530 wait complete. 2023-10-18T09:27:24.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.293+0000 7fc367388b80 1 -- start start 2023-10-18T09:27:24.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.293+0000 7fc367388b80 1 Processor -- start 2023-10-18T09:27:24.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.294+0000 7fc367388b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2fdde0 0x55c9cb307340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.294+0000 7fc367388b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.294+0000 7fc367388b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2ff440 0x55c9cb30e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:38888/0 (socket says 172.21.15.61:38888) 2023-10-18T09:27:24.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 -- 172.21.15.61:0/1061403557 learned_addr learned my addr 172.21.15.61:0/1061403557 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:24.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc367388b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb247e70 con 0x55c9cb2fdde0 2023-10-18T09:27:24.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb1e4710 con 0x55c9cb2ff440 2023-10-18T09:27:24.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb1e43b0 con 0x55c9cb2feae0 2023-10-18T09:27:24.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2ff440 0x55c9cb30e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc35179b700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2fdde0 0x55c9cb307340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc367388b80 10 client.0 _ll_register_callbacks cb 0x55c9cb12ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:27:24.302 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 -- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2ff440 msgr2=0x55c9cb30e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2ff440 0x55c9cb30e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 -- 172.21.15.61:0/1061403557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2fdde0 msgr2=0x55c9cb307340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2fdde0 0x55c9cb307340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.295+0000 7fc350f9a700 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c1094d0 con 0x55c9cb2feae0 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.296+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3480126d0 tx=0x7fc348004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bad89464d691fec server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.303 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.296+0000 7fc334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:27:24.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.310+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348020840 con 0x55c9cb2feae0 2023-10-18T09:27:24.313 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.311+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc3480209a0 con 0x55c9cb2feae0 2023-10-18T09:27:24.314 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.312+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9cb24cef0 con 0x55c9cb2feae0 2023-10-18T09:27:24.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.316+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc348010ec0 con 0x55c9cb2feae0 2023-10-18T09:27:24.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.316+0000 7fc367388b80 10 client.26263 fetch_fsmap learned FSMap version 43 2023-10-18T09:27:24.319 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.316+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c9cb072de0 con 0x55c9cb2feae0 2023-10-18T09:27:24.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c9cb24cef0 con 0x55c9cb2feae0 2023-10-18T09:27:24.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc348005240 con 0x55c9cb2feae0 2023-10-18T09:27:24.320 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 10 client.26263 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:27:24.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:27:24.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 20 client.26263 populate_metadata read hostname 'smithi061' 2023-10-18T09:27:24.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb3131c0 0x55c9cb3155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 4 client.26263 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:27:24.321 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.317+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9cb2c48b0 con 0x55c9cb3131c0 2023-10-18T09:27:24.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.319+0000 7fc35179b700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb3131c0 0x55c9cb3155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.320+0000 7fc35179b700 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb3131c0 0x55c9cb3155e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b31007baf699b1eb server_cookie=6aaed3fe4d05f718 in_seq=0 out_seq=0 2023-10-18T09:27:24.322 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.320+0000 7fc334ff9700 10 client.26263 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:24.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.323+0000 7fc334ff9700 1 -- 172.21.15.61:0/1061403557 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c9cb2c48b0 con 0x55c9cb3131c0 2023-10-18T09:27:24.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.323+0000 7fc334ff9700 10 client.26263 handle_command_reply: tid=0 2023-10-18T09:27:24.325 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.323+0000 7fc367388b80 1 client.26263 shutdown 2023-10-18T09:27:24.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.325+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 msgr2=0x55c9cb307880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.327 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.325+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3480126d0 tx=0x7fc348004750 comp rx=0 tx=0).stop 2023-10-18T09:27:24.328 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 shutdown_connections 2023-10-18T09:27:24.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb3131c0 0x55c9cb3155e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2ff440 0x55c9cb30e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.329 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2fdde0 0x55c9cb307340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 --2- 172.21.15.61:0/1061403557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2feae0 0x55c9cb307880 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.326+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 >> 172.21.15.61:0/1061403557 conn(0x55c9cb269720 msgr2=0x55c9cb2efb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.330 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.327+0000 7fc334ff9700 0 client.26263 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:24.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.328+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 shutdown_connections 2023-10-18T09:27:24.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.328+0000 7fc367388b80 1 -- 172.21.15.61:0/1061403557 wait complete. 2023-10-18T09:27:24.331 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.328+0000 7fc367388b80 20 client.26263 trim_cache size 0 max 16384 2023-10-18T09:27:24.341 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 490.0 s, 545.6 tps, lat 175.096 ms stddev 208.539 2023-10-18T09:27:24.364 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.362+0000 7fc367388b80 1 Processor -- start 2023-10-18T09:27:24.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.362+0000 7fc367388b80 1 -- start start 2023-10-18T09:27:24.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.365 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb30e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb306020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb0c8920 con 0x55c9cb2feae0 2023-10-18T09:27:24.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb306eb0 con 0x55c9cb2fdde0 2023-10-18T09:27:24.366 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc367388b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb2f00c0 con 0x55c9cb2ff440 2023-10-18T09:27:24.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.363+0000 7fc351f9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb306020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.367 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43698/0 (socket says 172.21.15.61:43698) 2023-10-18T09:27:24.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 -- 172.21.15.61:0/332277291 learned_addr learned my addr 172.21.15.61:0/332277291 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:24.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc350f9a700 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb30e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 -- 172.21.15.61:0/332277291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 msgr2=0x55c9cb306020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb306020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.368 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.364+0000 7fc35179b700 1 -- 172.21.15.61:0/332277291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 msgr2=0x55c9cb30e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.365+0000 7fc35179b700 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb30e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.365+0000 7fc35179b700 1 -- 172.21.15.61:0/332277291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9cb05bc60 con 0x55c9cb2fdde0 2023-10-18T09:27:24.369 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.365+0000 7fc350f9a700 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb30e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:27:24.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.366+0000 7fc35179b700 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0a1d30 tx=0x7fc33c0ca6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b97945aeabdce09 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.366+0000 7fc337fff700 1 -- 172.21.15.61:0/332277291 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c099ed0 con 0x55c9cb2fdde0 2023-10-18T09:27:24.370 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.366+0000 7fc337fff700 1 -- 172.21.15.61:0/332277291 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc33c09e020 con 0x55c9cb2fdde0 2023-10-18T09:27:24.371 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.366+0000 7fc337fff700 1 -- 172.21.15.61:0/332277291 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c099620 con 0x55c9cb2fdde0 2023-10-18T09:27:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 -- 172.21.15.61:0/332277291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 msgr2=0x55c9cb2fd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0a1d30 tx=0x7fc33c0ca6b0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 -- 172.21.15.61:0/332277291 shutdown_connections 2023-10-18T09:27:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2ff440 0x55c9cb306020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.372 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2fdde0 0x55c9cb2fd8a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 --2- 172.21.15.61:0/332277291 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2feae0 0x55c9cb30e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 -- 172.21.15.61:0/332277291 >> 172.21.15.61:0/332277291 conn(0x55c9cb269720 msgr2=0x55c9cb3052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 -- 172.21.15.61:0/332277291 shutdown_connections 2023-10-18T09:27:24.373 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.369+0000 7fc367388b80 1 -- 172.21.15.61:0/332277291 wait complete. 2023-10-18T09:27:24.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.370+0000 7fc367388b80 1 -- start start 2023-10-18T09:27:24.374 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.371+0000 7fc367388b80 1 Processor -- start 2023-10-18T09:27:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.371+0000 7fc367388b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2fdde0 0x55c9cb30e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.371+0000 7fc367388b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc367388b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2ff440 0x55c9cb30b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc367388b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb30bb20 con 0x55c9cb2ff440 2023-10-18T09:27:24.375 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc367388b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb30bc90 con 0x55c9cb2feae0 2023-10-18T09:27:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc367388b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cb378020 con 0x55c9cb2fdde0 2023-10-18T09:27:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc367388b80 10 client.0 _ll_register_callbacks cb 0x55c9cb12ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:27:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43708/0 (socket says 172.21.15.61:43708) 2023-10-18T09:27:24.376 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 -- 172.21.15.61:0/1650432437 learned_addr learned my addr 172.21.15.61:0/1650432437 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc35179b700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2fdde0 0x55c9cb30e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 -- 172.21.15.61:0/1650432437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2fdde0 msgr2=0x55c9cb30e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2fdde0 0x55c9cb30e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.377 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.372+0000 7fc350f9a700 1 -- 172.21.15.61:0/1650432437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2ff440 msgr2=0x55c9cb30b5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-18T09:27:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.374+0000 7fc351f9c700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2ff440 0x55c9cb30b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.378 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.374+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2ff440 0x55c9cb30b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.374+0000 7fc350f9a700 1 -- 172.21.15.61:0/1650432437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9cb072de0 con 0x55c9cb2feae0 2023-10-18T09:27:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.374+0000 7fc350f9a700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3480055a0 tx=0x7fc348004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d1247ad41fd496 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:27:24.379 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348020840 con 0x55c9cb2feae0 2023-10-18T09:27:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc348037070 con 0x55c9cb2feae0 2023-10-18T09:27:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc344007620 con 0x55c9cb2feae0 2023-10-18T09:27:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc348010ea0 con 0x55c9cb2feae0 2023-10-18T09:27:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.375+0000 7fc367388b80 10 client.26318 fetch_fsmap learned FSMap version 43 2023-10-18T09:27:24.380 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.376+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fc34c1a0e40 con 0x55c9cb2feae0 2023-10-18T09:27:24.381 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.376+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3480099e0 con 0x55c9cb2feae0 2023-10-18T09:27:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc348041060 con 0x55c9cb2feae0 2023-10-18T09:27:24.382 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 10 client.26318 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:27:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 10 client.26318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:27:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 20 client.26318 populate_metadata read hostname 'smithi061' 2023-10-18T09:27:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb379aa0 0x55c9cb37bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 4 client.26318 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:27:24.383 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fc344007620 con 0x55c9cb379aa0 2023-10-18T09:27:24.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.380+0000 7fc35179b700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb379aa0 0x55c9cb37bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:24.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc35179b700 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb379aa0 0x55c9cb37bec0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1433c097d283ca50 server_cookie=7199bf6d0367d947 in_seq=0 out_seq=0 2023-10-18T09:27:24.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc334ff9700 10 client.26318 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:24.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc334ff9700 1 -- 172.21.15.61:0/1650432437 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7fc344007620 con 0x55c9cb379aa0 2023-10-18T09:27:24.385 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc334ff9700 10 client.26318 handle_command_reply: tid=0 2023-10-18T09:27:24.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc367388b80 1 client.26318 shutdown 2023-10-18T09:27:24.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 msgr2=0x55c9cb30b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.381+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc3480055a0 tx=0x7fc348004b50 comp rx=0 tx=0).stop 2023-10-18T09:27:24.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 shutdown_connections 2023-10-18T09:27:24.386 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c9cb379aa0 0x55c9cb37bec0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c9cb2ff440 0x55c9cb30b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c9cb2fdde0 0x55c9cb30e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 --2- 172.21.15.61:0/1650432437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9cb2feae0 0x55c9cb30b060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 >> 172.21.15.61:0/1650432437 conn(0x55c9cb269720 msgr2=0x55c9cb304ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.387 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc334ff9700 0 client.26318 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:24.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 shutdown_connections 2023-10-18T09:27:24.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.382+0000 7fc367388b80 1 -- 172.21.15.61:0/1650432437 wait complete. 2023-10-18T09:27:24.388 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.383+0000 7fc367388b80 20 client.26318 trim_cache size 0 max 16384 2023-10-18T09:27:24.389 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:27:24.389 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:27:24.389 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:27:24.389 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:27:24.390 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:27:24.390 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:27:24.390 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:27:24.390 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:27:24.391 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:27:24.391 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:27:24.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 msgr2=0x7fc3380a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 0x7fc3380a3a20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc348009d70 tx=0x7fc348009260 comp rx=0 tx=0).stop 2023-10-18T09:27:24.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 msgr2=0x7fc34c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:24.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00e9c0 tx=0x7fc33c00ecd0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 shutdown_connections 2023-10-18T09:27:24.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc3380a1560 0x7fc3380a3a20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc34c106470 0x7fc34c19ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc34c101210 0x7fc34c198a30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 --2- 172.21.15.61:0/3427635894 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc34c103b40 0x7fc34c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:24.393 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.384+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 >> 172.21.15.61:0/3427635894 conn(0x7fc34c0fac00 msgr2=0x7fc34c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:24.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.385+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 shutdown_connections 2023-10-18T09:27:24.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:24.385+0000 7fc3539ff700 1 -- 172.21.15.61:0/3427635894 wait complete. 2023-10-18T09:27:24.422 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:27:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:25 smithi061 ceph-mon[98573]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2023-10-18T09:27:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:25 smithi170 ceph-mon[135150]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2023-10-18T09:27:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:25 smithi110 ceph-mon[134818]: pgmap v624: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 6.1 MiB/s wr, 568 op/s 2023-10-18T09:27:28.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:27 smithi170 ceph-mon[135150]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2023-10-18T09:27:28.214 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:27 smithi110 ceph-mon[134818]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2023-10-18T09:27:28.678 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:27 smithi061 ceph-mon[98573]: pgmap v625: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2023-10-18T09:27:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 495.0 s, 534.6 tps, lat 190.694 ms stddev 243.253 2023-10-18T09:27:29.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:29 smithi061 ceph-mon[98573]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2023-10-18T09:27:29.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:29 smithi170 ceph-mon[135150]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2023-10-18T09:27:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:29 smithi110 ceph-mon[134818]: pgmap v626: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2023-10-18T09:27:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:31 smithi061 ceph-mon[98573]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2023-10-18T09:27:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:31 smithi170 ceph-mon[135150]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2023-10-18T09:27:31.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:31 smithi110 ceph-mon[134818]: pgmap v627: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.8 MiB/s wr, 619 op/s 2023-10-18T09:27:33.509 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:33 smithi061 ceph-mon[98573]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2023-10-18T09:27:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:33 smithi170 ceph-mon[135150]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2023-10-18T09:27:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:33 smithi110 ceph-mon[134818]: pgmap v628: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2023-10-18T09:27:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 500.0 s, 547.4 tps, lat 187.792 ms stddev 206.285 2023-10-18T09:27:35.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:35 smithi061 ceph-mon[98573]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 554 op/s 2023-10-18T09:27:35.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:35 smithi170 ceph-mon[135150]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 554 op/s 2023-10-18T09:27:35.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:35 smithi110 ceph-mon[134818]: pgmap v629: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.8 MiB/s wr, 554 op/s 2023-10-18T09:27:38.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:37 smithi170 ceph-mon[135150]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2023-10-18T09:27:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:37 smithi110 ceph-mon[134818]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2023-10-18T09:27:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:37 smithi061 ceph-mon[98573]: pgmap v630: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.0 MiB/s wr, 574 op/s 2023-10-18T09:27:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 505.0 s, 510.4 tps, lat 190.045 ms stddev 219.258 2023-10-18T09:27:39.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:39 smithi061 ceph-mon[98573]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:39 smithi170 ceph-mon[135150]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:39.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:39 smithi110 ceph-mon[134818]: pgmap v631: 129 pgs: 129 active+clean; 1.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.3 MiB/s wr, 511 op/s 2023-10-18T09:27:41.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:41 smithi061 ceph-mon[98573]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.7 MiB/s wr, 584 op/s 2023-10-18T09:27:41.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:41 smithi170 ceph-mon[135150]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.7 MiB/s wr, 584 op/s 2023-10-18T09:27:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:41 smithi110 ceph-mon[134818]: pgmap v632: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.7 MiB/s wr, 584 op/s 2023-10-18T09:27:43.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:43 smithi061 ceph-mon[98573]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 498 op/s 2023-10-18T09:27:43.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:43 smithi170 ceph-mon[135150]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 498 op/s 2023-10-18T09:27:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:43 smithi110 ceph-mon[134818]: pgmap v633: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 498 op/s 2023-10-18T09:27:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 510.0 s, 554.0 tps, lat 178.675 ms stddev 192.048 2023-10-18T09:27:45.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:45 smithi061 ceph-mon[98573]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-10-18T09:27:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:45 smithi170 ceph-mon[135150]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-10-18T09:27:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:45 smithi110 ceph-mon[134818]: pgmap v634: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.3 MiB/s wr, 556 op/s 2023-10-18T09:27:48.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:47 smithi110 ceph-mon[134818]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2023-10-18T09:27:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:47 smithi061 ceph-mon[98573]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2023-10-18T09:27:48.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:47 smithi170 ceph-mon[135150]: pgmap v635: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2023-10-18T09:27:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 515.0 s, 520.0 tps, lat 197.414 ms stddev 196.162 2023-10-18T09:27:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:49 smithi061 ceph-mon[98573]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.6 MiB/s wr, 488 op/s 2023-10-18T09:27:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:49 smithi170 ceph-mon[135150]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.6 MiB/s wr, 488 op/s 2023-10-18T09:27:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:49 smithi110 ceph-mon[134818]: pgmap v636: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 4.6 MiB/s wr, 488 op/s 2023-10-18T09:27:51.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:51 smithi061 ceph-mon[98573]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.5 MiB/s wr, 561 op/s 2023-10-18T09:27:51.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:51 smithi170 ceph-mon[135150]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.5 MiB/s wr, 561 op/s 2023-10-18T09:27:51.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:51 smithi110 ceph-mon[134818]: pgmap v637: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.5 MiB/s wr, 561 op/s 2023-10-18T09:27:53.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:53 smithi061 ceph-mon[98573]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-10-18T09:27:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:53 smithi170 ceph-mon[135150]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-10-18T09:27:53.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:53 smithi110 ceph-mon[134818]: pgmap v638: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 4.7 MiB/s wr, 479 op/s 2023-10-18T09:27:54.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 520.0 s, 531.6 tps, lat 186.090 ms stddev 182.522 2023-10-18T09:27:54.423 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:27:54.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/2882154911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 msgr2=0x7fbf48100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.830 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/2882154911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf48100800 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009fc0 tx=0x7fbf4000b330 comp rx=0 tx=0).stop 2023-10-18T09:27:54.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/2882154911 shutdown_connections 2023-10-18T09:27:54.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/2882154911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf48100800 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/2882154911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf480ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/2882154911 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf480ff9c0 0x7fbf480ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.831 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.827+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/2882154911 >> 172.21.15.61:0/2882154911 conn(0x7fbf480fac00 msgr2=0x7fbf480fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.828+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/2882154911 shutdown_connections 2023-10-18T09:27:54.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.828+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/2882154911 wait complete. 2023-10-18T09:27:54.832 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.830+0000 7fbf4f9ff700 1 Processor -- start 2023-10-18T09:27:54.833 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.830+0000 7fbf4f9ff700 1 -- start start 2023-10-18T09:27:54.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.831+0000 7fbf4f9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.834 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.831+0000 7fbf4f9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf480ff9c0 0x7fbf48198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4f9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf4819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4f9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf481a0440 con 0x7fbf480ff9c0 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4f9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf481a0580 con 0x7fbf48100380 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4f9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf481a0710 con 0x7fbf480ff060 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.835 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48832/0 (socket says 172.21.15.61:48832) 2023-10-18T09:27:54.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 -- 172.21.15.61:0/1735322444 learned_addr learned my addr 172.21.15.61:0/1735322444 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:54.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf4819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.836 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 -- 172.21.15.61:0/1735322444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 msgr2=0x7fbf4819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf4819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 -- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf480ff9c0 msgr2=0x7fbf48198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf480ff9c0 0x7fbf48198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.837 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.832+0000 7fbf4d79b700 1 -- 172.21.15.61:0/1735322444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009c40 con 0x7fbf480ff060 2023-10-18T09:27:54.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.835+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.835+0000 7fbf3e7fc700 1 -- 172.21.15.61:0/1735322444 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cb50 con 0x7fbf480ff060 2023-10-18T09:27:54.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.835+0000 7fbf3e7fc700 1 -- 172.21.15.61:0/1735322444 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf4400eeb0 con 0x7fbf480ff060 2023-10-18T09:27:54.838 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.835+0000 7fbf3e7fc700 1 -- 172.21.15.61:0/1735322444 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf440054a0 con 0x7fbf480ff060 2023-10-18T09:27:54.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.835+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf481a09f0 con 0x7fbf480ff060 2023-10-18T09:27:54.839 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.836+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf481a0f00 con 0x7fbf480ff060 2023-10-18T09:27:54.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.848+0000 7fbf3e7fc700 1 -- 172.21.15.61:0/1735322444 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf44005600 con 0x7fbf480ff060 2023-10-18T09:27:54.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.851+0000 7fbf3e7fc700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 0x7fbf340a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.854 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.851+0000 7fbf3e7fc700 1 -- 172.21.15.61:0/1735322444 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbf44014070 con 0x7fbf480ff060 2023-10-18T09:27:54.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.853+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 0x7fbf340a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.855 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.853+0000 7fbf632b5b80 1 Processor -- start 2023-10-18T09:27:54.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf632b5b80 1 -- start start 2023-10-18T09:27:54.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc744090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 0x7fbf340a3a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbf380097b0 tx=0x7fbf38006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48838/0 (socket says 172.21.15.61:48838) 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.854+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc744a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf4d79b700 1 -- 172.21.15.61:0/2545899646 learned_addr learned my addr 172.21.15.61:0/2545899646 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc6b76e0 con 0x55c6bc745c30 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc744a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc69dc60 con 0x55c6bc7445d0 2023-10-18T09:27:54.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc69b580 con 0x55c6bc7452d0 2023-10-18T09:27:54.859 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.855+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc744090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4df9c700 1 -- 172.21.15.61:0/2545899646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 msgr2=0x55c6bc7456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4df9c700 1 -- 172.21.15.61:0/2545899646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 msgr2=0x55c6bc744090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.860 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc744090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4df9c700 1 -- 172.21.15.61:0/2545899646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bc6543f0 con 0x55c6bc7445d0 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.856+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc744090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.857+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc744a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf40007ba0 tx=0x7fbf40003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe27da055c19aced server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.857+0000 7fbf337fe700 1 -- 172.21.15.61:0/2545899646 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000e040 con 0x55c6bc7445d0 2023-10-18T09:27:54.861 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.857+0000 7fbf337fe700 1 -- 172.21.15.61:0/2545899646 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf40003c60 con 0x55c6bc7445d0 2023-10-18T09:27:54.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.857+0000 7fbf337fe700 1 -- 172.21.15.61:0/2545899646 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40005320 con 0x55c6bc7445d0 2023-10-18T09:27:54.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 msgr2=0x55c6bc744a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc744a50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbf40007ba0 tx=0x7fbf40003af0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.862 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 shutdown_connections 2023-10-18T09:27:54.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc744a50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc7456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2545899646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc744090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.858+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 >> 172.21.15.61:0/2545899646 conn(0x55c6bc6b8720 msgr2=0x55c6bc73eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.863 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.859+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 shutdown_connections 2023-10-18T09:27:54.864 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.859+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2545899646 wait complete. 2023-10-18T09:27:54.865 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.860+0000 7fbf632b5b80 1 -- start start 2023-10-18T09:27:54.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.863+0000 7fbf632b5b80 1 Processor -- start 2023-10-18T09:27:54.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.866 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc756850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc75d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc696e70 con 0x55c6bc745c30 2023-10-18T09:27:54.867 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc633710 con 0x55c6bc7445d0 2023-10-18T09:27:54.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.864+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc6333b0 con 0x55c6bc7452d0 2023-10-18T09:27:54.868 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.865+0000 7fbf632b5b80 10 client.0 _ll_register_callbacks cb 0x55c6bc57da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:27:54.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.869 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:53564/0 (socket says 172.21.15.61:53564) 2023-10-18T09:27:54.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 -- 172.21.15.61:0/130989740 learned_addr learned my addr 172.21.15.61:0/130989740 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:54.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 -- 172.21.15.61:0/130989740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 msgr2=0x55c6bc756850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc756850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 -- 172.21.15.61:0/130989740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 msgr2=0x55c6bc75d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.870 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc75d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.866+0000 7fbf4d79b700 1 -- 172.21.15.61:0/130989740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380154e0 con 0x55c6bc7445d0 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.867+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbf44096520 tx=0x7fbf4409a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=562afe196875dedb server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.867+0000 7fbf30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.867+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44099560 con 0x55c6bc7445d0 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.867+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbf440996c0 con 0x55c6bc7445d0 2023-10-18T09:27:54.871 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.867+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6bc69bef0 con 0x55c6bc7445d0 2023-10-18T09:27:54.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.869+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf440b8250 con 0x55c6bc7445d0 2023-10-18T09:27:54.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.869+0000 7fbf632b5b80 10 client.26330 fetch_fsmap learned FSMap version 43 2023-10-18T09:27:54.872 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.869+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fbf4810b700 con 0x55c6bc7445d0 2023-10-18T09:27:54.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.870+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6bc69bef0 con 0x55c6bc7445d0 2023-10-18T09:27:54.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.871+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf44095e60 con 0x55c6bc7445d0 2023-10-18T09:27:54.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 10 client.26330 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:27:54.874 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:27:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 20 client.26330 populate_metadata read hostname 'smithi061' 2023-10-18T09:27:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc7620f0 0x55c6bc764510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 4 client.26330 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:27:54.875 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6bc7138b0 con 0x55c6bc7620f0 2023-10-18T09:27:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.872+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc7620f0 0x55c6bc764510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.873+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc7620f0 0x55c6bc764510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63a199767e6b467 server_cookie=567d024bf850ed48 in_seq=0 out_seq=0 2023-10-18T09:27:54.876 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.873+0000 7fbf30ff9700 10 client.26330 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.874+0000 7fbf30ff9700 1 -- 172.21.15.61:0/130989740 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c6bc7138b0 con 0x55c6bc7620f0 2023-10-18T09:27:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.874+0000 7fbf30ff9700 10 client.26330 handle_command_reply: tid=0 2023-10-18T09:27:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.874+0000 7fbf632b5b80 1 client.26330 shutdown 2023-10-18T09:27:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 msgr2=0x55c6bc756310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.877 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbf44096520 tx=0x7fbf4409a120 comp rx=0 tx=0).stop 2023-10-18T09:27:54.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 shutdown_connections 2023-10-18T09:27:54.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc7620f0 0x55c6bc764510 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.878 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc745c30 0x55c6bc75d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc7445d0 0x55c6bc756310 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/130989740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc756850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf30ff9700 0 client.26330 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 >> 172.21.15.61:0/130989740 conn(0x55c6bc6b8720 msgr2=0x55c6bc73eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.879 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.875+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 shutdown_connections 2023-10-18T09:27:54.880 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.876+0000 7fbf632b5b80 1 -- 172.21.15.61:0/130989740 wait complete. 2023-10-18T09:27:54.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.879+0000 7fbf632b5b80 20 client.26330 trim_cache size 0 max 16384 2023-10-18T09:27:54.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.923+0000 7fbf632b5b80 1 Processor -- start 2023-10-18T09:27:54.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.925+0000 7fbf632b5b80 1 -- start start 2023-10-18T09:27:54.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc744090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc7395d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc517920 con 0x55c6bc7445d0 2023-10-18T09:27:54.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc7570d0 con 0x55c6bc745c30 2023-10-18T09:27:54.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.926+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc75e9c0 con 0x55c6bc7452d0 2023-10-18T09:27:54.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48864/0 (socket says 172.21.15.61:48864) 2023-10-18T09:27:54.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/1877060582 learned_addr learned my addr 172.21.15.61:0/1877060582 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:54.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc7395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/1877060582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 msgr2=0x55c6bc7395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc744090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc7395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/1877060582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 msgr2=0x55c6bc744090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc744090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/1877060582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6bc4c1de0 con 0x55c6bc7452d0 2023-10-18T09:27:54.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.927+0000 7fbf4df9c700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc7395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:27:54.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.928+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c6bc764e60 tx=0x7fbf38006f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dabde44066b46d02 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.928+0000 7fbf33fff700 1 -- 172.21.15.61:0/1877060582 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3801a040 con 0x55c6bc7452d0 2023-10-18T09:27:54.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.928+0000 7fbf33fff700 1 -- 172.21.15.61:0/1877060582 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3801b040 con 0x55c6bc7452d0 2023-10-18T09:27:54.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.928+0000 7fbf33fff700 1 -- 172.21.15.61:0/1877060582 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38013a00 con 0x55c6bc7452d0 2023-10-18T09:27:54.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.929+0000 7fbf632b5b80 1 -- 172.21.15.61:0/1877060582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 msgr2=0x55c6bc75d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.929+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c6bc764e60 tx=0x7fbf38006f90 comp rx=0 tx=0).stop 2023-10-18T09:27:54.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 -- 172.21.15.61:0/1877060582 shutdown_connections 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc7395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc744090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/1877060582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d800 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 -- 172.21.15.61:0/1877060582 >> 172.21.15.61:0/1877060582 conn(0x55c6bc6b8720 msgr2=0x55c6bc7449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.930+0000 7fbf632b5b80 1 -- 172.21.15.61:0/1877060582 shutdown_connections 2023-10-18T09:27:54.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.931+0000 7fbf632b5b80 1 -- 172.21.15.61:0/1877060582 wait complete. 2023-10-18T09:27:54.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.932+0000 7fbf632b5b80 1 -- start start 2023-10-18T09:27:54.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.932+0000 7fbf632b5b80 1 Processor -- start 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc75d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc759dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc75a4b0 con 0x55c6bc7445d0 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc75a5f0 con 0x55c6bc745c30 2023-10-18T09:27:54.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6bc75a7b0 con 0x55c6bc7452d0 2023-10-18T09:27:54.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.933+0000 7fbf632b5b80 10 client.0 _ll_register_callbacks cb 0x55c6bc57da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:27:54.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.934+0000 7fbf4cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.934+0000 7fbf4cf9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48866/0 (socket says 172.21.15.61:48866) 2023-10-18T09:27:54.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.934+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/2236086014 learned_addr learned my addr 172.21.15.61:0/2236086014 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:27:54.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 msgr2=0x55c6bc759dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc759dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/2236086014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 msgr2=0x55c6bc75d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:27:54.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc75d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 -- 172.21.15.61:0/2236086014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf38015a50 con 0x55c6bc7452d0 2023-10-18T09:27:54.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf4cf9a700 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf38006b70 tx=0x7fbf38015200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15196ca21bf0a41f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:27:54.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.935+0000 7fbf30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:27:54.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.944+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38022450 con 0x55c6bc7452d0 2023-10-18T09:27:54.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.944+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbf38013780 con 0x55c6bc7452d0 2023-10-18T09:27:54.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.945+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf40003c60 con 0x55c6bc7452d0 2023-10-18T09:27:54.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.945+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf380225b0 con 0x55c6bc7452d0 2023-10-18T09:27:54.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.945+0000 7fbf632b5b80 10 client.26284 fetch_fsmap learned FSMap version 43 2023-10-18T09:27:54.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.945+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c6bc4aac60 con 0x55c6bc7452d0 2023-10-18T09:27:54.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.946+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf40003c60 con 0x55c6bc7452d0 2023-10-18T09:27:54.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.946+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbf38034bb0 con 0x55c6bc7452d0 2023-10-18T09:27:54.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:27:54.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:27:54.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 20 client.26284 populate_metadata read hostname 'smithi061' 2023-10-18T09:27:54.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc74df10 0x55c6bc750330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:27:54.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 4 client.26284 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:27:54.950 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.947+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c6bc68fe50 con 0x55c6bc74df10 2023-10-18T09:27:54.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.952+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc74df10 0x55c6bc750330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:27:54.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.956+0000 7fbf30ff9700 10 client.26284 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:54.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.956+0000 7fbf4d79b700 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc74df10 0x55c6bc750330 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f043b6d61a9e6a server_cookie=749f39593061203e in_seq=0 out_seq=0 2023-10-18T09:27:54.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf30ff9700 1 -- 172.21.15.61:0/2236086014 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c6bc68fe50 con 0x55c6bc74df10 2023-10-18T09:27:54.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf30ff9700 10 client.26284 handle_command_reply: tid=0 2023-10-18T09:27:54.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 client.26284 shutdown 2023-10-18T09:27:54.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 msgr2=0x55c6bc75d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf38006b70 tx=0x7fbf38015200 comp rx=0 tx=0).stop 2023-10-18T09:27:54.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 shutdown_connections 2023-10-18T09:27:54.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55c6bc74df10 0x55c6bc750330 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c6bc745c30 0x55c6bc759dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55c6bc7445d0 0x55c6bc75d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf30ff9700 0 client.26284 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:27:54.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 --2- 172.21.15.61:0/2236086014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6bc7452d0 0x55c6bc75d780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.958+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 >> 172.21.15.61:0/2236086014 conn(0x55c6bc6b8720 msgr2=0x55c6bc73eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.959+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 shutdown_connections 2023-10-18T09:27:54.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.959+0000 7fbf632b5b80 1 -- 172.21.15.61:0/2236086014 wait complete. 2023-10-18T09:27:54.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.959+0000 7fbf632b5b80 20 client.26284 trim_cache size 0 max 16384 2023-10-18T09:27:54.964 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:27:54.964 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:27:54.964 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:27:54.965 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:27:54.966 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:27:54.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.960+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 msgr2=0x7fbf340a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.960+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 0x7fbf340a3a10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbf380097b0 tx=0x7fbf38006d20 comp rx=0 tx=0).stop 2023-10-18T09:27:54.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.960+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 msgr2=0x7fbf48198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:27:54.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.960+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ecd0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 shutdown_connections 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf340a1550 0x7fbf340a3a10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf48100380 0x7fbf4819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf480ff060 0x7fbf48198910 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 --2- 172.21.15.61:0/1735322444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf480ff9c0 0x7fbf48198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.961+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 >> 172.21.15.61:0/1735322444 conn(0x7fbf480fac00 msgr2=0x7fbf480fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:27:54.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.962+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 shutdown_connections 2023-10-18T09:27:54.969 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:27:54.962+0000 7fbf4f9ff700 1 -- 172.21.15.61:0/1735322444 wait complete. 2023-10-18T09:27:55.000 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:27:55.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:55 smithi061 ceph-mon[98573]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:27:55.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:55 smithi170 ceph-mon[135150]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:27:55.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:55 smithi110 ceph-mon[134818]: pgmap v639: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:27:58.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:57 smithi170 ceph-mon[135150]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2023-10-18T09:27:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:57 smithi110 ceph-mon[134818]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2023-10-18T09:27:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:57 smithi061 ceph-mon[98573]: pgmap v640: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.2 MiB/s wr, 562 op/s 2023-10-18T09:27:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 525.0 s, 457.2 tps, lat 217.242 ms stddev 247.475 2023-10-18T09:27:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:27:59 smithi061 ceph-mon[98573]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.2 MiB/s wr, 474 op/s 2023-10-18T09:27:59.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:27:59 smithi170 ceph-mon[135150]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.2 MiB/s wr, 474 op/s 2023-10-18T09:27:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:27:59 smithi110 ceph-mon[134818]: pgmap v641: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.2 MiB/s wr, 474 op/s 2023-10-18T09:28:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:01 smithi061 ceph-mon[98573]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 564 op/s 2023-10-18T09:28:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:01 smithi170 ceph-mon[135150]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 564 op/s 2023-10-18T09:28:01.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:01 smithi110 ceph-mon[134818]: pgmap v642: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 6.3 MiB/s wr, 564 op/s 2023-10-18T09:28:03.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:03 smithi061 ceph-mon[98573]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.7 MiB/s wr, 491 op/s 2023-10-18T09:28:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:03 smithi170 ceph-mon[135150]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.7 MiB/s wr, 491 op/s 2023-10-18T09:28:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:03 smithi110 ceph-mon[134818]: pgmap v643: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.7 MiB/s wr, 491 op/s 2023-10-18T09:28:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 530.0 s, 465.6 tps, lat 214.491 ms stddev 230.388 2023-10-18T09:28:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:05 smithi061 ceph-mon[98573]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.9 MiB/s wr, 599 op/s 2023-10-18T09:28:05.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:05 smithi170 ceph-mon[135150]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.9 MiB/s wr, 599 op/s 2023-10-18T09:28:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:05 smithi110 ceph-mon[134818]: pgmap v644: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 7.9 MiB/s wr, 599 op/s 2023-10-18T09:28:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:07 smithi110 ceph-mon[134818]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2023-10-18T09:28:08.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:07 smithi061 ceph-mon[98573]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2023-10-18T09:28:08.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:07 smithi170 ceph-mon[135150]: pgmap v645: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.9 MiB/s wr, 583 op/s 2023-10-18T09:28:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 535.0 s, 544.0 tps, lat 186.633 ms stddev 210.432 2023-10-18T09:28:09.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:09 smithi061 ceph-mon[98573]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 506 op/s 2023-10-18T09:28:09.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:09 smithi170 ceph-mon[135150]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 506 op/s 2023-10-18T09:28:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:09 smithi110 ceph-mon[134818]: pgmap v646: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 6.7 MiB/s wr, 506 op/s 2023-10-18T09:28:11.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:11 smithi170 ceph-mon[135150]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-10-18T09:28:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:11 smithi110 ceph-mon[134818]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-10-18T09:28:11.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:11 smithi061 ceph-mon[98573]: pgmap v647: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 8.7 MiB/s wr, 598 op/s 2023-10-18T09:28:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:13 smithi061 ceph-mon[98573]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 513 op/s 2023-10-18T09:28:13.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:13 smithi170 ceph-mon[135150]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 513 op/s 2023-10-18T09:28:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:13 smithi110 ceph-mon[134818]: pgmap v648: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.6 MiB/s wr, 513 op/s 2023-10-18T09:28:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 540.0 s, 575.8 tps, lat 165.828 ms stddev 182.038 2023-10-18T09:28:14.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:14 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:28:14.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:14 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:28:14.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:14 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:28:15.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:15 smithi170 ceph-mon[135150]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 579 op/s 2023-10-18T09:28:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:15 smithi110 ceph-mon[134818]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 579 op/s 2023-10-18T09:28:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:15 smithi061 ceph-mon[98573]: pgmap v649: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.9 MiB/s wr, 579 op/s 2023-10-18T09:28:16.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:28:16.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:28:16.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:28:16.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:28:16.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:28:16.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:28:16.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:28:16.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:28:16.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:28:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:17 smithi110 ceph-mon[134818]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-10-18T09:28:18.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:17 smithi061 ceph-mon[98573]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-10-18T09:28:18.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:17 smithi170 ceph-mon[135150]: pgmap v650: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.1 MiB/s wr, 509 op/s 2023-10-18T09:28:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 545.0 s, 530.0 tps, lat 199.200 ms stddev 253.998 2023-10-18T09:28:19.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:19 smithi061 ceph-mon[98573]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2023-10-18T09:28:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:19 smithi170 ceph-mon[135150]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2023-10-18T09:28:19.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:19 smithi110 ceph-mon[134818]: pgmap v651: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 7.5 MiB/s wr, 479 op/s 2023-10-18T09:28:21.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:21 smithi170 ceph-mon[135150]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.8 MiB/s wr, 536 op/s 2023-10-18T09:28:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:21 smithi110 ceph-mon[134818]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.8 MiB/s wr, 536 op/s 2023-10-18T09:28:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:21 smithi061 ceph-mon[98573]: pgmap v652: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.8 MiB/s wr, 536 op/s 2023-10-18T09:28:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:23 smithi170 ceph-mon[135150]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2023-10-18T09:28:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:23 smithi110 ceph-mon[134818]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2023-10-18T09:28:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:23 smithi061 ceph-mon[98573]: pgmap v653: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.9 MiB/s wr, 452 op/s 2023-10-18T09:28:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 550.0 s, 537.4 tps, lat 183.566 ms stddev 213.698 2023-10-18T09:28:25.001 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:28:25.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 -- 172.21.15.61:0/1285658444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0077ef0 msgr2=0x7f88a007a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 --2- 172.21.15.61:0/1285658444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0077ef0 0x7f88a007a3b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8890007570 tx=0x7f8890007880 comp rx=0 tx=0).stop 2023-10-18T09:28:25.457 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 -- 172.21.15.61:0/1285658444 shutdown_connections 2023-10-18T09:28:25.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 --2- 172.21.15.61:0/1285658444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0077ef0 0x7f88a007a3b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 --2- 172.21.15.61:0/1285658444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0072640 0x7f88a0072a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 --2- 172.21.15.61:0/1285658444 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0075520 0x7f88a00779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 -- 172.21.15.61:0/1285658444 >> 172.21.15.61:0/1285658444 conn(0x7f88a006d7d0 msgr2=0x7f88a006fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.458 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 -- 172.21.15.61:0/1285658444 shutdown_connections 2023-10-18T09:28:25.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.454+0000 7f88a65ff700 1 -- 172.21.15.61:0/1285658444 wait complete. 2023-10-18T09:28:25.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.456+0000 7f88a65ff700 1 Processor -- start 2023-10-18T09:28:25.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.457+0000 7f88a65ff700 1 -- start start 2023-10-18T09:28:25.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 0x7f88a01b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0077ef0 0x7f88a01b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a01b7b90 con 0x7f88a0077ef0 2023-10-18T09:28:25.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a01b7cd0 con 0x7f88a0075520 2023-10-18T09:28:25.461 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a65ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a01b7e60 con 0x7f88a0072640 2023-10-18T09:28:25.462 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46682/0 (socket says 172.21.15.61:46682) 2023-10-18T09:28:25.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f889f7fe700 1 -- 172.21.15.61:0/548320998 learned_addr learned my addr 172.21.15.61:0/548320998 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:25.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.458+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0077ef0 0x7f88a01b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889ffff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 0x7f88a01b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.463 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 -- 172.21.15.61:0/548320998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 msgr2=0x7f88a01b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 0x7f88a01b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 -- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0077ef0 msgr2=0x7f88a01b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0077ef0 0x7f88a01b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 -- 172.21.15.61:0/548320998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890007220 con 0x7f88a0075520 2023-10-18T09:28:25.464 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889ffff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 0x7f88a01b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:28:25.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889f7fe700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8894011d60 tx=0x7f8894009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.459+0000 7f889d7fa700 1 -- 172.21.15.61:0/548320998 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8894011070 con 0x7f88a0075520 2023-10-18T09:28:25.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.460+0000 7f889d7fa700 1 -- 172.21.15.61:0/548320998 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889401a850 con 0x7f88a0075520 2023-10-18T09:28:25.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.460+0000 7f889d7fa700 1 -- 172.21.15.61:0/548320998 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400f820 con 0x7f88a0075520 2023-10-18T09:28:25.465 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.460+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a0078330 con 0x7f88a0075520 2023-10-18T09:28:25.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.460+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a00787f0 con 0x7f88a0075520 2023-10-18T09:28:25.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 Processor -- start 2023-10-18T09:28:25.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 -- start start 2023-10-18T09:28:25.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f889d7fa700 1 -- 172.21.15.61:0/548320998 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f889401a9c0 con 0x7f88a0075520 2023-10-18T09:28:25.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adf46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf4c30 0x55a80adf3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ad666e0 con 0x55a80adf35d0 2023-10-18T09:28:25.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ad4cc60 con 0x55a80adf42d0 2023-10-18T09:28:25.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.468+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ad4a580 con 0x55a80adf4c30 2023-10-18T09:28:25.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.469+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.469+0000 7f88a4b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adf46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.469+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:46198/0 (socket says 172.21.15.61:46198) 2023-10-18T09:28:25.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.469+0000 7f889f7fe700 1 -- 172.21.15.61:0/883273702 learned_addr learned my addr 172.21.15.61:0/883273702 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:25.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 -- 172.21.15.61:0/883273702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf4c30 msgr2=0x55a80adf3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf4c30 0x55a80adf3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 -- 172.21.15.61:0/883273702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 msgr2=0x55a80adf46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adf46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 -- 172.21.15.61:0/883273702 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a80ad033f0 con 0x55a80adf35d0 2023-10-18T09:28:25.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.470+0000 7f889f7fe700 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f88940dba80 tx=0x7f88940dbd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20fb1eae09720b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.471+0000 7f889d7fa700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 0x7f88880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.471+0000 7f889d7fa700 1 -- 172.21.15.61:0/548320998 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f88940d5910 con 0x7f88a0075520 2023-10-18T09:28:25.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.472+0000 7f88867fc700 1 -- 172.21.15.61:0/883273702 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889409ec20 con 0x55a80adf35d0 2023-10-18T09:28:25.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.472+0000 7f88867fc700 1 -- 172.21.15.61:0/883273702 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88940da030 con 0x55a80adf35d0 2023-10-18T09:28:25.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.472+0000 7f88867fc700 1 -- 172.21.15.61:0/883273702 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940a3050 con 0x55a80adf35d0 2023-10-18T09:28:25.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.473+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 0x7f88880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.474+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 0x7f88880a39c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f889000c3b0 tx=0x7f8890005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.475+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/883273702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 msgr2=0x55a80adf3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f88940dba80 tx=0x7f88940dbd90 comp rx=0 tx=0).stop 2023-10-18T09:28:25.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/883273702 shutdown_connections 2023-10-18T09:28:25.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf35d0 0x55a80adf3a50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adf46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/883273702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf4c30 0x55a80adf3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.476+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/883273702 >> 172.21.15.61:0/883273702 conn(0x55a80ad67720 msgr2=0x55a80adedb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.477+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/883273702 shutdown_connections 2023-10-18T09:28:25.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.477+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/883273702 wait complete. 2023-10-18T09:28:25.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.477+0000 7f88b9fc3b80 1 -- start start 2023-10-18T09:28:25.481 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.478+0000 7f88b9fc3b80 1 Processor -- start 2023-10-18T09:28:25.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.479+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80adfca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.479+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf4c30 0x55a80ae03f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ad45e70 con 0x55a80adf4c30 2023-10-18T09:28:25.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ace2710 con 0x55a80adf42d0 2023-10-18T09:28:25.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46700/0 (socket says 172.21.15.61:46700) 2023-10-18T09:28:25.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f889f7fe700 1 -- 172.21.15.61:0/1925769252 learned_addr learned my addr 172.21.15.61:0/1925769252 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:25.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf4c30 0x55a80ae03f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f889ffff700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80adfca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ace23b0 con 0x55a80adf35d0 2023-10-18T09:28:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.480+0000 7f88b9fc3b80 10 client.0 _ll_register_callbacks cb 0x55a80ac2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:28:25.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.482+0000 7f889ffff700 1 -- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 msgr2=0x55a80adfcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.482+0000 7f889ffff700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.482+0000 7f889ffff700 1 -- 172.21.15.61:0/1925769252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf4c30 msgr2=0x55a80ae03f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.482+0000 7f889ffff700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf4c30 0x55a80ae03f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.484+0000 7f889ffff700 1 -- 172.21.15.61:0/1925769252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a00787f0 con 0x55a80adf35d0 2023-10-18T09:28:25.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.484+0000 7f889f7fe700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:28:25.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.485+0000 7f889ffff700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80adfca10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f889800c960 tx=0x7f889800cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78dfaa08e4ab85d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.485+0000 7f887f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:28:25.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.486+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889800e8a0 con 0x55a80adf35d0 2023-10-18T09:28:25.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.487+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8898006e40 con 0x55a80adf35d0 2023-10-18T09:28:25.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.487+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a80ad4aef0 con 0x55a80adf35d0 2023-10-18T09:28:25.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.487+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889800ea00 con 0x55a80adf35d0 2023-10-18T09:28:25.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.487+0000 7f88b9fc3b80 10 client.26290 fetch_fsmap learned FSMap version 43 2023-10-18T09:28:25.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.487+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a80ab70de0 con 0x55a80adf35d0 2023-10-18T09:28:25.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.488+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889800eb60 con 0x55a80adf35d0 2023-10-18T09:28:25.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f8898016020 con 0x55a80adf35d0 2023-10-18T09:28:25.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 10 client.26290 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:28:25.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:28:25.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 20 client.26290 populate_metadata read hostname 'smithi061' 2023-10-18T09:28:25.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae087a0 0x55a80ae0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 4 client.26290 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:28:25.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a80ad4aef0 con 0x55a80ae087a0 2023-10-18T09:28:25.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.489+0000 7f889f7fe700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae087a0 0x55a80ae0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.490+0000 7f887f7fe700 10 client.26290 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:25.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.490+0000 7f889f7fe700 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae087a0 0x55a80ae0ac40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1edb37fb99048426 server_cookie=ae64331c2f20da2a in_seq=0 out_seq=0 2023-10-18T09:28:25.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f887f7fe700 1 -- 172.21.15.61:0/1925769252 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a80ad4aef0 con 0x55a80ae087a0 2023-10-18T09:28:25.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f887f7fe700 10 client.26290 handle_command_reply: tid=0 2023-10-18T09:28:25.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 client.26290 shutdown 2023-10-18T09:28:25.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 msgr2=0x55a80adfca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80adfca10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f889800c960 tx=0x7f889800cc70 comp rx=0 tx=0).stop 2023-10-18T09:28:25.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 shutdown_connections 2023-10-18T09:28:25.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae087a0 0x55a80ae0ac40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf4c30 0x55a80ae03f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80adfca10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/1925769252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf42d0 0x55a80adfcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 >> 172.21.15.61:0/1925769252 conn(0x55a80ad67720 msgr2=0x55a80adf2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.492+0000 7f887f7fe700 0 client.26290 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:25.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.493+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 shutdown_connections 2023-10-18T09:28:25.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.493+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/1925769252 wait complete. 2023-10-18T09:28:25.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.493+0000 7f88b9fc3b80 20 client.26290 trim_cache size 0 max 16384 2023-10-18T09:28:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:25 smithi061 ceph-mon[98573]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-18T09:28:25.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.535+0000 7f88b9fc3b80 1 Processor -- start 2023-10-18T09:28:25.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.535+0000 7f88b9fc3b80 1 -- start start 2023-10-18T09:28:25.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.536+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ade85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.536+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.536+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80adf39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.536+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80abc6920 con 0x55a80adf42d0 2023-10-18T09:28:25.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f889f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:46216/0 (socket says 172.21.15.61:46216) 2023-10-18T09:28:25.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ade5fd0 con 0x55a80adf4c30 2023-10-18T09:28:25.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80adefb80 con 0x55a80adf35d0 2023-10-18T09:28:25.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f889f7fe700 1 -- 172.21.15.61:0/3333482241 learned_addr learned my addr 172.21.15.61:0/3333482241 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:25.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.537+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80adf39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.538+0000 7f88a4b9c700 1 -- 172.21.15.61:0/3333482241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 msgr2=0x55a80ade85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.538+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ade85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.538+0000 7f88a4b9c700 1 -- 172.21.15.61:0/3333482241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 msgr2=0x55a80ae03f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.542 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.538+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.538+0000 7f88a4b9c700 1 -- 172.21.15.61:0/3333482241 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a80ab59c60 con 0x55a80adf4c30 2023-10-18T09:28:25.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.539+0000 7f88a4b9c700 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80adf39b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8890005ef0 tx=0x7f889001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4945904bca9021 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.539+0000 7f8886ffd700 1 -- 172.21.15.61:0/3333482241 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000f040 con 0x55a80adf4c30 2023-10-18T09:28:25.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.539+0000 7f8886ffd700 1 -- 172.21.15.61:0/3333482241 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f889000a8d0 con 0x55a80adf4c30 2023-10-18T09:28:25.543 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.539+0000 7f8886ffd700 1 -- 172.21.15.61:0/3333482241 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889000c860 con 0x55a80adf4c30 2023-10-18T09:28:25.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.540+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3333482241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 msgr2=0x55a80adf39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.540+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80adf39b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8890005ef0 tx=0x7f889001d000 comp rx=0 tx=0).stop 2023-10-18T09:28:25.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.541+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3333482241 shutdown_connections 2023-10-18T09:28:25.544 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.541+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80adf39b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.541+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ade85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.541+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3333482241 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.541+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3333482241 >> 172.21.15.61:0/3333482241 conn(0x55a80ad67720 msgr2=0x55a80ae0f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.545 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.542+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3333482241 shutdown_connections 2023-10-18T09:28:25.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.542+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3333482241 wait complete. 2023-10-18T09:28:25.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.543+0000 7f88b9fc3b80 1 -- start start 2023-10-18T09:28:25.546 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.543+0000 7f88b9fc3b80 1 Processor -- start 2023-10-18T09:28:25.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.544+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ae03820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.547 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.544+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.544+0000 7f88b9fc3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80ae01e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88b9fc3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ae02480 con 0x55a80adf42d0 2023-10-18T09:28:25.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88a4b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80ae01e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88a4b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80ae01e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:46722/0 (socket says 172.21.15.61:46722) 2023-10-18T09:28:25.548 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88a4b9c700 1 -- 172.21.15.61:0/3023971732 learned_addr learned my addr 172.21.15.61:0/3023971732 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:25.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ae025f0 con 0x55a80adf4c30 2023-10-18T09:28:25.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a80ae027b0 con 0x55a80adf35d0 2023-10-18T09:28:25.549 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.545+0000 7f88b9fc3b80 10 client.0 _ll_register_callbacks cb 0x55a80ac2ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:28:25.550 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.548+0000 7f889f7fe700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.549+0000 7f889f7fe700 1 -- 172.21.15.61:0/3023971732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 msgr2=0x55a80ae03820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.551 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.549+0000 7f889f7fe700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ae03820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.549+0000 7f889f7fe700 1 -- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 msgr2=0x55a80ae01e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:25.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.549+0000 7f889f7fe700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80ae01e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.552 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.549+0000 7f889f7fe700 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a007ade0 con 0x55a80adf42d0 2023-10-18T09:28:25.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.550+0000 7f889f7fe700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03d60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88940dd5a0 tx=0x7f8894000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f87ce20bd61fe server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:25.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.550+0000 7f887f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:28:25.553 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.550+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940a2420 con 0x55a80adf42d0 2023-10-18T09:28:25.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.550+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f88940a28e0 con 0x55a80adf42d0 2023-10-18T09:28:25.554 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.551+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88940da030 con 0x55a80adf42d0 2023-10-18T09:28:25.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.552+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88940a2240 con 0x55a80adf42d0 2023-10-18T09:28:25.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.553+0000 7f88b9fc3b80 10 client.16149 fetch_fsmap learned FSMap version 43 2023-10-18T09:28:25.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.553+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a80ab97250 con 0x55a80adf42d0 2023-10-18T09:28:25.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.553+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88940da030 con 0x55a80adf42d0 2023-10-18T09:28:25.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f88940c20b0 con 0x55a80adf42d0 2023-10-18T09:28:25.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 10 client.16149 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:28:25.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:28:25.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 20 client.16149 populate_metadata read hostname 'smithi061' 2023-10-18T09:28:25.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae5d7e0 0x55a80ae5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:25.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 4 client.16149 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:28:25.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.554+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a80adc28b0 con 0x55a80ae5d7e0 2023-10-18T09:28:25.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.556+0000 7f889ffff700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae5d7e0 0x55a80ae5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:25.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.556+0000 7f889ffff700 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae5d7e0 0x55a80ae5fc00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5649041e4e64055 server_cookie=1576cdc8ffa5aec5 in_seq=0 out_seq=0 2023-10-18T09:28:25.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.556+0000 7f887f7fe700 10 client.16149 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:25.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.557+0000 7f887f7fe700 1 -- 172.21.15.61:0/3023971732 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55a80adc28b0 con 0x55a80ae5d7e0 2023-10-18T09:28:25.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.557+0000 7f887f7fe700 10 client.16149 handle_command_reply: tid=0 2023-10-18T09:28:25.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.557+0000 7f88b9fc3b80 1 client.16149 shutdown 2023-10-18T09:28:25.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.557+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 msgr2=0x55a80ae03d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.557+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03d60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f88940dd5a0 tx=0x7f8894000bc0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 shutdown_connections 2023-10-18T09:28:25.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55a80ae5d7e0 0x55a80ae5fc00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a80adf4c30 0x55a80ae01e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a80adf35d0 0x55a80ae03820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 --2- 172.21.15.61:0/3023971732 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55a80adf42d0 0x55a80ae03d60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f887f7fe700 0 client.16149 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:25.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 >> 172.21.15.61:0/3023971732 conn(0x55a80ad67720 msgr2=0x55a80adfbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 shutdown_connections 2023-10-18T09:28:25.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 1 -- 172.21.15.61:0/3023971732 wait complete. 2023-10-18T09:28:25.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.558+0000 7f88b9fc3b80 20 client.16149 trim_cache size 0 max 16384 2023-10-18T09:28:25.564 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:28:25.564 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:28:25.564 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:28:25.564 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:28:25.565 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:28:25.565 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:28:25.565 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:28:25.565 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:28:25.565 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:28:25.566 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:28:25.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.559+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 msgr2=0x7f88880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.559+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 0x7f88880a39c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f889000c3b0 tx=0x7f8890005e20 comp rx=0 tx=0).stop 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.559+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 msgr2=0x7f88a01b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.559+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8894011d60 tx=0x7f8894009520 comp rx=0 tx=0).stop 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 shutdown_connections 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f88880a1500 0x7f88880a39c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f88a0077ef0 0x7f88a01b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a0072640 0x7f88a01b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 --2- 172.21.15.61:0/548320998 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88a0075520 0x7f88a01b05a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:25.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 >> 172.21.15.61:0/548320998 conn(0x7f88a006d7d0 msgr2=0x7f88a0077780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:25.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 shutdown_connections 2023-10-18T09:28:25.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:25.561+0000 7f88a65ff700 1 -- 172.21.15.61:0/548320998 wait complete. 2023-10-18T09:28:25.596 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:28:25.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:25 smithi170 ceph-mon[135150]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-18T09:28:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:25 smithi110 ceph-mon[134818]: pgmap v654: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 8.4 MiB/s wr, 532 op/s 2023-10-18T09:28:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:27 smithi061 ceph-mon[98573]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 488 op/s 2023-10-18T09:28:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:27 smithi170 ceph-mon[135150]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 488 op/s 2023-10-18T09:28:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:27 smithi110 ceph-mon[134818]: pgmap v655: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.6 MiB/s wr, 488 op/s 2023-10-18T09:28:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 555.0 s, 562.6 tps, lat 182.033 ms stddev 190.121 2023-10-18T09:28:29.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:29 smithi170 ceph-mon[135150]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2023-10-18T09:28:29.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:29 smithi110 ceph-mon[134818]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2023-10-18T09:28:29.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:29 smithi061 ceph-mon[98573]: pgmap v656: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.5 MiB/s wr, 463 op/s 2023-10-18T09:28:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:31 smithi170 ceph-mon[135150]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.8 MiB/s wr, 547 op/s 2023-10-18T09:28:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:31 smithi110 ceph-mon[134818]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.8 MiB/s wr, 547 op/s 2023-10-18T09:28:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:31 smithi061 ceph-mon[98573]: pgmap v657: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.8 MiB/s wr, 547 op/s 2023-10-18T09:28:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:33 smithi170 ceph-mon[135150]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 480 op/s 2023-10-18T09:28:33.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:33 smithi110 ceph-mon[134818]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 480 op/s 2023-10-18T09:28:33.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:33 smithi061 ceph-mon[98573]: pgmap v658: 129 pgs: 129 active+clean; 1.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 480 op/s 2023-10-18T09:28:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 560.0 s, 511.4 tps, lat 179.194 ms stddev 229.341 2023-10-18T09:28:35.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:35 smithi170 ceph-mon[135150]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.4 MiB/s wr, 571 op/s 2023-10-18T09:28:35.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:35 smithi110 ceph-mon[134818]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.4 MiB/s wr, 571 op/s 2023-10-18T09:28:35.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:35 smithi061 ceph-mon[98573]: pgmap v659: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.4 MiB/s wr, 571 op/s 2023-10-18T09:28:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:37 smithi110 ceph-mon[134818]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2023-10-18T09:28:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:37 smithi061 ceph-mon[98573]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2023-10-18T09:28:38.403 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:37 smithi170 ceph-mon[135150]: pgmap v660: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 7.0 MiB/s wr, 506 op/s 2023-10-18T09:28:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 565.0 s, 498.6 tps, lat 215.525 ms stddev 293.458 2023-10-18T09:28:39.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:39 smithi170 ceph-mon[135150]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2023-10-18T09:28:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:39 smithi110 ceph-mon[134818]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2023-10-18T09:28:39.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:39 smithi061 ceph-mon[98573]: pgmap v661: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 6.9 MiB/s wr, 502 op/s 2023-10-18T09:28:41.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:41 smithi170 ceph-mon[135150]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.7 MiB/s wr, 555 op/s 2023-10-18T09:28:41.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:41 smithi110 ceph-mon[134818]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.7 MiB/s wr, 555 op/s 2023-10-18T09:28:41.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:41 smithi061 ceph-mon[98573]: pgmap v662: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.7 MiB/s wr, 555 op/s 2023-10-18T09:28:43.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:43 smithi170 ceph-mon[135150]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-18T09:28:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:43 smithi110 ceph-mon[134818]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-18T09:28:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:43 smithi061 ceph-mon[98573]: pgmap v663: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.7 MiB/s wr, 478 op/s 2023-10-18T09:28:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 570.0 s, 536.2 tps, lat 180.476 ms stddev 211.704 2023-10-18T09:28:45.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:45 smithi170 ceph-mon[135150]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.4 MiB/s wr, 551 op/s 2023-10-18T09:28:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:45 smithi110 ceph-mon[134818]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.4 MiB/s wr, 551 op/s 2023-10-18T09:28:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:45 smithi061 ceph-mon[98573]: pgmap v664: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 7.4 MiB/s wr, 551 op/s 2023-10-18T09:28:48.215 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:47 smithi110 ceph-mon[134818]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.8 MiB/s wr, 494 op/s 2023-10-18T09:28:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:47 smithi061 ceph-mon[98573]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.8 MiB/s wr, 494 op/s 2023-10-18T09:28:48.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:47 smithi170 ceph-mon[135150]: pgmap v665: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 5.8 MiB/s wr, 494 op/s 2023-10-18T09:28:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 575.0 s, 479.6 tps, lat 204.132 ms stddev 232.980 2023-10-18T09:28:49.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:49 smithi170 ceph-mon[135150]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-10-18T09:28:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:49 smithi110 ceph-mon[134818]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-10-18T09:28:49.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:49 smithi061 ceph-mon[98573]: pgmap v666: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 478 op/s 2023-10-18T09:28:51.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:51 smithi170 ceph-mon[135150]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.7 MiB/s wr, 530 op/s 2023-10-18T09:28:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:51 smithi110 ceph-mon[134818]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.7 MiB/s wr, 530 op/s 2023-10-18T09:28:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:51 smithi061 ceph-mon[98573]: pgmap v667: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.7 MiB/s wr, 530 op/s 2023-10-18T09:28:53.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:53 smithi170 ceph-mon[135150]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2023-10-18T09:28:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:53 smithi110 ceph-mon[134818]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2023-10-18T09:28:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:53 smithi061 ceph-mon[98573]: pgmap v668: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 6.3 MiB/s wr, 499 op/s 2023-10-18T09:28:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 580.0 s, 499.4 tps, lat 205.480 ms stddev 253.237 2023-10-18T09:28:55.597 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:28:55.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:55 smithi170 ceph-mon[135150]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.2 MiB/s wr, 565 op/s 2023-10-18T09:28:55.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:55 smithi110 ceph-mon[134818]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.2 MiB/s wr, 565 op/s 2023-10-18T09:28:55.721 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:55 smithi061 ceph-mon[98573]: pgmap v669: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 7.2 MiB/s wr, 565 op/s 2023-10-18T09:28:55.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 -- 172.21.15.61:0/3406785340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc981012b0 msgr2=0x7fcc981036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:55.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/3406785340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc981012b0 0x7fcc981036a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009990 tx=0x7fcc80009ca0 comp rx=0 tx=0).stop 2023-10-18T09:28:55.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 -- 172.21.15.61:0/3406785340 shutdown_connections 2023-10-18T09:28:56.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/3406785340 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 0x7fcc98108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/3406785340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc981012b0 0x7fcc981036a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/3406785340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc98103be0 0x7fcc98105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.996+0000 7fcc9cdff700 1 -- 172.21.15.61:0/3406785340 >> 172.21.15.61:0/3406785340 conn(0x7fcc980fac00 msgr2=0x7fcc980fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.000 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.997+0000 7fcc9cdff700 1 -- 172.21.15.61:0/3406785340 shutdown_connections 2023-10-18T09:28:56.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.997+0000 7fcc9cdff700 1 -- 172.21.15.61:0/3406785340 wait complete. 2023-10-18T09:28:56.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.998+0000 7fcc9cdff700 1 Processor -- start 2023-10-18T09:28:56.001 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 -- start start 2023-10-18T09:28:56.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc98103be0 0x7fcc98198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 0x7fcc9819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a0480 con 0x7fcc98106510 2023-10-18T09:28:56.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a05c0 con 0x7fcc981012b0 2023-10-18T09:28:56.003 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:55.999+0000 7fcc9cdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981a0750 con 0x7fcc98103be0 2023-10-18T09:28:56.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39760/0 (socket says 172.21.15.61:39760) 2023-10-18T09:28:56.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc9659c700 1 -- 172.21.15.61:0/2550433252 learned_addr learned my addr 172.21.15.61:0/2550433252 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:56.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 0x7fcc9819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.004 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc9659c700 1 -- 172.21.15.61:0/2550433252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc98103be0 msgr2=0x7fcc98198e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-18T09:28:56.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.000+0000 7fcc9659c700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc98103be0 0x7fcc98198e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.001+0000 7fcc9659c700 1 -- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 msgr2=0x7fcc9819feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.001+0000 7fcc9659c700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 0x7fcc9819feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.001+0000 7fcc9659c700 1 -- 172.21.15.61:0/2550433252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc800095f0 con 0x7fcc981012b0 2023-10-18T09:28:56.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.004+0000 7fcc9659c700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc800037f0 tx=0x7fcc8000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.007 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.004+0000 7fcc8f7fe700 1 -- 172.21.15.61:0/2550433252 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80025070 con 0x7fcc981012b0 2023-10-18T09:28:56.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.004+0000 7fcc8f7fe700 1 -- 172.21.15.61:0/2550433252 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8000fdc0 con 0x7fcc981012b0 2023-10-18T09:28:56.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.004+0000 7fcc8f7fe700 1 -- 172.21.15.61:0/2550433252 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8002be50 con 0x7fcc981012b0 2023-10-18T09:28:56.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.004+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9810ae40 con 0x7fcc981012b0 2023-10-18T09:28:56.008 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.005+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9810b2f0 con 0x7fcc981012b0 2023-10-18T09:28:56.013 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.011+0000 7fccb0652b80 1 Processor -- start 2023-10-18T09:28:56.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.011+0000 7fccb0652b80 1 -- start start 2023-10-18T09:28:56.014 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b6c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6cc30 0x560f24b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.015 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24ade6e0 con 0x560f24b6cc30 2023-10-18T09:28:56.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24ac4c60 con 0x560f24b6b5d0 2023-10-18T09:28:56.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fccb0652b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24ac2580 con 0x560f24b6c2d0 2023-10-18T09:28:56.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fcc95d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fcc95d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39774/0 (socket says 172.21.15.61:39774) 2023-10-18T09:28:56.016 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fcc95d9b700 1 -- 172.21.15.61:0/3458535685 learned_addr learned my addr 172.21.15.61:0/3458535685 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:56.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.012+0000 7fcc9659c700 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6cc30 0x560f24b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.013+0000 7fcc95d9b700 1 -- 172.21.15.61:0/3458535685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 msgr2=0x560f24b6c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.013+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b6c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.013+0000 7fcc95d9b700 1 -- 172.21.15.61:0/3458535685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6cc30 msgr2=0x560f24b6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.013+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6cc30 0x560f24b6b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.013+0000 7fcc95d9b700 1 -- 172.21.15.61:0/3458535685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f24a7b3f0 con 0x560f24b6b5d0 2023-10-18T09:28:56.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.015+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800bb10 tx=0x7fcc8800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c21ba45a41371cf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.019 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.017+0000 7fcc8f7fe700 1 -- 172.21.15.61:0/2550433252 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fcc80029070 con 0x7fcc981012b0 2023-10-18T09:28:56.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.018+0000 7fcc8cff9700 1 -- 172.21.15.61:0/3458535685 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004a30 con 0x560f24b6b5d0 2023-10-18T09:28:56.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.018+0000 7fcc8cff9700 1 -- 172.21.15.61:0/3458535685 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8800dc90 con 0x560f24b6b5d0 2023-10-18T09:28:56.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.018+0000 7fcc8cff9700 1 -- 172.21.15.61:0/3458535685 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005580 con 0x560f24b6b5d0 2023-10-18T09:28:56.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.019+0000 7fccb0652b80 1 -- 172.21.15.61:0/3458535685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 msgr2=0x560f24b6ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.019+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800bb10 tx=0x7fcc8800be20 comp rx=0 tx=0).stop 2023-10-18T09:28:56.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.019+0000 7fcc8f7fe700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 0x7fcc840a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.019+0000 7fcc8f7fe700 1 -- 172.21.15.61:0/2550433252 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc800d6df0 con 0x7fcc981012b0 2023-10-18T09:28:56.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 0x7fcc840a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.024 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 -- 172.21.15.61:0/3458535685 shutdown_connections 2023-10-18T09:28:56.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6b5d0 0x560f24b6ba50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b6c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3458535685 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6cc30 0x560f24b6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 -- 172.21.15.61:0/3458535685 >> 172.21.15.61:0/3458535685 conn(0x560f24adf720 msgr2=0x560f24b65b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 -- 172.21.15.61:0/3458535685 shutdown_connections 2023-10-18T09:28:56.028 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.022+0000 7fccb0652b80 1 -- 172.21.15.61:0/3458535685 wait complete. 2023-10-18T09:28:56.029 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.023+0000 7fccb0652b80 1 -- start start 2023-10-18T09:28:56.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.023+0000 7fccb0652b80 1 Processor -- start 2023-10-18T09:28:56.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.025+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.025+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.025+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b8d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.030 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.026+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.026+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45378/0 (socket says 172.21.15.61:45378) 2023-10-18T09:28:56.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 0x7fcc840a3ac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000a600 tx=0x7fcc9000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.031 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 -- 172.21.15.61:0/1068931025 learned_addr learned my addr 172.21.15.61:0/1068931025 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24abde70 con 0x560f24b6b5d0 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24a5a710 con 0x560f24b6cc30 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24a5a3b0 con 0x560f24b6c2d0 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b8d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fccb0652b80 10 client.0 _ll_register_callbacks cb 0x560f249a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:28:56.032 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 -- 172.21.15.61:0/1068931025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 msgr2=0x560f24b86120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:28:56.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b86120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 -- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 msgr2=0x560f24b8d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b8d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fcc9659c700 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8800b740 con 0x560f24b6b5d0 2023-10-18T09:28:56.033 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.028+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f24ac2ef0 con 0x560f24b6b5d0 2023-10-18T09:28:56.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.029+0000 7fcc9659c700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc800abc20 tx=0x7fcc8002b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5362ad9e995fa772 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.034 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.029+0000 7fcc7dffb700 10 client.16158 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:28:56.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.029+0000 7fcc7dffb700 1 -- 172.21.15.61:0/1068931025 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800a3540 con 0x560f24b6b5d0 2023-10-18T09:28:56.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.029+0000 7fcc7dffb700 1 -- 172.21.15.61:0/1068931025 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc800a3e50 con 0x560f24b6b5d0 2023-10-18T09:28:56.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.030+0000 7fcc7dffb700 1 -- 172.21.15.61:0/1068931025 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc800aa0c0 con 0x560f24b6b5d0 2023-10-18T09:28:56.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.030+0000 7fccb0652b80 10 client.16158 fetch_fsmap learned FSMap version 43 2023-10-18T09:28:56.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.030+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({fsmap=43}) v3 -- 0x560f248e8de0 con 0x560f24b6b5d0 2023-10-18T09:28:56.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fcc7dffb700 1 -- 172.21.15.61:0/1068931025 <== mon.0 v2:172.21.15.61:3300/0 4 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc800abcc0 con 0x560f24b6b5d0 2023-10-18T09:28:56.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 10 client.16158 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:28:56.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:28:56.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 20 client.16158 populate_metadata read hostname 'smithi061' 2023-10-18T09:28:56.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24b939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 4 client.16158 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:28:56.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.031+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f24b3a8b0 con 0x560f24b91570 2023-10-18T09:28:56.037 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.032+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24b939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.038 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.032+0000 7fcc7dffb700 10 client.16158 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:56.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.032+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24b939b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=595e8900b5707f8f server_cookie=3f3a38f911fce8c7 in_seq=0 out_seq=0 2023-10-18T09:28:56.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.033+0000 7fcc7dffb700 1 -- 172.21.15.61:0/1068931025 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560f24b3a8b0 con 0x560f24b91570 2023-10-18T09:28:56.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.033+0000 7fcc7dffb700 10 client.16158 handle_command_reply: tid=0 2023-10-18T09:28:56.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.033+0000 7fccb0652b80 1 client.16158 shutdown 2023-10-18T09:28:56.039 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 msgr2=0x560f24b85be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc800abc20 tx=0x7fcc8002b890 comp rx=0 tx=0).stop 2023-10-18T09:28:56.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 shutdown_connections 2023-10-18T09:28:56.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24b939b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fcc7dffb700 0 client.16158 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:56.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b8d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b85be0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 --2- 172.21.15.61:0/1068931025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b86120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 >> 172.21.15.61:0/1068931025 conn(0x560f24adf720 msgr2=0x560f24b65b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.035+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 shutdown_connections 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.036+0000 7fccb0652b80 1 -- 172.21.15.61:0/1068931025 wait complete. 2023-10-18T09:28:56.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.036+0000 7fccb0652b80 20 client.16158 trim_cache size 0 max 16384 2023-10-18T09:28:56.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.071+0000 7fccb0652b80 1 Processor -- start 2023-10-18T09:28:56.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.071+0000 7fccb0652b80 1 -- start start 2023-10-18T09:28:56.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.071+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.074 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.071+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.071+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6cc30 0x560f24b605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fccb0652b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2493e920 con 0x560f24b6b5d0 2023-10-18T09:28:56.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fccb0652b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24b5e000 con 0x560f24b6c2d0 2023-10-18T09:28:56.075 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fccb0652b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24b912c0 con 0x560f24b6cc30 2023-10-18T09:28:56.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fcc95d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fcc95d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39786/0 (socket says 172.21.15.61:39786) 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.072+0000 7fcc95d9b700 1 -- 172.21.15.61:0/4232513338 learned_addr learned my addr 172.21.15.61:0/4232513338 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6cc30 0x560f24b605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc95d9b700 1 -- 172.21.15.61:0/4232513338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6cc30 msgr2=0x560f24b605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6cc30 0x560f24b605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc95d9b700 1 -- 172.21.15.61:0/4232513338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 msgr2=0x560f24b6b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.073+0000 7fcc95d9b700 1 -- 172.21.15.61:0/4232513338 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f248d1c60 con 0x560f24b6c2d0 2023-10-18T09:28:56.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.074+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d5c0 tx=0x7fcc88000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02b0f787560ead2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.074+0000 7fcc8d7fa700 1 -- 172.21.15.61:0/4232513338 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88026070 con 0x560f24b6c2d0 2023-10-18T09:28:56.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.075+0000 7fcc8d7fa700 1 -- 172.21.15.61:0/4232513338 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc88004eb0 con 0x560f24b6c2d0 2023-10-18T09:28:56.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.075+0000 7fcc8d7fa700 1 -- 172.21.15.61:0/4232513338 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88020040 con 0x560f24b6c2d0 2023-10-18T09:28:56.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 -- 172.21.15.61:0/4232513338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 msgr2=0x560f24b8d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800d5c0 tx=0x7fcc88000bc0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 -- 172.21.15.61:0/4232513338 shutdown_connections 2023-10-18T09:28:56.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6cc30 0x560f24b605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b6b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 --2- 172.21.15.61:0/4232513338 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6c2d0 0x560f24b8d0d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.076+0000 7fccb0652b80 1 -- 172.21.15.61:0/4232513338 >> 172.21.15.61:0/4232513338 conn(0x560f24adf720 msgr2=0x560f24b98310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.077+0000 7fccb0652b80 1 -- 172.21.15.61:0/4232513338 shutdown_connections 2023-10-18T09:28:56.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.077+0000 7fccb0652b80 1 -- 172.21.15.61:0/4232513338 wait complete. 2023-10-18T09:28:56.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.077+0000 7fccb0652b80 1 -- start start 2023-10-18T09:28:56.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.078+0000 7fccb0652b80 1 Processor -- start 2023-10-18T09:28:56.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.078+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b8c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.078+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b8cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.078+0000 7fccb0652b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fccb0652b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24b89e90 con 0x560f24b6b5d0 2023-10-18T09:28:56.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fccb0652b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24b8a000 con 0x560f24b6cc30 2023-10-18T09:28:56.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fccb0652b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f24b8a1c0 con 0x560f24b6c2d0 2023-10-18T09:28:56.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fcc96d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fcc96d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:39802/0 (socket says 172.21.15.61:39802) 2023-10-18T09:28:56.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fcc96d9d700 1 -- 172.21.15.61:0/3026712847 learned_addr learned my addr 172.21.15.61:0/3026712847 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:28:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.079+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b8cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fccb0652b80 10 client.0 _ll_register_callbacks cb 0x560f249a4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:28:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fcc96d9d700 1 -- 172.21.15.61:0/3026712847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 msgr2=0x560f24b8cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b8cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fcc96d9d700 1 -- 172.21.15.61:0/3026712847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 msgr2=0x560f24b8c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:28:56.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b8c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.080+0000 7fcc96d9d700 1 -- 172.21.15.61:0/3026712847 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f24a7b3f0 con 0x560f24b6cc30 2023-10-18T09:28:56.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.081+0000 7fcc95d9b700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:28:56.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.081+0000 7fcc96d9d700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc90016aa0 tx=0x7fcc90004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763a0bdce0db4666 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:28:56.087 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.081+0000 7fcc7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:28:56.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.082+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004bb0 con 0x560f24b6cc30 2023-10-18T09:28:56.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.082+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcc90004d10 con 0x560f24b6cc30 2023-10-18T09:28:56.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.082+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc8800dc90 con 0x560f24b6cc30 2023-10-18T09:28:56.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.082+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc90014a60 con 0x560f24b6cc30 2023-10-18T09:28:56.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.084+0000 7fccb0652b80 10 client.26363 fetch_fsmap learned FSMap version 43 2023-10-18T09:28:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.084+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x560f2490f250 con 0x560f24b6cc30 2023-10-18T09:28:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.084+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc8800dc90 con 0x560f24b6cc30 2023-10-18T09:28:56.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcc90023c20 con 0x560f24b6cc30 2023-10-18T09:28:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 10 client.26363 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:28:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:28:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 20 client.26363 populate_metadata read hostname 'smithi061' 2023-10-18T09:28:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24be8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:28:56.090 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 4 client.26363 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:28:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.085+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560f24ab6e50 con 0x560f24b91570 2023-10-18T09:28:56.091 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.088+0000 7fcc9659c700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24be8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:28:56.092 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.090+0000 7fcc7dffb700 10 client.26363 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:56.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.090+0000 7fcc9659c700 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24be8b20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480a2cf107f1a128 server_cookie=117a7fb0bcc0cfbf in_seq=0 out_seq=0 2023-10-18T09:28:56.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.091+0000 7fcc7dffb700 1 -- 172.21.15.61:0/3026712847 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x560f24ab6e50 con 0x560f24b91570 2023-10-18T09:28:56.093 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.091+0000 7fcc7dffb700 10 client.26363 handle_command_reply: tid=0 2023-10-18T09:28:56.094 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.091+0000 7fccb0652b80 1 client.26363 shutdown 2023-10-18T09:28:56.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 msgr2=0x560f24b89750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcc90016aa0 tx=0x7fcc90004b40 comp rx=0 tx=0).stop 2023-10-18T09:28:56.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 shutdown_connections 2023-10-18T09:28:56.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560f24b91570 0x560f24be8b20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560f24b6cc30 0x560f24b89750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560f24b6b5d0 0x560f24b8c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fcc7dffb700 0 client.26363 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:28:56.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 --2- 172.21.15.61:0/3026712847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f24b6c2d0 0x560f24b8cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 >> 172.21.15.61:0/3026712847 conn(0x560f24adf720 msgr2=0x560f24b6b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.092+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 shutdown_connections 2023-10-18T09:28:56.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.093+0000 7fccb0652b80 1 -- 172.21.15.61:0/3026712847 wait complete. 2023-10-18T09:28:56.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.093+0000 7fccb0652b80 20 client.26363 trim_cache size 0 max 16384 2023-10-18T09:28:56.097 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:28:56.098 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:28:56.098 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:28:56.098 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:28:56.098 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:28:56.099 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:28:56.099 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:28:56.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:28:56.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:28:56.099 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:28:56.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 msgr2=0x7fcc840a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.100 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 0x7fcc840a3ac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000a600 tx=0x7fcc9000c040 comp rx=0 tx=0).stop 2023-10-18T09:28:56.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 msgr2=0x7fcc98198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:28:56.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcc800037f0 tx=0x7fcc8000f690 comp rx=0 tx=0).stop 2023-10-18T09:28:56.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 shutdown_connections 2023-10-18T09:28:56.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fcc840a1600 0x7fcc840a3ac0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.101 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fcc98106510 0x7fcc9819feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc981012b0 0x7fcc98198950 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.094+0000 7fcc9cdff700 1 --2- 172.21.15.61:0/2550433252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc98103be0 0x7fcc98198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:28:56.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.095+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 >> 172.21.15.61:0/2550433252 conn(0x7fcc980fac00 msgr2=0x7fcc98101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:28:56.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.095+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 shutdown_connections 2023-10-18T09:28:56.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:28:56.095+0000 7fcc9cdff700 1 -- 172.21.15.61:0/2550433252 wait complete. 2023-10-18T09:28:56.131 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:28:58.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:57 smithi110 ceph-mon[134818]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 542 op/s 2023-10-18T09:28:58.256 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:57 smithi061 ceph-mon[98573]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 542 op/s 2023-10-18T09:28:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:57 smithi170 ceph-mon[135150]: pgmap v670: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.3 MiB/s wr, 542 op/s 2023-10-18T09:28:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 585.0 s, 502.8 tps, lat 196.121 ms stddev 229.820 2023-10-18T09:28:59.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:28:59 smithi170 ceph-mon[135150]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 501 op/s 2023-10-18T09:28:59.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:28:59 smithi110 ceph-mon[134818]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 501 op/s 2023-10-18T09:28:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:28:59 smithi061 ceph-mon[98573]: pgmap v671: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.7 MiB/s wr, 501 op/s 2023-10-18T09:29:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:01 smithi170 ceph-mon[135150]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2023-10-18T09:29:01.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:01 smithi110 ceph-mon[134818]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2023-10-18T09:29:01.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:01 smithi061 ceph-mon[98573]: pgmap v672: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.9 MiB/s wr, 566 op/s 2023-10-18T09:29:03.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:03 smithi170 ceph-mon[135150]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-10-18T09:29:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:03 smithi110 ceph-mon[134818]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-10-18T09:29:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:03 smithi061 ceph-mon[98573]: pgmap v673: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 513 op/s 2023-10-18T09:29:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 590.0 s, 493.4 tps, lat 202.176 ms stddev 260.308 2023-10-18T09:29:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:05 smithi170 ceph-mon[135150]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 539 op/s 2023-10-18T09:29:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:05 smithi110 ceph-mon[134818]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 539 op/s 2023-10-18T09:29:05.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:05 smithi061 ceph-mon[98573]: pgmap v674: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.8 MiB/s wr, 539 op/s 2023-10-18T09:29:08.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:07 smithi110 ceph-mon[134818]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-18T09:29:08.255 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:07 smithi061 ceph-mon[98573]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-18T09:29:08.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:07 smithi170 ceph-mon[135150]: pgmap v675: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 6.2 MiB/s wr, 509 op/s 2023-10-18T09:29:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 595.0 s, 444.6 tps, lat 231.188 ms stddev 312.705 2023-10-18T09:29:09.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:09 smithi170 ceph-mon[135150]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2023-10-18T09:29:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:09 smithi110 ceph-mon[134818]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2023-10-18T09:29:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:09 smithi061 ceph-mon[98573]: pgmap v676: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.1 MiB/s wr, 441 op/s 2023-10-18T09:29:11.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:11 smithi170 ceph-mon[135150]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 514 op/s 2023-10-18T09:29:11.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:11 smithi110 ceph-mon[134818]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 514 op/s 2023-10-18T09:29:11.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:11 smithi061 ceph-mon[98573]: pgmap v677: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 514 op/s 2023-10-18T09:29:13.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:13 smithi170 ceph-mon[135150]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2023-10-18T09:29:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:13 smithi110 ceph-mon[134818]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2023-10-18T09:29:13.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:13 smithi061 ceph-mon[98573]: pgmap v678: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.8 MiB/s wr, 468 op/s 2023-10-18T09:29:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 600.0 s, 530.4 tps, lat 188.699 ms stddev 233.793 2023-10-18T09:29:15.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:15 smithi170 ceph-mon[135150]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-18T09:29:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:15 smithi110 ceph-mon[134818]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-18T09:29:15.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:15 smithi061 ceph-mon[98573]: pgmap v679: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.1 MiB/s wr, 500 op/s 2023-10-18T09:29:16.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:16 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:29:16.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:16 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:29:16.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:16 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:29:18.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:17 smithi110 ceph-mon[134818]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2023-10-18T09:29:18.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:17 smithi061 ceph-mon[98573]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2023-10-18T09:29:18.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:17 smithi170 ceph-mon[135150]: pgmap v680: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.3 MiB/s wr, 474 op/s 2023-10-18T09:29:19.342 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 605.0 s, 468.3 tps, lat 215.616 ms stddev 268.498 2023-10-18T09:29:20.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:20 smithi061 ceph-mon[98573]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 438 op/s 2023-10-18T09:29:20.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:20 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:20 smithi170 ceph-mon[135150]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 438 op/s 2023-10-18T09:29:20.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:20 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:20 smithi110 ceph-mon[134818]: pgmap v681: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.2 MiB/s wr, 438 op/s 2023-10-18T09:29:20.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:20.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:20 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:21 smithi170 ceph-mon[135150]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2023-10-18T09:29:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:21 smithi110 ceph-mon[134818]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2023-10-18T09:29:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:21 smithi061 ceph-mon[98573]: pgmap v682: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2023-10-18T09:29:23.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:23 smithi170 ceph-mon[135150]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 473 op/s 2023-10-18T09:29:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:23 smithi110 ceph-mon[134818]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 473 op/s 2023-10-18T09:29:23.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:23 smithi061 ceph-mon[98573]: pgmap v683: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 473 op/s 2023-10-18T09:29:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 610.0 s, 428.6 tps, lat 233.275 ms stddev 300.749 2023-10-18T09:29:24.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:24.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:24 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:24.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:24 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:25.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:25.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:24 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:25.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:29:25.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:29:25.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:25 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:25.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:25 smithi170 ceph-mon[135150]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2023-10-18T09:29:25.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:29:25.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:29:25.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:25 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:25.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:25 smithi110 ceph-mon[134818]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2023-10-18T09:29:26.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:29:26.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:29:26.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:25 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:29:26.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:25 smithi061 ceph-mon[98573]: pgmap v684: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 486 op/s 2023-10-18T09:29:26.131 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:29:26.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.492+0000 7f186a99d700 1 -- 172.21.15.61:0/1462194061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1864107e40 msgr2=0x7f18641082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.492+0000 7f186a99d700 1 --2- 172.21.15.61:0/1462194061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1864107e40 0x7f18641082a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f18540099c0 tx=0x7f1854009cd0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.494+0000 7f186a99d700 1 -- 172.21.15.61:0/1462194061 shutdown_connections 2023-10-18T09:29:26.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.494+0000 7f186a99d700 1 --2- 172.21.15.61:0/1462194061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18641087e0 0x7f186410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.494+0000 7f186a99d700 1 --2- 172.21.15.61:0/1462194061 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f1864107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.494+0000 7f186a99d700 1 --2- 172.21.15.61:0/1462194061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1864107e40 0x7f18641082a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.494+0000 7f186a99d700 1 -- 172.21.15.61:0/1462194061 >> 172.21.15.61:0/1462194061 conn(0x7f1864075bf0 msgr2=0x7f1864078030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.495+0000 7f186a99d700 1 -- 172.21.15.61:0/1462194061 shutdown_connections 2023-10-18T09:29:26.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.495+0000 7f186a99d700 1 -- 172.21.15.61:0/1462194061 wait complete. 2023-10-18T09:29:26.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.496+0000 7f186a99d700 1 Processor -- start 2023-10-18T09:29:26.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.497+0000 7f186a99d700 1 -- start start 2023-10-18T09:29:26.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.497+0000 7f186a99d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f186407f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186a99d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 0x7f186407f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f186407f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f186407f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52840/0 (socket says 172.21.15.61:52840) 2023-10-18T09:29:26.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186999b700 1 -- 172.21.15.61:0/2281397353 learned_addr learned my addr 172.21.15.61:0/2281397353 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:26.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18641087e0 0x7f18640806f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864080cc0 con 0x7f1864106c50 2023-10-18T09:29:26.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864080e30 con 0x7f18641087e0 2023-10-18T09:29:26.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864080ff0 con 0x7f1864107e40 2023-10-18T09:29:26.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.499+0000 7f186919a700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 0x7f186407f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186a19c700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18641087e0 0x7f18640806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186919a700 1 -- 172.21.15.61:0/2281397353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18641087e0 msgr2=0x7f18640806f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186919a700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18641087e0 0x7f18640806f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186919a700 1 -- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 msgr2=0x7f186407f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186919a700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f186407f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.500+0000 7f186919a700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540095f0 con 0x7f1864107e40 2023-10-18T09:29:26.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.502+0000 7f186919a700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 0x7f186407f700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1854009f50 tx=0x7f1854005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.503+0000 7f185affd700 1 -- 172.21.15.61:0/2281397353 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185401d070 con 0x7f1864107e40 2023-10-18T09:29:26.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.503+0000 7f185affd700 1 -- 172.21.15.61:0/2281397353 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1854003f90 con 0x7f1864107e40 2023-10-18T09:29:26.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.503+0000 7f185affd700 1 -- 172.21.15.61:0/2281397353 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854022bc0 con 0x7f1864107e40 2023-10-18T09:29:26.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.503+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1864081210 con 0x7f1864107e40 2023-10-18T09:29:26.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.503+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1864081640 con 0x7f1864107e40 2023-10-18T09:29:26.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.510+0000 7f185affd700 1 -- 172.21.15.61:0/2281397353 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f185400f460 con 0x7f1864107e40 2023-10-18T09:29:26.515 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.512+0000 7f185affd700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 0x7f18500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.514+0000 7f185affd700 1 -- 172.21.15.61:0/2281397353 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f18540ce440 con 0x7f1864107e40 2023-10-18T09:29:26.516 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.514+0000 7f187f438b80 1 Processor -- start 2023-10-18T09:29:26.517 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.514+0000 7f187f438b80 1 -- start start 2023-10-18T09:29:26.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666ccb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666ccc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666cccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666c3e6e0 con 0x55f666ccb4d0 2023-10-18T09:29:26.518 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666c24c60 con 0x55f666ccbe30 2023-10-18T09:29:26.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f187f438b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666c22580 con 0x55f666ccc7f0 2023-10-18T09:29:26.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666ccc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666ccc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40570/0 (socket says 172.21.15.61:40570) 2023-10-18T09:29:26.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f186919a700 1 -- 172.21.15.61:0/2803216137 learned_addr learned my addr 172.21.15.61:0/2803216137 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:26.519 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f186a19c700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666cccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.520 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.515+0000 7f186999b700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666ccb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.516+0000 7f186a19c700 1 -- 172.21.15.61:0/2803216137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 msgr2=0x55f666ccc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.516+0000 7f186a19c700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666ccc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.516+0000 7f186a19c700 1 -- 172.21.15.61:0/2803216137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 msgr2=0x55f666ccb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.521 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.517+0000 7f186a19c700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666ccb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.517+0000 7f186a19c700 1 -- 172.21.15.61:0/2803216137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f666bdb3f0 con 0x55f666ccc7f0 2023-10-18T09:29:26.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.517+0000 7f186999b700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666ccb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:29:26.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.517+0000 7f186999b700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 0x7f18500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.522 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.517+0000 7f186a19c700 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666cccc50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f185c00a5d0 tx=0x7f185c0078d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ed8a0ab2510fcd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.518+0000 7f184ffff700 1 -- 172.21.15.61:0/2803216137 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c018070 con 0x55f666ccc7f0 2023-10-18T09:29:26.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.518+0000 7f184ffff700 1 -- 172.21.15.61:0/2803216137 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c010040 con 0x55f666ccc7f0 2023-10-18T09:29:26.523 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.518+0000 7f184ffff700 1 -- 172.21.15.61:0/2803216137 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00f050 con 0x55f666ccc7f0 2023-10-18T09:29:26.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.519+0000 7f187f438b80 1 -- 172.21.15.61:0/2803216137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 msgr2=0x55f666cccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.519+0000 7f187f438b80 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666cccc50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f185c00a5d0 tx=0x7f185c0078d0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.524 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.519+0000 7f186999b700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 0x7f18500a3940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f186000bec0 tx=0x7f18600047c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 -- 172.21.15.61:0/2803216137 shutdown_connections 2023-10-18T09:29:26.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666cccc50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666ccb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 --2- 172.21.15.61:0/2803216137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666ccc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.525 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 -- 172.21.15.61:0/2803216137 >> 172.21.15.61:0/2803216137 conn(0x55f666c3f720 msgr2=0x55f666cc5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 -- 172.21.15.61:0/2803216137 shutdown_connections 2023-10-18T09:29:26.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.520+0000 7f187f438b80 1 -- 172.21.15.61:0/2803216137 wait complete. 2023-10-18T09:29:26.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.521+0000 7f187f438b80 1 -- start start 2023-10-18T09:29:26.526 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.521+0000 7f187f438b80 1 Processor -- start 2023-10-18T09:29:26.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.522+0000 7f187f438b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cd4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.522+0000 7f187f438b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.522+0000 7f187f438b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.528 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40574/0 (socket says 172.21.15.61:40574) 2023-10-18T09:29:26.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f186a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f186a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:52858/0 (socket says 172.21.15.61:52858) 2023-10-18T09:29:26.529 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f187f438b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666c1de70 con 0x55f666ccc7f0 2023-10-18T09:29:26.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f187f438b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666bba710 con 0x55f666ccbe30 2023-10-18T09:29:26.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f187f438b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666bba3b0 con 0x55f666ccb4d0 2023-10-18T09:29:26.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f186919a700 1 -- 172.21.15.61:0/964933599 learned_addr learned my addr 172.21.15.61:0/964933599 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:26.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.523+0000 7f187f438b80 10 client.0 _ll_register_callbacks cb 0x55f666b04a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:29:26.530 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186999b700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cd4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186919a700 1 -- 172.21.15.61:0/964933599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 msgr2=0x55f666cd4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186919a700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cd4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186919a700 1 -- 172.21.15.61:0/964933599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 msgr2=0x55f666cdc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186919a700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186919a700 1 -- 172.21.15.61:0/964933599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c007e00 con 0x55f666ccbe30 2023-10-18T09:29:26.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186999b700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cd4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:29:26.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f666c22ef0 con 0x55f666ccbe30 2023-10-18T09:29:26.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.524+0000 7f186a19c700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:29:26.532 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.525+0000 7f186919a700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18540cbf60 tx=0x7f18540037c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876da9da439ffd79 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.525+0000 7f184d7fa700 10 client.26375 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:29:26.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.525+0000 7f184d7fa700 1 -- 172.21.15.61:0/964933599 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185409bba0 con 0x55f666ccbe30 2023-10-18T09:29:26.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.525+0000 7f184d7fa700 1 -- 172.21.15.61:0/964933599 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f185409b4a0 con 0x55f666ccbe30 2023-10-18T09:29:26.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.525+0000 7f184d7fa700 1 -- 172.21.15.61:0/964933599 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f185409f5f0 con 0x55f666ccbe30 2023-10-18T09:29:26.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 10 client.26375 fetch_fsmap learned FSMap version 43 2023-10-18T09:29:26.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=43}) v3 -- 0x55f666a48de0 con 0x55f666ccbe30 2023-10-18T09:29:26.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f184d7fa700 1 -- 172.21.15.61:0/964933599 <== mon.2 v2:172.21.15.110:3300/0 4 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f185409f170 con 0x55f666ccbe30 2023-10-18T09:29:26.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 10 client.26375 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:29:26.535 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:29:26.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 20 client.26375 populate_metadata read hostname 'smithi061' 2023-10-18T09:29:26.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666ce04c0 0x55f666ce28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 4 client.26375 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:29:26.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.526+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f666c9a8b0 con 0x55f666ce04c0 2023-10-18T09:29:26.536 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.527+0000 7f186999b700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666ce04c0 0x55f666ce28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.529+0000 7f184d7fa700 10 client.26375 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:26.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.529+0000 7f186999b700 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666ce04c0 0x55f666ce28e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5756628e738f81d server_cookie=43b8f3f5384e2fe2 in_seq=0 out_seq=0 2023-10-18T09:29:26.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.530+0000 7f184d7fa700 1 -- 172.21.15.61:0/964933599 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f666c9a8b0 con 0x55f666ce04c0 2023-10-18T09:29:26.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.530+0000 7f184d7fa700 10 client.26375 handle_command_reply: tid=0 2023-10-18T09:29:26.537 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.530+0000 7f187f438b80 1 client.26375 shutdown 2023-10-18T09:29:26.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.530+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 msgr2=0x55f666cd51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.530+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18540cbf60 tx=0x7f18540037c0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 shutdown_connections 2023-10-18T09:29:26.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666ce04c0 0x55f666ce28e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.538 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccc7f0 0x55f666cdc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cd4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 --2- 172.21.15.61:0/964933599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cd51d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 >> 172.21.15.61:0/964933599 conn(0x55f666c3f720 msgr2=0x55f666cca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f184d7fa700 0 client.26375 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:26.539 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 shutdown_connections 2023-10-18T09:29:26.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.531+0000 7f187f438b80 1 -- 172.21.15.61:0/964933599 wait complete. 2023-10-18T09:29:26.540 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.532+0000 7f187f438b80 20 client.26375 trim_cache size 0 max 16384 2023-10-18T09:29:26.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 Processor -- start 2023-10-18T09:29:26.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 -- start start 2023-10-18T09:29:26.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccbe30 0x55f666cdc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666a9e920 con 0x55f666ccbe30 2023-10-18T09:29:26.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666cd5b90 con 0x55f666ccc7f0 2023-10-18T09:29:26.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.572+0000 7f187f438b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666cdd390 con 0x55f666ccb4d0 2023-10-18T09:29:26.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.573+0000 7f186a19c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.573+0000 7f186a19c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40582/0 (socket says 172.21.15.61:40582) 2023-10-18T09:29:26.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.573+0000 7f186a19c700 1 -- 172.21.15.61:0/3034726769 learned_addr learned my addr 172.21.15.61:0/3034726769 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:26.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.573+0000 7f186919a700 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccbe30 0x55f666cdc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 -- 172.21.15.61:0/3034726769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 msgr2=0x55f666cca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 -- 172.21.15.61:0/3034726769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccbe30 msgr2=0x55f666cdc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccbe30 0x55f666cdc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 -- 172.21.15.61:0/3034726769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f666a31c60 con 0x55f666ccc7f0 2023-10-18T09:29:26.579 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f186a19c700 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f185c00a570 tx=0x7f185c0076c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5d66d5f4a30fd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.574+0000 7f1858bff700 1 -- 172.21.15.61:0/3034726769 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c018030 con 0x55f666ccc7f0 2023-10-18T09:29:26.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.575+0000 7f1858bff700 1 -- 172.21.15.61:0/3034726769 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c019040 con 0x55f666ccc7f0 2023-10-18T09:29:26.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.575+0000 7f1858bff700 1 -- 172.21.15.61:0/3034726769 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c004700 con 0x55f666ccc7f0 2023-10-18T09:29:26.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 -- 172.21.15.61:0/3034726769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 msgr2=0x55f666cca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.580 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f185c00a570 tx=0x7f185c0076c0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 -- 172.21.15.61:0/3034726769 shutdown_connections 2023-10-18T09:29:26.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccc7f0 0x55f666cca810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccb4d0 0x55f666cca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 --2- 172.21.15.61:0/3034726769 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccbe30 0x55f666cdc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 -- 172.21.15.61:0/3034726769 >> 172.21.15.61:0/3034726769 conn(0x55f666c3f720 msgr2=0x55f666cd4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 -- 172.21.15.61:0/3034726769 shutdown_connections 2023-10-18T09:29:26.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.576+0000 7f187f438b80 1 -- 172.21.15.61:0/3034726769 wait complete. 2023-10-18T09:29:26.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.577+0000 7f187f438b80 1 -- start start 2023-10-18T09:29:26.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 Processor -- start 2023-10-18T09:29:26.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666cdb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cdbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666d34630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666d34b70 con 0x55f666ccb4d0 2023-10-18T09:29:26.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666d34ce0 con 0x55f666ccbe30 2023-10-18T09:29:26.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f666d34ea0 con 0x55f666ccc7f0 2023-10-18T09:29:26.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.578+0000 7f187f438b80 10 client.0 _ll_register_callbacks cb 0x55f666b04a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:29:26.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.579+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cdbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.579+0000 7f186919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cdbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40584/0 (socket says 172.21.15.61:40584) 2023-10-18T09:29:26.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.579+0000 7f186919a700 1 -- 172.21.15.61:0/1488669708 learned_addr learned my addr 172.21.15.61:0/1488669708 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:26.585 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.579+0000 7f186a19c700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666d34630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.579+0000 7f186999b700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666cdb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 -- 172.21.15.61:0/1488669708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 msgr2=0x55f666d34630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666d34630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 -- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 msgr2=0x55f666cdbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:26.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cdbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 -- 172.21.15.61:0/1488669708 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f666bdb3f0 con 0x55f666ccb4d0 2023-10-18T09:29:26.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.580+0000 7f186999b700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666cdb5b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55f666ce31b0 tx=0x7f1860004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ff426606c71f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:26.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.581+0000 7f184d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:29:26.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.581+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860024040 con 0x55f666ccb4d0 2023-10-18T09:29:26.589 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.581+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1860018070 con 0x55f666ccb4d0 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.582+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f185c010040 con 0x55f666ccb4d0 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.582+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18600264b0 con 0x55f666ccb4d0 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.582+0000 7f187f438b80 10 client.16173 fetch_fsmap learned FSMap version 43 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.582+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f666a6f250 con 0x55f666ccb4d0 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c010040 con 0x55f666ccb4d0 2023-10-18T09:29:26.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f1860026d10 con 0x55f666ccb4d0 2023-10-18T09:29:26.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 10 client.16173 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:29:26.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:29:26.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 20 client.16173 populate_metadata read hostname 'smithi061' 2023-10-18T09:29:26.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666d36900 0x55f666d38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:26.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 4 client.16173 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:29:26.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.583+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f666c16e50 con 0x55f666d36900 2023-10-18T09:29:26.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.584+0000 7f186919a700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666d36900 0x55f666d38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:26.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.584+0000 7f186919a700 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666d36900 0x55f666d38cf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dad62564d9daf7 server_cookie=646da535a33dd139 in_seq=0 out_seq=0 2023-10-18T09:29:26.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.584+0000 7f184d7fa700 10 client.16173 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:26.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f184d7fa700 1 -- 172.21.15.61:0/1488669708 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f666c16e50 con 0x55f666d36900 2023-10-18T09:29:26.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f184d7fa700 10 client.16173 handle_command_reply: tid=0 2023-10-18T09:29:26.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 client.16173 shutdown 2023-10-18T09:29:26.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 msgr2=0x55f666cdb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666cdb5b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55f666ce31b0 tx=0x7f1860004750 comp rx=0 tx=0).stop 2023-10-18T09:29:26.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 shutdown_connections 2023-10-18T09:29:26.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f666d36900 0x55f666d38cf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f666ccc7f0 0x55f666d34630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f666ccb4d0 0x55f666cdb5b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 --2- 172.21.15.61:0/1488669708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f666ccbe30 0x55f666cdbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f184d7fa700 0 client.16173 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:26.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 >> 172.21.15.61:0/1488669708 conn(0x55f666c3f720 msgr2=0x55f666cd4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.585+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 shutdown_connections 2023-10-18T09:29:26.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f187f438b80 1 -- 172.21.15.61:0/1488669708 wait complete. 2023-10-18T09:29:26.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f187f438b80 20 client.16173 trim_cache size 0 max 16384 2023-10-18T09:29:26.595 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:29:26.596 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:29:26.596 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:29:26.596 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:29:26.596 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:29:26.596 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:29:26.597 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:29:26.597 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:29:26.597 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:29:26.597 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:29:26.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 msgr2=0x7f18500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 0x7f18500a3940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f186000bec0 tx=0x7f18600047c0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 msgr2=0x7f186407f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:26.598 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 0x7f186407f700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1854009f50 tx=0x7f1854005740 comp rx=0 tx=0).stop 2023-10-18T09:29:26.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 shutdown_connections 2023-10-18T09:29:26.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f18500a1480 0x7f18500a3940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18641087e0 0x7f18640806f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1864106c50 0x7f186407f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.599 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 --2- 172.21.15.61:0/2281397353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1864107e40 0x7f186407f700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:26.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.586+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 >> 172.21.15.61:0/2281397353 conn(0x7f1864075bf0 msgr2=0x7f186410b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:26.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.587+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 shutdown_connections 2023-10-18T09:29:26.600 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:26.587+0000 7f186a99d700 1 -- 172.21.15.61:0/2281397353 wait complete. 2023-10-18T09:29:26.612 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:29:28.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:27 smithi061 ceph-mon[98573]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-10-18T09:29:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:28 smithi170 ceph-mon[135150]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-10-18T09:29:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:27 smithi110 ceph-mon[134818]: pgmap v685: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 5.6 MiB/s wr, 471 op/s 2023-10-18T09:29:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 615.0 s, 492.2 tps, lat 200.825 ms stddev 212.300 2023-10-18T09:29:29.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:29 smithi170 ceph-mon[135150]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2023-10-18T09:29:29.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:29 smithi110 ceph-mon[134818]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2023-10-18T09:29:29.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:29 smithi061 ceph-mon[98573]: pgmap v686: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 5.1 MiB/s wr, 424 op/s 2023-10-18T09:29:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:31 smithi170 ceph-mon[135150]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-18T09:29:31.726 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:31 smithi110 ceph-mon[134818]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-18T09:29:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:31 smithi061 ceph-mon[98573]: pgmap v687: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 5.6 MiB/s wr, 489 op/s 2023-10-18T09:29:33.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:33 smithi170 ceph-mon[135150]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2023-10-18T09:29:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:33 smithi110 ceph-mon[134818]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2023-10-18T09:29:33.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:33 smithi061 ceph-mon[98573]: pgmap v688: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.0 MiB/s wr, 441 op/s 2023-10-18T09:29:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 620.0 s, 514.6 tps, lat 190.462 ms stddev 228.735 2023-10-18T09:29:35.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:35 smithi170 ceph-mon[135150]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 490 op/s 2023-10-18T09:29:35.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:35 smithi110 ceph-mon[134818]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 490 op/s 2023-10-18T09:29:35.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:35 smithi061 ceph-mon[98573]: pgmap v689: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.9 MiB/s wr, 490 op/s 2023-10-18T09:29:38.212 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:37 smithi110 ceph-mon[134818]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 489 op/s 2023-10-18T09:29:38.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:37 smithi061 ceph-mon[98573]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 489 op/s 2023-10-18T09:29:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:37 smithi170 ceph-mon[135150]: pgmap v690: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.7 MiB/s wr, 489 op/s 2023-10-18T09:29:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 625.0 s, 499.2 tps, lat 204.956 ms stddev 269.713 2023-10-18T09:29:39.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:39 smithi170 ceph-mon[135150]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 442 op/s 2023-10-18T09:29:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:39 smithi110 ceph-mon[134818]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 442 op/s 2023-10-18T09:29:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:39 smithi061 ceph-mon[98573]: pgmap v691: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.0 MiB/s wr, 442 op/s 2023-10-18T09:29:41.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:41 smithi170 ceph-mon[135150]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 531 op/s 2023-10-18T09:29:41.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:41 smithi110 ceph-mon[134818]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 531 op/s 2023-10-18T09:29:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:41 smithi061 ceph-mon[98573]: pgmap v692: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.3 MiB/s wr, 531 op/s 2023-10-18T09:29:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:43 smithi170 ceph-mon[135150]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2023-10-18T09:29:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:43 smithi110 ceph-mon[134818]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2023-10-18T09:29:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:43 smithi061 ceph-mon[98573]: pgmap v693: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.5 MiB/s wr, 482 op/s 2023-10-18T09:29:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 630.0 s, 515.4 tps, lat 194.769 ms stddev 236.183 2023-10-18T09:29:45.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:45 smithi170 ceph-mon[135150]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 495 op/s 2023-10-18T09:29:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:45 smithi110 ceph-mon[134818]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 495 op/s 2023-10-18T09:29:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:45 smithi061 ceph-mon[98573]: pgmap v694: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 5.4 MiB/s wr, 495 op/s 2023-10-18T09:29:48.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:48 smithi170 ceph-mon[135150]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2023-10-18T09:29:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:48 smithi110 ceph-mon[134818]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2023-10-18T09:29:48.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:48 smithi061 ceph-mon[98573]: pgmap v695: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 5.2 MiB/s wr, 498 op/s 2023-10-18T09:29:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 635.0 s, 463.0 tps, lat 210.557 ms stddev 234.129 2023-10-18T09:29:49.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:49 smithi170 ceph-mon[135150]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:29:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:49 smithi110 ceph-mon[134818]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:29:49.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:49 smithi061 ceph-mon[98573]: pgmap v696: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.9 MiB/s wr, 460 op/s 2023-10-18T09:29:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:51 smithi170 ceph-mon[135150]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 543 op/s 2023-10-18T09:29:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:51 smithi110 ceph-mon[134818]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 543 op/s 2023-10-18T09:29:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:51 smithi061 ceph-mon[98573]: pgmap v697: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 543 op/s 2023-10-18T09:29:53.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:53 smithi170 ceph-mon[135150]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2023-10-18T09:29:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:53 smithi110 ceph-mon[134818]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2023-10-18T09:29:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:53 smithi061 ceph-mon[98573]: pgmap v698: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 480 op/s 2023-10-18T09:29:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 640.0 s, 521.8 tps, lat 193.045 ms stddev 260.156 2023-10-18T09:29:55.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:55 smithi170 ceph-mon[135150]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2023-10-18T09:29:55.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:55 smithi110 ceph-mon[134818]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2023-10-18T09:29:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:55 smithi061 ceph-mon[98573]: pgmap v699: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.5 MiB/s wr, 508 op/s 2023-10-18T09:29:56.614 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:29:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.386+0000 7f33755ff700 1 -- 172.21.15.61:0/3398425645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3370102360 msgr2=0x7f3370108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.389 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.386+0000 7f33755ff700 1 --2- 172.21.15.61:0/3398425645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3370102360 0x7f3370108c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f336800b0d0 tx=0x7f336800b3e0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.388+0000 7f33755ff700 1 -- 172.21.15.61:0/3398425645 shutdown_connections 2023-10-18T09:29:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.388+0000 7f33755ff700 1 --2- 172.21.15.61:0/3398425645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3370102360 0x7f3370108c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.388+0000 7f33755ff700 1 --2- 172.21.15.61:0/3398425645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701007d0 0x7f3370100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.388+0000 7f33755ff700 1 --2- 172.21.15.61:0/3398425645 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33701019c0 0x7f3370101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.388+0000 7f33755ff700 1 -- 172.21.15.61:0/3398425645 >> 172.21.15.61:0/3398425645 conn(0x7f33700fbd40 msgr2=0x7f33700fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.389+0000 7f33755ff700 1 -- 172.21.15.61:0/3398425645 shutdown_connections 2023-10-18T09:29:57.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.389+0000 7f33755ff700 1 -- 172.21.15.61:0/3398425645 wait complete. 2023-10-18T09:29:57.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.392+0000 7f33755ff700 1 Processor -- start 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 -- start start 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 0x7f337010f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 0x7f337010fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3370102360 0x7f3370110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33701112b0 con 0x7f3370102360 2023-10-18T09:29:57.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3370111420 con 0x7f33701007d0 2023-10-18T09:29:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f33755ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33701115e0 con 0x7f33701019c0 2023-10-18T09:29:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f336effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 0x7f337010f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f336e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 0x7f337010fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.393+0000 7f336effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 0x7f337010f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60726/0 (socket says 172.21.15.61:60726) 2023-10-18T09:29:57.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336effd700 1 -- 172.21.15.61:0/288686384 learned_addr learned my addr 172.21.15.61:0/288686384 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:57.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336f7fe700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3370102360 0x7f3370110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336e7fc700 1 -- 172.21.15.61:0/288686384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 msgr2=0x7f337010f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336e7fc700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 0x7f337010f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336e7fc700 1 -- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3370102360 msgr2=0x7f3370110ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336e7fc700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3370102360 0x7f3370110ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.394+0000 7f336e7fc700 1 -- 172.21.15.61:0/288686384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368009d00 con 0x7f33701019c0 2023-10-18T09:29:57.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.395+0000 7f336e7fc700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 0x7f337010fcf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f335800e9f0 tx=0x7f335800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.395+0000 7f3367fff700 1 -- 172.21.15.61:0/288686384 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800cbd0 con 0x7f33701019c0 2023-10-18T09:29:57.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.396+0000 7f3367fff700 1 -- 172.21.15.61:0/288686384 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f335800cd30 con 0x7f33701019c0 2023-10-18T09:29:57.400 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.396+0000 7f3367fff700 1 -- 172.21.15.61:0/288686384 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800cbd0 con 0x7f33701019c0 2023-10-18T09:29:57.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.397+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3370111860 con 0x7f33701019c0 2023-10-18T09:29:57.401 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.397+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3370111cf0 con 0x7f33701019c0 2023-10-18T09:29:57.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.404+0000 7f3388e29b80 1 Processor -- start 2023-10-18T09:29:57.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.404+0000 7f3388e29b80 1 -- start start 2023-10-18T09:29:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.405+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.406+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.406+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60740/0 (socket says 172.21.15.61:60740) 2023-10-18T09:29:57.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f336f7fe700 1 -- 172.21.15.61:0/3142564081 learned_addr learned my addr 172.21.15.61:0/3142564081 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd43f6e0 con 0x5607fd4cdc30 2023-10-18T09:29:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd425c60 con 0x5607fd4cd2d0 2023-10-18T09:29:57.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.407+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd423580 con 0x5607fd4cc5d0 2023-10-18T09:29:57.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.409+0000 7f3367fff700 1 -- 172.21.15.61:0/288686384 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f3358004020 con 0x7f33701019c0 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.410+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f336e7fc700 1 -- 172.21.15.61:0/3142564081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 msgr2=0x5607fd4cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f336e7fc700 1 -- 172.21.15.61:0/3142564081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 msgr2=0x5607fd4cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f336e7fc700 1 -- 172.21.15.61:0/3142564081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607fd3dc3f0 con 0x5607fd4cc5d0 2023-10-18T09:29:57.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.411+0000 7f3367fff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 0x7f335c0a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.412+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4cca50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f33580ca760 tx=0x7f33580d0cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ef70d735f4e6c7d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.412+0000 7f33657fa700 1 -- 172.21.15.61:0/3142564081 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580d8070 con 0x5607fd4cc5d0 2023-10-18T09:29:57.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.412+0000 7f336f7fe700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 0x7f335c0a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.412+0000 7f33657fa700 1 -- 172.21.15.61:0/3142564081 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33580d0e80 con 0x5607fd4cc5d0 2023-10-18T09:29:57.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.412+0000 7f33657fa700 1 -- 172.21.15.61:0/3142564081 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358099fe0 con 0x5607fd4cc5d0 2023-10-18T09:29:57.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.413+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 msgr2=0x5607fd4cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.413+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4cca50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f33580ca760 tx=0x7f33580d0cf0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f336f7fe700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 0x7f335c0a3ae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3368009610 tx=0x7f336800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 shutdown_connections 2023-10-18T09:29:57.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4cca50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3142564081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 >> 172.21.15.61:0/3142564081 conn(0x5607fd440720 msgr2=0x5607fd4c6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.413+0000 7f3367fff700 1 -- 172.21.15.61:0/288686384 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3358014070 con 0x7f33701019c0 2023-10-18T09:29:57.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 shutdown_connections 2023-10-18T09:29:57.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.414+0000 7f3388e29b80 1 -- 172.21.15.61:0/3142564081 wait complete. 2023-10-18T09:29:57.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.415+0000 7f3388e29b80 1 -- start start 2023-10-18T09:29:57.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.415+0000 7f3388e29b80 1 Processor -- start 2023-10-18T09:29:57.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd41ee70 con 0x5607fd4cc5d0 2023-10-18T09:29:57.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd3bb710 con 0x5607fd4cd2d0 2023-10-18T09:29:57.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd3bb3b0 con 0x5607fd4cdc30 2023-10-18T09:29:57.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.416+0000 7f3388e29b80 10 client.0 _ll_register_callbacks cb 0x5607fd305a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:29:57.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60750/0 (socket says 172.21.15.61:60750) 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 -- 172.21.15.61:0/3598597482 learned_addr learned my addr 172.21.15.61:0/3598597482 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 -- 172.21.15.61:0/3598597482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 msgr2=0x5607fd4dd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4dd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 -- 172.21.15.61:0/3598597482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cc5d0 msgr2=0x5607fd4d5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 -- 172.21.15.61:0/3598597482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3370111cf0 con 0x5607fd4cd2d0 2023-10-18T09:29:57.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f33567fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:29:57.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.417+0000 7f336e7fc700 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f33580ca730 tx=0x7f33580ca790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a126931d5280f62 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580d0370 con 0x5607fd4cd2d0 2023-10-18T09:29:57.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f33580d0e80 con 0x5607fd4cd2d0 2023-10-18T09:29:57.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607fd49b8b0 con 0x5607fd4cd2d0 2023-10-18T09:29:57.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33580d0370 con 0x5607fd4cd2d0 2023-10-18T09:29:57.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f3388e29b80 10 client.26393 fetch_fsmap learned FSMap version 43 2023-10-18T09:29:57.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.418+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5607fd249de0 con 0x5607fd4cd2d0 2023-10-18T09:29:57.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3358096320 con 0x5607fd4cd2d0 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33580994d0 con 0x5607fd4cd2d0 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 20 client.26393 populate_metadata read hostname 'smithi061' 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd4e1610 0x5607fd4e3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 4 client.26393 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:29:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607fd49b8b0 con 0x5607fd4e1610 2023-10-18T09:29:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.419+0000 7f336effd700 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd4e1610 0x5607fd4e3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.420+0000 7f33567fc700 10 client.26393 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.422+0000 7f336effd700 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd4e1610 0x5607fd4e3ad0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3c0ab358fcae server_cookie=32dd04c0cd52da43 in_seq=0 out_seq=0 2023-10-18T09:29:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.423+0000 7f33567fc700 1 -- 172.21.15.61:0/3598597482 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607fd49b8b0 con 0x5607fd4e1610 2023-10-18T09:29:57.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.423+0000 7f33567fc700 10 client.26393 handle_command_reply: tid=0 2023-10-18T09:29:57.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.423+0000 7f3388e29b80 1 client.26393 shutdown 2023-10-18T09:29:57.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 msgr2=0x5607fd4d6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f33580ca730 tx=0x7f33580ca790 comp rx=0 tx=0).stop 2023-10-18T09:29:57.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 shutdown_connections 2023-10-18T09:29:57.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd4e1610 0x5607fd4e3ad0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4dd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f33567fc700 0 client.26393 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:57.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 --2- 172.21.15.61:0/3598597482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4d6070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.424+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 >> 172.21.15.61:0/3598597482 conn(0x5607fd440720 msgr2=0x5607fd4cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.425+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 shutdown_connections 2023-10-18T09:29:57.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.425+0000 7f3388e29b80 1 -- 172.21.15.61:0/3598597482 wait complete. 2023-10-18T09:29:57.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.426+0000 7f3388e29b80 20 client.26393 trim_cache size 0 max 16384 2023-10-18T09:29:57.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 Processor -- start 2023-10-18T09:29:57.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 -- start start 2023-10-18T09:29:57.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd29f920 con 0x5607fd4cdc30 2023-10-18T09:29:57.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd4c8b80 con 0x5607fd4cd2d0 2023-10-18T09:29:57.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.464+0000 7f3388e29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd4d6d20 con 0x5607fd4cc5d0 2023-10-18T09:29:57.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60752/0 (socket says 172.21.15.61:60752) 2023-10-18T09:29:57.469 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 -- 172.21.15.61:0/1079680587 learned_addr learned my addr 172.21.15.61:0/1079680587 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:57.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336effd700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336f7fe700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 -- 172.21.15.61:0/1079680587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 msgr2=0x5607fd4d4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 -- 172.21.15.61:0/1079680587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 msgr2=0x5607fd4d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4d5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336e7fc700 1 -- 172.21.15.61:0/1079680587 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607fd232c60 con 0x5607fd4cd2d0 2023-10-18T09:29:57.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.465+0000 7f336f7fe700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:29:57.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.466+0000 7f336e7fc700 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3358096a50 tx=0x7f33580d0ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe87a5aa7d26b2f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.466+0000 7f3365ffb700 1 -- 172.21.15.61:0/1079680587 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580d0370 con 0x5607fd4cd2d0 2023-10-18T09:29:57.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.466+0000 7f3365ffb700 1 -- 172.21.15.61:0/1079680587 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3358096320 con 0x5607fd4cd2d0 2023-10-18T09:29:57.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.466+0000 7f3365ffb700 1 -- 172.21.15.61:0/1079680587 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33580d0530 con 0x5607fd4cd2d0 2023-10-18T09:29:57.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 -- 172.21.15.61:0/1079680587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 msgr2=0x5607fd4dd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3358096a50 tx=0x7f33580d0ca0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 -- 172.21.15.61:0/1079680587 shutdown_connections 2023-10-18T09:29:57.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cdc30 0x5607fd4d5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cc5d0 0x5607fd4d4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 --2- 172.21.15.61:0/1079680587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dd070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.469+0000 7f3388e29b80 1 -- 172.21.15.61:0/1079680587 >> 172.21.15.61:0/1079680587 conn(0x5607fd440720 msgr2=0x5607fd4cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.470+0000 7f3388e29b80 1 -- 172.21.15.61:0/1079680587 shutdown_connections 2023-10-18T09:29:57.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.470+0000 7f3388e29b80 1 -- 172.21.15.61:0/1079680587 wait complete. 2023-10-18T09:29:57.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.472+0000 7f3388e29b80 1 -- start start 2023-10-18T09:29:57.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.473+0000 7f3388e29b80 1 Processor -- start 2023-10-18T09:29:57.476 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.477 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd4db7d0 con 0x5607fd4cd2d0 2023-10-18T09:29:57.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd4db940 con 0x5607fd4cc5d0 2023-10-18T09:29:57.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fd4dbb00 con 0x5607fd4cdc30 2023-10-18T09:29:57.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f336effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.478 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f3388e29b80 10 client.0 _ll_register_callbacks cb 0x5607fd305a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:29:57.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f336effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60768/0 (socket says 172.21.15.61:60768) 2023-10-18T09:29:57.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.474+0000 7f336effd700 1 -- 172.21.15.61:0/2784528150 learned_addr learned my addr 172.21.15.61:0/2784528150 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:29:57.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 -- 172.21.15.61:0/2784528150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 msgr2=0x5607fd4db0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:29:57.479 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4db0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 -- 172.21.15.61:0/2784528150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cd2d0 msgr2=0x5607fd4dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:29:57.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.480 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 -- 172.21.15.61:0/2784528150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607fd249de0 con 0x5607fd4cc5d0 2023-10-18T09:29:57.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f336effd700 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f3360018560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f3713e0955476b server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:29:57.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.475+0000 7f33567fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:29:57.482 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360004220 con 0x5607fd4cc5d0 2023-10-18T09:29:57.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607fd423ef0 con 0x5607fd4cc5d0 2023-10-18T09:29:57.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3360016040 con 0x5607fd4cc5d0 2023-10-18T09:29:57.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3360004380 con 0x5607fd4cc5d0 2023-10-18T09:29:57.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f3388e29b80 10 client.26399 fetch_fsmap learned FSMap version 43 2023-10-18T09:29:57.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.476+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5607fd270250 con 0x5607fd4cc5d0 2023-10-18T09:29:57.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607fd423ef0 con 0x5607fd4cc5d0 2023-10-18T09:29:57.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3360011770 con 0x5607fd4cc5d0 2023-10-18T09:29:57.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:29:57.484 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:29:57.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 20 client.26399 populate_metadata read hostname 'smithi061' 2023-10-18T09:29:57.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd536a00 0x5607fd538e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:29:57.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 4 client.26399 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:29:57.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.477+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5607fd417e50 con 0x5607fd536a00 2023-10-18T09:29:57.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.478+0000 7f336e7fc700 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd536a00 0x5607fd538e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:29:57.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.478+0000 7f33567fc700 10 client.26399 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:57.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.478+0000 7f336e7fc700 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd536a00 0x5607fd538e20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b9c264562bf67d server_cookie=69a820ac93536bfc in_seq=0 out_seq=0 2023-10-18T09:29:57.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f33567fc700 1 -- 172.21.15.61:0/2784528150 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5607fd417e50 con 0x5607fd536a00 2023-10-18T09:29:57.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f33567fc700 10 client.26399 handle_command_reply: tid=0 2023-10-18T09:29:57.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 client.26399 shutdown 2023-10-18T09:29:57.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 msgr2=0x5607fd4dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3360000c00 tx=0x7f3360018560 comp rx=0 tx=0).stop 2023-10-18T09:29:57.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 shutdown_connections 2023-10-18T09:29:57.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5607fd536a00 0x5607fd538e20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607fd4cdc30 0x5607fd4db0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5607fd4cc5d0 0x5607fd4dca10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 --2- 172.21.15.61:0/2784528150 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5607fd4cd2d0 0x5607fd4dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f33567fc700 0 client.26399 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:29:57.488 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.479+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 >> 172.21.15.61:0/2784528150 conn(0x5607fd440720 msgr2=0x5607fd4d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.480+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 shutdown_connections 2023-10-18T09:29:57.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.480+0000 7f3388e29b80 1 -- 172.21.15.61:0/2784528150 wait complete. 2023-10-18T09:29:57.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.481+0000 7f3388e29b80 20 client.26399 trim_cache size 0 max 16384 2023-10-18T09:29:57.490 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:29:57.490 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:29:57.491 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:29:57.492 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:29:57.492 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:29:57.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.481+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 msgr2=0x7f335c0a3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.481+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 0x7f335c0a3ae0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3368009610 tx=0x7f336800b040 comp rx=0 tx=0).stop 2023-10-18T09:29:57.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 msgr2=0x7f337010fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:29:57.493 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 0x7f337010fcf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f335800e9f0 tx=0x7f335800edb0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 shutdown_connections 2023-10-18T09:29:57.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f335c0a1620 0x7f335c0a3ae0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f3370102360 0x7f3370110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33701007d0 0x7f337010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 --2- 172.21.15.61:0/288686384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33701019c0 0x7f337010fcf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:29:57.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.482+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 >> 172.21.15.61:0/288686384 conn(0x7f33700fbd40 msgr2=0x7f3370104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:29:57.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.483+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 shutdown_connections 2023-10-18T09:29:57.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:29:57.483+0000 7f33755ff700 1 -- 172.21.15.61:0/288686384 wait complete. 2023-10-18T09:29:57.512 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:29:58.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:57 smithi061 ceph-mon[98573]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-10-18T09:29:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:57 smithi170 ceph-mon[135150]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-10-18T09:29:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:57 smithi110 ceph-mon[134818]: pgmap v700: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 5.6 MiB/s wr, 497 op/s 2023-10-18T09:29:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 645.0 s, 501.8 tps, lat 199.360 ms stddev 247.417 2023-10-18T09:29:59.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:29:59 smithi170 ceph-mon[135150]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2023-10-18T09:29:59.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:29:59 smithi110 ceph-mon[134818]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2023-10-18T09:29:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:29:59 smithi061 ceph-mon[98573]: pgmap v701: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 4.6 MiB/s wr, 441 op/s 2023-10-18T09:30:00.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:00 smithi170 ceph-mon[135150]: overall HEALTH_OK 2023-10-18T09:30:00.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:00 smithi110 ceph-mon[134818]: overall HEALTH_OK 2023-10-18T09:30:00.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:00 smithi061 ceph-mon[98573]: overall HEALTH_OK 2023-10-18T09:30:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:01 smithi170 ceph-mon[135150]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 530 op/s 2023-10-18T09:30:01.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:01 smithi110 ceph-mon[134818]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 530 op/s 2023-10-18T09:30:01.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:01 smithi061 ceph-mon[98573]: pgmap v702: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.9 MiB/s wr, 530 op/s 2023-10-18T09:30:03.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:03 smithi170 ceph-mon[135150]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:30:03.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:03 smithi110 ceph-mon[134818]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:30:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:03 smithi061 ceph-mon[98573]: pgmap v703: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:30:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 650.0 s, 505.6 tps, lat 199.076 ms stddev 244.760 2023-10-18T09:30:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:05 smithi170 ceph-mon[135150]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2023-10-18T09:30:05.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:05 smithi110 ceph-mon[134818]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2023-10-18T09:30:05.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:05 smithi061 ceph-mon[98573]: pgmap v704: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 5.6 MiB/s wr, 523 op/s 2023-10-18T09:30:07.972 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:07 smithi061 ceph-mon[98573]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-18T09:30:08.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:07 smithi170 ceph-mon[135150]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-18T09:30:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:07 smithi110 ceph-mon[134818]: pgmap v705: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.2 MiB/s wr, 490 op/s 2023-10-18T09:30:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 655.0 s, 514.0 tps, lat 195.024 ms stddev 253.436 2023-10-18T09:30:09.653 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:09 smithi170 ceph-mon[135150]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-10-18T09:30:09.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:09 smithi110 ceph-mon[134818]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-10-18T09:30:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:09 smithi061 ceph-mon[98573]: pgmap v706: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 493 op/s 2023-10-18T09:30:11.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:11 smithi170 ceph-mon[135150]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-10-18T09:30:11.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:11 smithi110 ceph-mon[134818]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-10-18T09:30:11.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:11 smithi061 ceph-mon[98573]: pgmap v707: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.8 MiB/s wr, 555 op/s 2023-10-18T09:30:13.563 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:13 smithi061 ceph-mon[98573]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-10-18T09:30:13.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:13 smithi170 ceph-mon[135150]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-10-18T09:30:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:13 smithi110 ceph-mon[134818]: pgmap v708: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.0 MiB/s wr, 497 op/s 2023-10-18T09:30:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 660.0 s, 476.2 tps, lat 211.852 ms stddev 255.114 2023-10-18T09:30:15.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:15 smithi170 ceph-mon[135150]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.7 MiB/s wr, 532 op/s 2023-10-18T09:30:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:15 smithi110 ceph-mon[134818]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.7 MiB/s wr, 532 op/s 2023-10-18T09:30:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:15 smithi061 ceph-mon[98573]: pgmap v709: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.7 MiB/s wr, 532 op/s 2023-10-18T09:30:18.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:18 smithi170 ceph-mon[135150]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-18T09:30:18.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:18 smithi110 ceph-mon[134818]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-18T09:30:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:18 smithi061 ceph-mon[98573]: pgmap v710: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.7 MiB/s wr, 484 op/s 2023-10-18T09:30:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 665.0 s, 519.2 tps, lat 187.975 ms stddev 204.363 2023-10-18T09:30:19.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:19 smithi170 ceph-mon[135150]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-10-18T09:30:19.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:19 smithi110 ceph-mon[134818]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-10-18T09:30:19.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:19 smithi061 ceph-mon[98573]: pgmap v711: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.3 MiB/s wr, 496 op/s 2023-10-18T09:30:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:21 smithi170 ceph-mon[135150]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-10-18T09:30:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:21 smithi110 ceph-mon[134818]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-10-18T09:30:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:21 smithi061 ceph-mon[98573]: pgmap v712: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 540 op/s 2023-10-18T09:30:23.589 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:23 smithi061 ceph-mon[98573]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 477 op/s 2023-10-18T09:30:23.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:23 smithi170 ceph-mon[135150]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 477 op/s 2023-10-18T09:30:23.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:23 smithi110 ceph-mon[134818]: pgmap v713: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 477 op/s 2023-10-18T09:30:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 670.0 s, 487.8 tps, lat 207.900 ms stddev 215.087 2023-10-18T09:30:25.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:25 smithi170 ceph-mon[135150]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2023-10-18T09:30:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:25 smithi110 ceph-mon[134818]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2023-10-18T09:30:25.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:25 smithi061 ceph-mon[98573]: pgmap v714: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 521 op/s 2023-10-18T09:30:26.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:30:26.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:26.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:30:26.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:26.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:26.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:30:26.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:26.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.513 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:30:27.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:27 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.757 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:27 smithi061 ceph-mon[98573]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.6 MiB/s wr, 476 op/s 2023-10-18T09:30:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:27 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:27 smithi170 ceph-mon[135150]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.6 MiB/s wr, 476 op/s 2023-10-18T09:30:27.904 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 -- 172.21.15.61:0/3174674856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c103c00 msgr2=0x7fc76c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 --2- 172.21.15.61:0/3174674856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c103c00 0x7fc76c105ff0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760009ca0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 -- 172.21.15.61:0/3174674856 shutdown_connections 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 --2- 172.21.15.61:0/3174674856 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c106530 0x7fc76c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 --2- 172.21.15.61:0/3174674856 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c1012d0 0x7fc76c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 --2- 172.21.15.61:0/3174674856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c103c00 0x7fc76c105ff0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.905 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 -- 172.21.15.61:0/3174674856 >> 172.21.15.61:0/3174674856 conn(0x7fc76c0fac00 msgr2=0x7fc76c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:27.906 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 -- 172.21.15.61:0/3174674856 shutdown_connections 2023-10-18T09:30:27.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.902+0000 7fc7717ff700 1 -- 172.21.15.61:0/3174674856 wait complete. 2023-10-18T09:30:27.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.904+0000 7fc7717ff700 1 Processor -- start 2023-10-18T09:30:27.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.904+0000 7fc7717ff700 1 -- start start 2023-10-18T09:30:27.907 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.904+0000 7fc7717ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.904+0000 7fc7717ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c103c00 0x7fc76c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.908 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc7717ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c106530 0x7fc76c19ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc7717ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a0580 con 0x7fc76c103c00 2023-10-18T09:30:27.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc7717ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a06c0 con 0x7fc76c106530 2023-10-18T09:30:27.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc7717ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1a0850 con 0x7fc76c1012d0 2023-10-18T09:30:27.909 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:58274/0 (socket says 172.21.15.61:58274) 2023-10-18T09:30:27.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 -- 172.21.15.61:0/2358518761 learned_addr learned my addr 172.21.15.61:0/2358518761 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:27.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c103c00 0x7fc76c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c106530 0x7fc76c19ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.910 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 -- 172.21.15.61:0/2358518761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c106530 msgr2=0x7fc76c19ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c106530 0x7fc76c19ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 -- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c103c00 msgr2=0x7fc76c198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c103c00 0x7fc76c198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.911 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.905+0000 7fc76affd700 1 -- 172.21.15.61:0/2358518761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7600095f0 con 0x7fc76c1012d0 2023-10-18T09:30:27.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc76affd700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00b550 tx=0x7fc75c00b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc753fff700 1 -- 172.21.15.61:0/2358518761 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c011830 con 0x7fc76c1012d0 2023-10-18T09:30:27.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc753fff700 1 -- 172.21.15.61:0/2358518761 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc75c011e50 con 0x7fc76c1012d0 2023-10-18T09:30:27.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc753fff700 1 -- 172.21.15.61:0/2358518761 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00f2e0 con 0x7fc76c1012d0 2023-10-18T09:30:27.912 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c1a0b30 con 0x7fc76c1012d0 2023-10-18T09:30:27.913 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.906+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c1a0ff0 con 0x7fc76c1012d0 2023-10-18T09:30:27.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.913+0000 7fc753fff700 1 -- 172.21.15.61:0/2358518761 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fc75c011990 con 0x7fc76c1012d0 2023-10-18T09:30:27.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.915+0000 7fc753fff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 0x7fc7540a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc78505ab80 1 Processor -- start 2023-10-18T09:30:27.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc78505ab80 1 -- start start 2023-10-18T09:30:27.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc753fff700 1 -- 172.21.15.61:0/2358518761 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc75c0cca80 con 0x7fc76c1012d0 2023-10-18T09:30:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 0x564491b246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 0x7fc7540a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.916+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc78505ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a966e0 con 0x564491b235d0 2023-10-18T09:30:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc78505ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a7cc60 con 0x564491b242d0 2023-10-18T09:30:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc78505ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a7a580 con 0x564491b24c30 2023-10-18T09:30:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45236/0 (socket says 172.21.15.61:45236) 2023-10-18T09:30:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.917+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3229370255 learned_addr learned my addr 172.21.15.61:0/3229370255 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76affd700 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 0x564491b246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3229370255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 msgr2=0x564491b23090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-18T09:30:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b23090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3229370255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 msgr2=0x564491b246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 0x564491b246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3229370255 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564491a333f0 con 0x564491b235d0 2023-10-18T09:30:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76affd700 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 0x564491b246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:30:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 0x7fc7540a3a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc7600038c0 tx=0x7fc760005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.918+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc76400a430 tx=0x7fc764009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12471d37ad67f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.919+0000 7fc7517fa700 1 -- 172.21.15.61:0/3229370255 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764010040 con 0x564491b235d0 2023-10-18T09:30:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.919+0000 7fc7517fa700 1 -- 172.21.15.61:0/3229370255 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc764013a50 con 0x564491b235d0 2023-10-18T09:30:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.919+0000 7fc7517fa700 1 -- 172.21.15.61:0/3229370255 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764014aa0 con 0x564491b235d0 2023-10-18T09:30:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 -- 172.21.15.61:0/3229370255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 msgr2=0x564491b23a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc76400a430 tx=0x7fc764009650 comp rx=0 tx=0).stop 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 -- 172.21.15.61:0/3229370255 shutdown_connections 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b235d0 0x564491b23a50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b242d0 0x564491b246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3229370255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 -- 172.21.15.61:0/3229370255 >> 172.21.15.61:0/3229370255 conn(0x564491a97720 msgr2=0x564491b1db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 -- 172.21.15.61:0/3229370255 shutdown_connections 2023-10-18T09:30:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.920+0000 7fc78505ab80 1 -- 172.21.15.61:0/3229370255 wait complete. 2023-10-18T09:30:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.921+0000 7fc78505ab80 1 -- start start 2023-10-18T09:30:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 Processor -- start 2023-10-18T09:30:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b2d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a75e70 con 0x564491b242d0 2023-10-18T09:30:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a12710 con 0x564491b235d0 2023-10-18T09:30:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491a123b0 con 0x564491b24c30 2023-10-18T09:30:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.922+0000 7fc78505ab80 10 client.0 _ll_register_callbacks cb 0x56449195ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47722/0 (socket says 172.21.15.61:47722) 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 -- 172.21.15.61:0/1746133252 learned_addr learned my addr 172.21.15.61:0/1746133252 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b2d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 -- 172.21.15.61:0/1746133252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 msgr2=0x564491b340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 -- 172.21.15.61:0/1746133252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 msgr2=0x564491b2d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.923+0000 7fc76affd700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b2d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.924+0000 7fc76affd700 1 -- 172.21.15.61:0/1746133252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc764009c60 con 0x564491b235d0 2023-10-18T09:30:27.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.924+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:30:27.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.924+0000 7fc76affd700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc75c096460 tx=0x7fc75c010fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5597af55dc930f41 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.924+0000 7fc74eff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:30:27.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.925+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0999f0 con 0x564491b235d0 2023-10-18T09:30:27.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.925+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc75c09db40 con 0x564491b235d0 2023-10-18T09:30:27.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.925+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564491a7aef0 con 0x564491b235d0 2023-10-18T09:30:27.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.925+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc75c0a1d80 con 0x564491b235d0 2023-10-18T09:30:27.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.929+0000 7fc78505ab80 10 client.26405 fetch_fsmap learned FSMap version 43 2023-10-18T09:30:27.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.929+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5644918a0de0 con 0x564491b235d0 2023-10-18T09:30:27.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564491a7aef0 con 0x564491b235d0 2023-10-18T09:30:27.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc75c09ad20 con 0x564491b235d0 2023-10-18T09:30:27.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 10 client.26405 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:30:27.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 10 client.26405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:30:27.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 20 client.26405 populate_metadata read hostname 'smithi061' 2023-10-18T09:30:27.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b38830 0x564491b3ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 4 client.26405 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564491af28b0 con 0x564491b38830 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.930+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b38830 0x564491b3ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.932+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b38830 0x564491b3ac50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=210e99d286efd38e server_cookie=9267fde208fc6b0c in_seq=0 out_seq=0 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.932+0000 7fc74eff5700 10 client.26405 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.932+0000 7fc74eff5700 1 -- 172.21.15.61:0/1746133252 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564491af28b0 con 0x564491b38830 2023-10-18T09:30:27.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.932+0000 7fc74eff5700 10 client.26405 handle_command_reply: tid=0 2023-10-18T09:30:27.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 client.26405 shutdown 2023-10-18T09:30:27.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 msgr2=0x564491b2cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc75c096460 tx=0x7fc75c010fa0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 shutdown_connections 2023-10-18T09:30:27.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b38830 0x564491b3ac50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.933+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b2cb30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc78505ab80 1 --2- 172.21.15.61:0/1746133252 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b2d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 >> 172.21.15.61:0/1746133252 conn(0x564491a97720 msgr2=0x564491b1db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:27.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc74eff5700 0 client.26405 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:27.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 shutdown_connections 2023-10-18T09:30:27.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc78505ab80 1 -- 172.21.15.61:0/1746133252 wait complete. 2023-10-18T09:30:27.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.934+0000 7fc78505ab80 20 client.26405 trim_cache size 0 max 16384 2023-10-18T09:30:27.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:27 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:27.964 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:27 smithi110 ceph-mon[134818]: pgmap v715: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 4.6 MiB/s wr, 476 op/s 2023-10-18T09:30:27.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.969+0000 7fc78505ab80 1 Processor -- start 2023-10-18T09:30:27.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.969+0000 7fc78505ab80 1 -- start start 2023-10-18T09:30:27.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.969+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.970+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.972+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b185d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.972+0000 7fc78505ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644918f6920 con 0x564491b242d0 2023-10-18T09:30:27.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.972+0000 7fc78505ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491b35230 con 0x564491b235d0 2023-10-18T09:30:27.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.972+0000 7fc78505ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491b1f2e0 con 0x564491b24c30 2023-10-18T09:30:27.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.971+0000 7fc76a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.973+0000 7fc76a7fc700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45260/0 (socket says 172.21.15.61:45260) 2023-10-18T09:30:27.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.973+0000 7fc76a7fc700 1 -- 172.21.15.61:0/2065257581 learned_addr learned my addr 172.21.15.61:0/2065257581 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:27.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.979+0000 7fc76affd700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.979+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b185d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 -- 172.21.15.61:0/2065257581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 msgr2=0x564491b185d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b185d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 -- 172.21.15.61:0/2065257581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 msgr2=0x564491b23090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b23090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.983 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 -- 172.21.15.61:0/2065257581 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564491889c60 con 0x564491b242d0 2023-10-18T09:30:27.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76affd700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:30:27.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:30:27.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.980+0000 7fc76a7fc700 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6983f0bd0487805 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.981+0000 7fc751ffb700 1 -- 172.21.15.61:0/2065257581 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76002c040 con 0x564491b242d0 2023-10-18T09:30:27.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.984+0000 7fc751ffb700 1 -- 172.21.15.61:0/2065257581 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc76001e040 con 0x564491b242d0 2023-10-18T09:30:27.986 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.984+0000 7fc751ffb700 1 -- 172.21.15.61:0/2065257581 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760005200 con 0x564491b242d0 2023-10-18T09:30:27.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.985+0000 7fc78505ab80 1 -- 172.21.15.61:0/2065257581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 msgr2=0x564491b34040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:27.987 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.985+0000 7fc78505ab80 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc760009990 tx=0x7fc760022000 comp rx=0 tx=0).stop 2023-10-18T09:30:27.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.985+0000 7fc78505ab80 1 -- 172.21.15.61:0/2065257581 shutdown_connections 2023-10-18T09:30:27.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.985+0000 7fc78505ab80 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b185d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 --2- 172.21.15.61:0/2065257581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b34040 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 -- 172.21.15.61:0/2065257581 >> 172.21.15.61:0/2065257581 conn(0x564491a97720 msgr2=0x564491b239b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:27.989 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 -- 172.21.15.61:0/2065257581 shutdown_connections 2023-10-18T09:30:27.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 -- 172.21.15.61:0/2065257581 wait complete. 2023-10-18T09:30:27.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.986+0000 7fc78505ab80 1 -- start start 2023-10-18T09:30:27.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.987+0000 7fc78505ab80 1 Processor -- start 2023-10-18T09:30:27.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b300d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.992 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b30610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:27.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491b339e0 con 0x564491b242d0 2023-10-18T09:30:27.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491b33b50 con 0x564491b235d0 2023-10-18T09:30:27.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564491b33cc0 con 0x564491b24c30 2023-10-18T09:30:27.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.990+0000 7fc78505ab80 10 client.0 _ll_register_callbacks cb 0x56449195ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:30:27.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:27.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:58320/0 (socket says 172.21.15.61:58320) 2023-10-18T09:30:27.997 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3522783654 learned_addr learned my addr 172.21.15.61:0/3522783654 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:27.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 msgr2=0x564491b300d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b300d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3522783654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 msgr2=0x564491b30610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:27.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b30610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:27.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.991+0000 7fc76b7fe700 1 -- 172.21.15.61:0/3522783654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c1094f0 con 0x564491b24c30 2023-10-18T09:30:27.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.992+0000 7fc76b7fe700 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc764004c20 tx=0x7fc7640144d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb5bfe4adbec805 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:27.999 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:27.992+0000 7fc74eff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:30:28.002 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.000+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc764010040 con 0x564491b24c30 2023-10-18T09:30:28.005 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.000+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc764006e40 con 0x564491b24c30 2023-10-18T09:30:28.006 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.000+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc764013a50 con 0x564491b24c30 2023-10-18T09:30:28.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.022+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc764013590 con 0x564491b24c30 2023-10-18T09:30:28.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.022+0000 7fc78505ab80 10 client.26335 fetch_fsmap learned FSMap version 43 2023-10-18T09:30:28.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.022+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5644918c7250 con 0x564491b24c30 2023-10-18T09:30:28.025 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.022+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc764013a50 con 0x564491b24c30 2023-10-18T09:30:28.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.022+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc764008030 con 0x564491b24c30 2023-10-18T09:30:28.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 10 client.26335 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:30:28.026 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:30:28.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 20 client.26335 populate_metadata read hostname 'smithi061' 2023-10-18T09:30:28.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b8d700 0x564491b8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:28.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 4 client.26335 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:30:28.027 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.023+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564491a6ee50 con 0x564491b8d700 2023-10-18T09:30:28.035 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.033+0000 7fc76affd700 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b8d700 0x564491b8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:28.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.033+0000 7fc76affd700 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b8d700 0x564491b8fb80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff75789af4c02de server_cookie=366afe55fb1441fa in_seq=0 out_seq=0 2023-10-18T09:30:28.036 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.033+0000 7fc74eff5700 10 client.26335 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:28.039 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:30:28.039 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:30:28.039 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:30:28.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:30:28.041 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:30:28.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.034+0000 7fc74eff5700 1 -- 172.21.15.61:0/3522783654 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564491a6ee50 con 0x564491b8d700 2023-10-18T09:30:28.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.034+0000 7fc74eff5700 10 client.26335 handle_command_reply: tid=0 2023-10-18T09:30:28.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.034+0000 7fc78505ab80 1 client.26335 shutdown 2023-10-18T09:30:28.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.034+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 msgr2=0x564491b334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:28.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.034+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc764004c20 tx=0x7fc7640144d0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 shutdown_connections 2023-10-18T09:30:28.042 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x564491b8d700 0x564491b8fb80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564491b24c30 0x564491b334a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564491b235d0 0x564491b300d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc74eff5700 0 client.26335 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:28.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 --2- 172.21.15.61:0/3522783654 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x564491b242d0 0x564491b30610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 >> 172.21.15.61:0/3522783654 conn(0x564491a97720 msgr2=0x564491b2aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:28.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.035+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 shutdown_connections 2023-10-18T09:30:28.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.036+0000 7fc78505ab80 1 -- 172.21.15.61:0/3522783654 wait complete. 2023-10-18T09:30:28.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.036+0000 7fc78505ab80 20 client.26335 trim_cache size 0 max 16384 2023-10-18T09:30:28.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.037+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 msgr2=0x7fc7540a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:28.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.037+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 0x7fc7540a3a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc7600038c0 tx=0x7fc760005c00 comp rx=0 tx=0).stop 2023-10-18T09:30:28.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.037+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 msgr2=0x7fc76c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:28.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.037+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00b550 tx=0x7fc75c00b860 comp rx=0 tx=0).stop 2023-10-18T09:30:28.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 shutdown_connections 2023-10-18T09:30:28.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fc7540a15d0 0x7fc7540a3a90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc76c106530 0x7fc76c19ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc76c1012d0 0x7fc76c198a50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 --2- 172.21.15.61:0/2358518761 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fc76c103c00 0x7fc76c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:28.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 >> 172.21.15.61:0/2358518761 conn(0x7fc76c0fac00 msgr2=0x7fc76c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:28.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 shutdown_connections 2023-10-18T09:30:28.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:28.040+0000 7fc7717ff700 1 -- 172.21.15.61:0/2358518761 wait complete. 2023-10-18T09:30:28.074 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:30:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 675.0 s, 445.0 tps, lat 223.952 ms stddev 239.359 2023-10-18T09:30:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:30:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:30:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:30:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:30:29.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:30:29.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:30:29.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:30:30.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:30 smithi170 ceph-mon[135150]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2023-10-18T09:30:30.467 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:30 smithi110 ceph-mon[134818]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2023-10-18T09:30:30.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:30 smithi061 ceph-mon[98573]: pgmap v716: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 4.9 MiB/s wr, 481 op/s 2023-10-18T09:30:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:31 smithi170 ceph-mon[135150]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.2 MiB/s wr, 518 op/s 2023-10-18T09:30:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:31 smithi110 ceph-mon[134818]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.2 MiB/s wr, 518 op/s 2023-10-18T09:30:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:31 smithi061 ceph-mon[98573]: pgmap v717: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.2 MiB/s wr, 518 op/s 2023-10-18T09:30:33.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:33 smithi170 ceph-mon[135150]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2023-10-18T09:30:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:33 smithi110 ceph-mon[134818]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2023-10-18T09:30:33.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:33 smithi061 ceph-mon[98573]: pgmap v718: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 4.6 MiB/s wr, 452 op/s 2023-10-18T09:30:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 680.0 s, 585.6 tps, lat 175.640 ms stddev 201.279 2023-10-18T09:30:35.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:35 smithi170 ceph-mon[135150]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2023-10-18T09:30:35.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:35 smithi110 ceph-mon[134818]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2023-10-18T09:30:35.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:35 smithi061 ceph-mon[98573]: pgmap v719: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2023-10-18T09:30:38.254 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:37 smithi061 ceph-mon[98573]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2023-10-18T09:30:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:37 smithi170 ceph-mon[135150]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2023-10-18T09:30:38.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:37 smithi110 ceph-mon[134818]: pgmap v720: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2023-10-18T09:30:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 685.0 s, 658.4 tps, lat 150.423 ms stddev 145.470 2023-10-18T09:30:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:39 smithi170 ceph-mon[135150]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2023-10-18T09:30:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:39 smithi110 ceph-mon[134818]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2023-10-18T09:30:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:39 smithi061 ceph-mon[98573]: pgmap v721: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.0 MiB/s wr, 488 op/s 2023-10-18T09:30:41.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:41 smithi170 ceph-mon[135150]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2023-10-18T09:30:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:41 smithi110 ceph-mon[134818]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2023-10-18T09:30:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:41 smithi061 ceph-mon[98573]: pgmap v722: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.7 MiB/s wr, 575 op/s 2023-10-18T09:30:43.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:43 smithi170 ceph-mon[135150]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2023-10-18T09:30:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:43 smithi110 ceph-mon[134818]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2023-10-18T09:30:43.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:43 smithi061 ceph-mon[98573]: pgmap v723: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 5.4 MiB/s wr, 555 op/s 2023-10-18T09:30:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 690.0 s, 461.0 tps, lat 213.247 ms stddev 246.978 2023-10-18T09:30:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:45 smithi170 ceph-mon[135150]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2023-10-18T09:30:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:45 smithi110 ceph-mon[134818]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2023-10-18T09:30:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:45 smithi061 ceph-mon[98573]: pgmap v724: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2023-10-18T09:30:48.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:47 smithi061 ceph-mon[98573]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 570 op/s 2023-10-18T09:30:48.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:47 smithi170 ceph-mon[135150]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 570 op/s 2023-10-18T09:30:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:47 smithi110 ceph-mon[134818]: pgmap v725: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.6 MiB/s wr, 570 op/s 2023-10-18T09:30:49.340 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 695.0 s, 512.0 tps, lat 197.770 ms stddev 235.723 2023-10-18T09:30:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:49 smithi170 ceph-mon[135150]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2023-10-18T09:30:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:49 smithi110 ceph-mon[134818]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2023-10-18T09:30:49.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:49 smithi061 ceph-mon[98573]: pgmap v726: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 5.6 MiB/s wr, 554 op/s 2023-10-18T09:30:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:51 smithi170 ceph-mon[135150]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2023-10-18T09:30:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:51 smithi110 ceph-mon[134818]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2023-10-18T09:30:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:51 smithi061 ceph-mon[98573]: pgmap v727: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.4 MiB/s wr, 572 op/s 2023-10-18T09:30:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:53 smithi170 ceph-mon[135150]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-10-18T09:30:53.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:53 smithi110 ceph-mon[134818]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-10-18T09:30:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:53 smithi061 ceph-mon[98573]: pgmap v728: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.8 MiB/s wr, 497 op/s 2023-10-18T09:30:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 700.0 s, 480.6 tps, lat 207.815 ms stddev 260.157 2023-10-18T09:30:55.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:55 smithi170 ceph-mon[135150]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 469 op/s 2023-10-18T09:30:55.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:55 smithi110 ceph-mon[134818]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 469 op/s 2023-10-18T09:30:55.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:55 smithi061 ceph-mon[98573]: pgmap v729: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.3 MiB/s wr, 469 op/s 2023-10-18T09:30:58.076 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:30:58.396 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:58 smithi170 ceph-mon[135150]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-18T09:30:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:58 smithi110 ceph-mon[134818]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-18T09:30:58.466 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.463+0000 7f2e431ff700 1 -- 172.21.15.61:0/2016211982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c0ff690 msgr2=0x7f2e3c0ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.463+0000 7f2e431ff700 1 --2- 172.21.15.61:0/2016211982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c0ffab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009990 tx=0x7f2e30009ca0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 -- 172.21.15.61:0/2016211982 shutdown_connections 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 --2- 172.21.15.61:0/2016211982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c101220 0x7f2e3c067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 --2- 172.21.15.61:0/2016211982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c0ffab0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 --2- 172.21.15.61:0/2016211982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.467 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 -- 172.21.15.61:0/2016211982 >> 172.21.15.61:0/2016211982 conn(0x7f2e3c0fac00 msgr2=0x7f2e3c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 -- 172.21.15.61:0/2016211982 shutdown_connections 2023-10-18T09:30:58.468 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.464+0000 7f2e431ff700 1 -- 172.21.15.61:0/2016211982 wait complete. 2023-10-18T09:30:58.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.466+0000 7f2e431ff700 1 Processor -- start 2023-10-18T09:30:58.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.466+0000 7f2e431ff700 1 -- start start 2023-10-18T09:30:58.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.467+0000 7f2e431ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.470 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.467+0000 7f2e431ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.467+0000 7f2e431ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c101220 0x7f2e3c19dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.467+0000 7f2e431ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19e2f0 con 0x7f2e3c101220 2023-10-18T09:30:58.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e431ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19e430 con 0x7f2e3c100880 2023-10-18T09:30:58.471 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e431ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c19e5c0 con 0x7f2e3c0ff690 2023-10-18T09:30:58.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.472 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40306/0 (socket says 172.21.15.61:40306) 2023-10-18T09:30:58.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 -- 172.21.15.61:0/3734398054 learned_addr learned my addr 172.21.15.61:0/3734398054 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:58.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e4179c700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c101220 0x7f2e3c19dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 -- 172.21.15.61:0/3734398054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c0ff690 msgr2=0x7f2e3c1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.473 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c1967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 -- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c101220 msgr2=0x7f2e3c19dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c101220 0x7f2e3c19dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.468+0000 7f2e3bfff700 1 -- 172.21.15.61:0/3734398054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300095f0 con 0x7f2e3c100880 2023-10-18T09:30:58.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.469+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b530 tx=0x7f2e2800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.474 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.469+0000 7f2e39ffb700 1 -- 172.21.15.61:0/3734398054 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28011830 con 0x7f2e3c100880 2023-10-18T09:30:58.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.469+0000 7f2e39ffb700 1 -- 172.21.15.61:0/3734398054 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e28011e50 con 0x7f2e3c100880 2023-10-18T09:30:58.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.469+0000 7f2e39ffb700 1 -- 172.21.15.61:0/3734398054 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2800f2e0 con 0x7f2e3c100880 2023-10-18T09:30:58.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.469+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c19e8a0 con 0x7f2e3c100880 2023-10-18T09:30:58.475 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.470+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c19ed60 con 0x7f2e3c100880 2023-10-18T09:30:58.483 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.481+0000 7f2e39ffb700 1 -- 172.21.15.61:0/3734398054 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f2e28011990 con 0x7f2e3c100880 2023-10-18T09:30:58.486 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.483+0000 7f2e39ffb700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 0x7f2e2c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.484+0000 7f2e39ffb700 1 -- 172.21.15.61:0/3734398054 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2e280ccb70 con 0x7f2e3c100880 2023-10-18T09:30:58.487 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.485+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 0x7f2e2c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.489 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.487+0000 7f2e56bd6b80 1 Processor -- start 2023-10-18T09:30:58.490 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.487+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 0x7f2e2c0a39c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e30005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 -- start start 2023-10-18T09:30:58.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af6746090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.491 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6746a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af66b96e0 con 0x560af6747c30 2023-10-18T09:30:58.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af669fc60 con 0x560af67465d0 2023-10-18T09:30:58.492 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.488+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af669d580 con 0x560af67472d0 2023-10-18T09:30:58.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e4179c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6746a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.494 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46644/0 (socket says 172.21.15.61:46644) 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e40f9b700 1 -- 172.21.15.61:0/1469646088 learned_addr learned my addr 172.21.15.61:0/1469646088 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af6746090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e4179c700 1 -- 172.21.15.61:0/1469646088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 msgr2=0x560af67476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.490+0000 7f2e4179c700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e4179c700 1 -- 172.21.15.61:0/1469646088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 msgr2=0x560af6746090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.495 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e4179c700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af6746090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e4179c700 1 -- 172.21.15.61:0/1469646088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af66563f0 con 0x560af67465d0 2023-10-18T09:30:58.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:30:58.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af6746090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:30:58.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e4179c700 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6746a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e340084a0 tx=0x7f2e340087b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab496374911e1a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.496 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e26ffd700 1 -- 172.21.15.61:0/1469646088 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34003c00 con 0x560af67465d0 2023-10-18T09:30:58.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e26ffd700 1 -- 172.21.15.61:0/1469646088 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e34010040 con 0x560af67465d0 2023-10-18T09:30:58.497 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.491+0000 7f2e26ffd700 1 -- 172.21.15.61:0/1469646088 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34012760 con 0x560af67465d0 2023-10-18T09:30:58.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.492+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1469646088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 msgr2=0x560af6746a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.492+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6746a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2e340084a0 tx=0x7f2e340087b0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1469646088 shutdown_connections 2023-10-18T09:30:58.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6746a50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af67476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1469646088 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af6746090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1469646088 >> 172.21.15.61:0/1469646088 conn(0x560af66ba720 msgr2=0x560af6740b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1469646088 shutdown_connections 2023-10-18T09:30:58.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.493+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1469646088 wait complete. 2023-10-18T09:30:58.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.494+0000 7f2e56bd6b80 1 -- start start 2023-10-18T09:30:58.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.494+0000 7f2e56bd6b80 1 Processor -- start 2023-10-18T09:30:58.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.496+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 0x560af6758340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.500 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.496+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af6758880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.496+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e4179c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e4179c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40330/0 (socket says 172.21.15.61:40330) 2023-10-18T09:30:58.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e4179c700 1 -- 172.21.15.61:0/2175488924 learned_addr learned my addr 172.21.15.61:0/2175488924 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:58.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af6758880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 0x560af6758340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6698e70 con 0x560af67465d0 2023-10-18T09:30:58.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6635710 con 0x560af6747c30 2023-10-18T09:30:58.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.497+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af66353b0 con 0x560af67472d0 2023-10-18T09:30:58.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 -- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 msgr2=0x560af675f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 -- 172.21.15.61:0/2175488924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 msgr2=0x560af6758340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 0x560af6758340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c068260 con 0x560af67472d0 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e4179c700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 0x560af6758340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e56bd6b80 10 client.0 _ll_register_callbacks cb 0x560af657fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af669def0 con 0x560af67472d0 2023-10-18T09:30:58.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.499+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af6758880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e280ca370 tx=0x7f2e28099590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c4a11845461a589 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e1bfff700 10 client.26341 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:30:58.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e1bfff700 1 -- 172.21.15.61:0/2175488924 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28099b40 con 0x560af67472d0 2023-10-18T09:30:58.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e1bfff700 1 -- 172.21.15.61:0/2175488924 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2e2809dc90 con 0x560af67472d0 2023-10-18T09:30:58.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e1bfff700 1 -- 172.21.15.61:0/2175488924 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e28099b40 con 0x560af67472d0 2023-10-18T09:30:58.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e56bd6b80 10 client.26341 fetch_fsmap learned FSMap version 43 2023-10-18T09:30:58.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.500+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=43}) v3 -- 0x7f2e34007cd0 con 0x560af67472d0 2023-10-18T09:30:58.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e1bfff700 1 -- 172.21.15.61:0/2175488924 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2e280b2200 con 0x560af67472d0 2023-10-18T09:30:58.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:30:58.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:30:58.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 20 client.26341 populate_metadata read hostname 'smithi061' 2023-10-18T09:30:58.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af6763de0 0x560af6766220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 4 client.26341 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:30:58.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.501+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af67158b0 con 0x560af6763de0 2023-10-18T09:30:58.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.502+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af6763de0 0x560af6766220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:58 smithi061 ceph-mon[98573]: pgmap v730: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.8 MiB/s wr, 493 op/s 2023-10-18T09:30:58.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.505+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af6763de0 0x560af6766220 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26711ee2ffaac92 server_cookie=1ce360f33457002c in_seq=0 out_seq=0 2023-10-18T09:30:58.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.505+0000 7f2e1bfff700 10 client.26341 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:58.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.505+0000 7f2e1bfff700 1 -- 172.21.15.61:0/2175488924 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560af67158b0 con 0x560af6763de0 2023-10-18T09:30:58.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.505+0000 7f2e1bfff700 10 client.26341 handle_command_reply: tid=0 2023-10-18T09:30:58.509 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.505+0000 7f2e56bd6b80 1 client.26341 shutdown 2023-10-18T09:30:58.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 msgr2=0x560af6758880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af6758880 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2e280ca370 tx=0x7f2e28099590 comp rx=0 tx=0).stop 2023-10-18T09:30:58.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 shutdown_connections 2023-10-18T09:30:58.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af6763de0 0x560af6766220 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af6747c30 0x560af675f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67465d0 0x560af6758340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/2175488924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af67472d0 0x560af6758880 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 >> 172.21.15.61:0/2175488924 conn(0x560af66ba720 msgr2=0x560af6745080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e1bfff700 0 client.26341 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:58.511 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 shutdown_connections 2023-10-18T09:30:58.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.506+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/2175488924 wait complete. 2023-10-18T09:30:58.512 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.507+0000 7f2e56bd6b80 20 client.26341 trim_cache size 0 max 16384 2023-10-18T09:30:58.555 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 Processor -- start 2023-10-18T09:30:58.556 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 -- start start 2023-10-18T09:30:58.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6757850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 0x560af675f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 0x560af674dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6519920 con 0x560af67472d0 2023-10-18T09:30:58.557 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af6741470 con 0x560af67465d0 2023-10-18T09:30:58.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.553+0000 7f2e56bd6b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af67572f0 con 0x560af6747c30 2023-10-18T09:30:58.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6757850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e4179c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 0x560af674dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e4179c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 0x560af674dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46666/0 (socket says 172.21.15.61:46666) 2023-10-18T09:30:58.558 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e4179c700 1 -- 172.21.15.61:0/1658440734 learned_addr learned my addr 172.21.15.61:0/1658440734 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:58.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6757850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40346/0 (socket says 172.21.15.61:40346) 2023-10-18T09:30:58.559 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.554+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 0x560af675f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.555+0000 7f2e3bfff700 1 -- 172.21.15.61:0/1658440734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 msgr2=0x560af674dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.555+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 0x560af674dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.555+0000 7f2e3bfff700 1 -- 172.21.15.61:0/1658440734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 msgr2=0x560af6757850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.560 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.555+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6757850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.555+0000 7f2e3bfff700 1 -- 172.21.15.61:0/1658440734 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af64c3de0 con 0x560af67472d0 2023-10-18T09:30:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.556+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 0x560af675f830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b500 tx=0x7f2e28095fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352b8335a2b907cd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.556+0000 7f2e277fe700 1 -- 172.21.15.61:0/1658440734 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28099b40 con 0x560af67472d0 2023-10-18T09:30:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.556+0000 7f2e277fe700 1 -- 172.21.15.61:0/1658440734 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af669def0 con 0x560af67472d0 2023-10-18T09:30:58.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.557+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1658440734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 msgr2=0x560af675f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.557+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 0x560af675f830 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b500 tx=0x7f2e28095fb0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1658440734 shutdown_connections 2023-10-18T09:30:58.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af6747c30 0x560af674dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af6757850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/1658440734 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af67472d0 0x560af675f830 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1658440734 >> 172.21.15.61:0/1658440734 conn(0x560af66ba720 msgr2=0x560af6740b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.556+0000 7f2e277fe700 1 -- 172.21.15.61:0/1658440734 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28010580 con 0x560af67472d0 2023-10-18T09:30:58.563 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1658440734 shutdown_connections 2023-10-18T09:30:58.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.559+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/1658440734 wait complete. 2023-10-18T09:30:58.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.560+0000 7f2e56bd6b80 1 -- start start 2023-10-18T09:30:58.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.560+0000 7f2e56bd6b80 1 Processor -- start 2023-10-18T09:30:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.561+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.561+0000 7f2e56bd6b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af675be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.561+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.561+0000 7f2e40f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:40360/0 (socket says 172.21.15.61:40360) 2023-10-18T09:30:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.561+0000 7f2e40f9b700 1 -- 172.21.15.61:0/3434651371 learned_addr learned my addr 172.21.15.61:0/3434651371 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:30:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af675ee30 0x560af675f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af675f7f0 con 0x560af6747c30 2023-10-18T09:30:58.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af675f960 con 0x560af67465d0 2023-10-18T09:30:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af675fb20 con 0x560af675ee30 2023-10-18T09:30:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e56bd6b80 10 client.0 _ll_register_callbacks cb 0x560af657fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:30:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af675be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e40f9b700 1 -- 172.21.15.61:0/3434651371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af675ee30 msgr2=0x560af675f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af675ee30 0x560af675f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e40f9b700 1 -- 172.21.15.61:0/3434651371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 msgr2=0x560af675be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:30:58.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af675be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e40f9b700 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e34007cd0 con 0x560af67465d0 2023-10-18T09:30:58.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.562+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af675be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:30:58.568 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.563+0000 7f2e40f9b700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c1006e0 tx=0x7f2e30009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99f293eba1d3316 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:30:58.569 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.563+0000 7f2e1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:30:58.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.568+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3002c040 con 0x560af67465d0 2023-10-18T09:30:58.570 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.568+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2e300056c0 con 0x560af67465d0 2023-10-18T09:30:58.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.568+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af6742610 con 0x560af67465d0 2023-10-18T09:30:58.571 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.569+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e3001e070 con 0x560af67465d0 2023-10-18T09:30:58.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.569+0000 7f2e56bd6b80 10 client.26420 fetch_fsmap learned FSMap version 43 2023-10-18T09:30:58.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.569+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x560af64acc60 con 0x560af67465d0 2023-10-18T09:30:58.572 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.569+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af6742610 con 0x560af67465d0 2023-10-18T09:30:58.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.569+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2e30004dc0 con 0x560af67465d0 2023-10-18T09:30:58.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:30:58.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:30:58.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 20 client.26420 populate_metadata read hostname 'smithi061' 2023-10-18T09:30:58.573 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af67ca360 0x560af67cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:30:58.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 4 client.26420 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:30:58.574 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f2e34010040 con 0x560af67ca360 2023-10-18T09:30:58.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.570+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af67ca360 0x560af67cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:30:58.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e1bfff700 10 client.26420 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:58.575 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e3bfff700 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af67ca360 0x560af67cc780 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322d459cabb5eb81 server_cookie=26211ba0f6b609f3 in_seq=0 out_seq=0 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e1bfff700 1 -- 172.21.15.61:0/3434651371 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x7f2e34010040 con 0x560af67ca360 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e1bfff700 10 client.26420 handle_command_reply: tid=0 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e56bd6b80 1 client.26420 shutdown 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 msgr2=0x560af675b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.571+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c1006e0 tx=0x7f2e30009ca0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.576 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 shutdown_connections 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x560af67ca360 0x560af67cc780 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560af675ee30 0x560af675f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af67465d0 0x560af675b920 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c1006e0 tx=0x7f2e30009ca0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 --2- 172.21.15.61:0/3434651371 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x560af6747c30 0x560af675be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.572+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 >> 172.21.15.61:0/3434651371 conn(0x560af66ba720 msgr2=0x560af67582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.577 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.573+0000 7f2e1bfff700 0 client.26420 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:30:58.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.573+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 shutdown_connections 2023-10-18T09:30:58.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.573+0000 7f2e56bd6b80 1 -- 172.21.15.61:0/3434651371 wait complete. 2023-10-18T09:30:58.578 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.573+0000 7f2e56bd6b80 20 client.26420 trim_cache size 0 max 16384 2023-10-18T09:30:58.579 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:30:58.579 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:30:58.579 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:30:58.580 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:30:58.581 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:30:58.581 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 msgr2=0x7f2e2c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 0x7f2e2c0a39c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e30005e80 comp rx=0 tx=0).stop 2023-10-18T09:30:58.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 msgr2=0x7f2e3c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:30:58.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800b530 tx=0x7f2e2800b8f0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 shutdown_connections 2023-10-18T09:30:58.582 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f2e2c0a1500 0x7f2e2c0a39c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f2e3c101220 0x7f2e3c19dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e3c0ff690 0x7f2e3c1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 --2- 172.21.15.61:0/3734398054 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2e3c100880 0x7f2e3c196d00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:30:58.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 >> 172.21.15.61:0/3734398054 conn(0x7f2e3c0fac00 msgr2=0x7f2e3c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:30:58.583 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 shutdown_connections 2023-10-18T09:30:58.584 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:30:58.574+0000 7f2e431ff700 1 -- 172.21.15.61:0/3734398054 wait complete. 2023-10-18T09:30:58.610 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:30:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 705.0 s, 446.2 tps, lat 221.549 ms stddev 297.349 2023-10-18T09:30:59.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:30:59 smithi170 ceph-mon[135150]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2023-10-18T09:30:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:30:59 smithi110 ceph-mon[134818]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2023-10-18T09:30:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:30:59 smithi061 ceph-mon[98573]: pgmap v731: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2023-10-18T09:31:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:01 smithi170 ceph-mon[135150]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-18T09:31:01.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:01 smithi110 ceph-mon[134818]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-18T09:31:01.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:01 smithi061 ceph-mon[98573]: pgmap v732: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.7 MiB/s wr, 512 op/s 2023-10-18T09:31:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:03 smithi170 ceph-mon[135150]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2023-10-18T09:31:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:03 smithi110 ceph-mon[134818]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2023-10-18T09:31:03.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:03 smithi061 ceph-mon[98573]: pgmap v733: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.9 MiB/s wr, 490 op/s 2023-10-18T09:31:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 710.0 s, 482.6 tps, lat 209.675 ms stddev 226.235 2023-10-18T09:31:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:05 smithi170 ceph-mon[135150]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:05 smithi110 ceph-mon[134818]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:05.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:05 smithi061 ceph-mon[98573]: pgmap v734: 129 pgs: 129 active+clean; 1.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:08.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:08 smithi170 ceph-mon[135150]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 498 op/s 2023-10-18T09:31:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:08 smithi110 ceph-mon[134818]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 498 op/s 2023-10-18T09:31:08.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:08 smithi061 ceph-mon[98573]: pgmap v735: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 498 op/s 2023-10-18T09:31:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 715.0 s, 481.6 tps, lat 197.626 ms stddev 248.640 2023-10-18T09:31:09.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:09 smithi170 ceph-mon[135150]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:09.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:09 smithi110 ceph-mon[134818]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:09 smithi061 ceph-mon[98573]: pgmap v736: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 4.4 MiB/s wr, 475 op/s 2023-10-18T09:31:11.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:11 smithi170 ceph-mon[135150]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2023-10-18T09:31:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:11 smithi110 ceph-mon[134818]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2023-10-18T09:31:11.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:11 smithi061 ceph-mon[98573]: pgmap v737: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2023-10-18T09:31:13.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:13 smithi170 ceph-mon[135150]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2023-10-18T09:31:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:13 smithi110 ceph-mon[134818]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2023-10-18T09:31:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:13 smithi061 ceph-mon[98573]: pgmap v738: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.7 MiB/s wr, 501 op/s 2023-10-18T09:31:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 720.0 s, 592.2 tps, lat 172.434 ms stddev 223.339 2023-10-18T09:31:15.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:15 smithi170 ceph-mon[135150]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-10-18T09:31:15.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:15 smithi110 ceph-mon[134818]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-10-18T09:31:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:15 smithi061 ceph-mon[98573]: pgmap v739: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 5.1 MiB/s wr, 501 op/s 2023-10-18T09:31:18.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:18 smithi170 ceph-mon[135150]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-18T09:31:18.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:18 smithi110 ceph-mon[134818]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-18T09:31:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:18 smithi061 ceph-mon[98573]: pgmap v740: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.9 MiB/s wr, 549 op/s 2023-10-18T09:31:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 725.0 s, 439.8 tps, lat 224.327 ms stddev 298.405 2023-10-18T09:31:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:19 smithi170 ceph-mon[135150]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.8 MiB/s wr, 534 op/s 2023-10-18T09:31:19.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:19 smithi110 ceph-mon[134818]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.8 MiB/s wr, 534 op/s 2023-10-18T09:31:19.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:19 smithi061 ceph-mon[98573]: pgmap v741: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 5.8 MiB/s wr, 534 op/s 2023-10-18T09:31:21.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:21 smithi170 ceph-mon[135150]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2023-10-18T09:31:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:21 smithi110 ceph-mon[134818]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2023-10-18T09:31:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:21 smithi061 ceph-mon[98573]: pgmap v742: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.6 MiB/s wr, 599 op/s 2023-10-18T09:31:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:23 smithi170 ceph-mon[135150]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.4 MiB/s wr, 547 op/s 2023-10-18T09:31:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:23 smithi110 ceph-mon[134818]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.4 MiB/s wr, 547 op/s 2023-10-18T09:31:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:23 smithi061 ceph-mon[98573]: pgmap v743: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.4 MiB/s wr, 547 op/s 2023-10-18T09:31:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 730.0 s, 485.8 tps, lat 208.945 ms stddev 264.160 2023-10-18T09:31:25.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:25 smithi170 ceph-mon[135150]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2023-10-18T09:31:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:25 smithi110 ceph-mon[134818]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2023-10-18T09:31:25.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:25 smithi061 ceph-mon[98573]: pgmap v744: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 7.8 MiB/s wr, 546 op/s 2023-10-18T09:31:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:28 smithi170 ceph-mon[135150]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-10-18T09:31:28.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:28 smithi110 ceph-mon[134818]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-10-18T09:31:28.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:28 smithi061 ceph-mon[98573]: pgmap v745: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 7.7 MiB/s wr, 559 op/s 2023-10-18T09:31:28.611 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:31:28.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.937+0000 7fbf067ff700 1 -- 172.21.15.61:0/3824755823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf00100880 msgr2=0x7fbf00100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:28.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.937+0000 7fbf067ff700 1 --2- 172.21.15.61:0/3824755823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf00100880 0x7fbf00100ce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbef40099c0 tx=0x7fbef4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 -- 172.21.15.61:0/3824755823 shutdown_connections 2023-10-18T09:31:28.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 --2- 172.21.15.61:0/3824755823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00101220 0x7fbf00107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 --2- 172.21.15.61:0/3824755823 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf000ff690 0x7fbf000ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 --2- 172.21.15.61:0/3824755823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf00100880 0x7fbf00100ce0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 -- 172.21.15.61:0/3824755823 >> 172.21.15.61:0/3824755823 conn(0x7fbf000fac00 msgr2=0x7fbf000fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:28.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 -- 172.21.15.61:0/3824755823 shutdown_connections 2023-10-18T09:31:28.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.938+0000 7fbf067ff700 1 -- 172.21.15.61:0/3824755823 wait complete. 2023-10-18T09:31:28.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.940+0000 7fbf067ff700 1 Processor -- start 2023-10-18T09:31:28.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.940+0000 7fbf067ff700 1 -- start start 2023-10-18T09:31:28.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 0x7fbf0010f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 0x7fbf0010fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00110270 con 0x7fbf00101220 2023-10-18T09:31:28.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001ad140 con 0x7fbf000ff690 2023-10-18T09:31:28.944 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf067ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf001ad280 con 0x7fbf00100880 2023-10-18T09:31:28.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.941+0000 7fbf04d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 0x7fbf0010fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:47478/0 (socket says 172.21.15.61:47478) 2023-10-18T09:31:28.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 -- 172.21.15.61:0/1478388980 learned_addr learned my addr 172.21.15.61:0/1478388980 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:28.946 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbefffff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 0x7fbf0010f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 -- 172.21.15.61:0/1478388980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 msgr2=0x7fbf0010f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:28.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 0x7fbf0010f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 -- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 msgr2=0x7fbf0010fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:28.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 0x7fbf0010fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.947 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 -- 172.21.15.61:0/1478388980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40095f0 con 0x7fbf00100880 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 0x7fbf0010fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbefffff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 0x7fbf0010f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.942+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef4003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.943+0000 7fbefd7fa700 1 -- 172.21.15.61:0/1478388980 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef401d070 con 0x7fbf00100880 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.943+0000 7fbefd7fa700 1 -- 172.21.15.61:0/1478388980 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef4004440 con 0x7fbf00100880 2023-10-18T09:31:28.948 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.943+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf001ad480 con 0x7fbf00100880 2023-10-18T09:31:28.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.943+0000 7fbefd7fa700 1 -- 172.21.15.61:0/1478388980 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4004de0 con 0x7fbf00100880 2023-10-18T09:31:28.949 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.943+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf001ad900 con 0x7fbf00100880 2023-10-18T09:31:28.951 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.948+0000 7fbefd7fa700 1 -- 172.21.15.61:0/1478388980 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbef400f460 con 0x7fbf00100880 2023-10-18T09:31:28.953 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.949+0000 7fbf1a0e0b80 1 Processor -- start 2023-10-18T09:31:28.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.951+0000 7fbefd7fa700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 0x7fbee80a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.954 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.952+0000 7fbf1a0e0b80 1 -- start start 2023-10-18T09:31:28.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.953+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 0x7fbee80a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.954+0000 7fbefd7fa700 1 -- 172.21.15.61:0/1478388980 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbef40cdf00 con 0x7fbf00100880 2023-10-18T09:31:28.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.954+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x5574444086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.954+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444407090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.954+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.955+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744437a6e0 con 0x5574444075d0 2023-10-18T09:31:28.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.955+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444360c60 con 0x5574444082d0 2023-10-18T09:31:28.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.955+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55744435e580 con 0x557444408c30 2023-10-18T09:31:28.959 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.955+0000 7fbf04d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55090/0 (socket says 172.21.15.61:55090) 2023-10-18T09:31:28.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 -- 172.21.15.61:0/1985019457 learned_addr learned my addr 172.21.15.61:0/1985019457 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:28.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 -- 172.21.15.61:0/1985019457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 msgr2=0x557444407090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:28.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444407090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 -- 172.21.15.61:0/1985019457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 msgr2=0x5574444086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:28.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x5574444086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.956+0000 7fbf04d9c700 1 -- 172.21.15.61:0/1985019457 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574443173f0 con 0x5574444075d0 2023-10-18T09:31:28.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.960+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007a60 tx=0x7fbef8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ea3c58309a3a7c server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:28.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.961+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 0x7fbee80a39f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbef80175d0 tx=0x7fbef8011000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:28.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.962+0000 7fbee67fc700 1 -- 172.21.15.61:0/1985019457 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800e030 con 0x5574444075d0 2023-10-18T09:31:28.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.962+0000 7fbee67fc700 1 -- 172.21.15.61:0/1985019457 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef800f040 con 0x5574444075d0 2023-10-18T09:31:28.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1985019457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 msgr2=0x557444407a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:28.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbee67fc700 1 -- 172.21.15.61:0/1985019457 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef800e030 con 0x5574444075d0 2023-10-18T09:31:28.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbef8007a60 tx=0x7fbef8007d70 comp rx=0 tx=0).stop 2023-10-18T09:31:28.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1985019457 shutdown_connections 2023-10-18T09:31:28.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444407a50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x5574444086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1985019457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444407090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.963+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1985019457 >> 172.21.15.61:0/1985019457 conn(0x55744437b720 msgr2=0x557444401b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:28.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.964+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1985019457 shutdown_connections 2023-10-18T09:31:28.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.964+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1985019457 wait complete. 2023-10-18T09:31:28.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.965+0000 7fbf1a0e0b80 1 -- start start 2023-10-18T09:31:28.968 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.965+0000 7fbf1a0e0b80 1 Processor -- start 2023-10-18T09:31:28.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.969+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.969+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x557444410f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.969+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444359e70 con 0x5574444075d0 2023-10-18T09:31:28.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574442f6710 con 0x5574444082d0 2023-10-18T09:31:28.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574442f63b0 con 0x557444408c30 2023-10-18T09:31:28.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbf1a0e0b80 10 client.0 _ll_register_callbacks cb 0x557444240a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:31:28.974 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbefffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbefffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55096/0 (socket says 172.21.15.61:55096) 2023-10-18T09:31:28.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.971+0000 7fbefffff700 1 -- 172.21.15.61:0/1174728791 learned_addr learned my addr 172.21.15.61:0/1174728791 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:28.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.972+0000 7fbefffff700 1 -- 172.21.15.61:0/1174728791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 msgr2=0x557444417f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:28.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.972+0000 7fbefffff700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.973+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x557444410f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.973+0000 7fbefffff700 1 -- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 msgr2=0x557444410f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:28.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.973+0000 7fbefffff700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x557444410f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.974+0000 7fbefffff700 1 -- 172.21.15.61:0/1174728791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef8007690 con 0x5574444075d0 2023-10-18T09:31:28.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.974+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x557444410f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:31:28.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.975+0000 7fbefffff700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbef000beb0 tx=0x7fbef000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fcd1701c429891 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:28.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.975+0000 7fbedf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:31:28.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.977+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0004a30 con 0x5574444075d0 2023-10-18T09:31:28.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.977+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbef0004b90 con 0x5574444075d0 2023-10-18T09:31:28.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.977+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55744435eef0 con 0x5574444075d0 2023-10-18T09:31:28.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.978+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef0004d50 con 0x5574444075d0 2023-10-18T09:31:28.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.978+0000 7fbf1a0e0b80 10 client.16215 fetch_fsmap learned FSMap version 43 2023-10-18T09:31:28.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.978+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x557444184de0 con 0x5574444075d0 2023-10-18T09:31:28.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.981+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55744435eef0 con 0x5574444075d0 2023-10-18T09:31:28.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbef00117b0 con 0x5574444075d0 2023-10-18T09:31:28.984 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:31:28.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:31:28.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 20 client.16215 populate_metadata read hostname 'smithi061' 2023-10-18T09:31:28.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55744441c930 0x55744441ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:28.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 4 client.16215 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:31:28.985 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.982+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574443d68b0 con 0x55744441c930 2023-10-18T09:31:28.988 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.985+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55744441c930 0x55744441ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:28.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.987+0000 7fbedf7fe700 10 client.16215 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:28.990 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.987+0000 7fbeff7fe700 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55744441c930 0x55744441ed50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1036e75eca1be5 server_cookie=ae3ecfd90286dfa6 in_seq=0 out_seq=0 2023-10-18T09:31:28.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.990+0000 7fbedf7fe700 1 -- 172.21.15.61:0/1174728791 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5574443d68b0 con 0x55744441c930 2023-10-18T09:31:28.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.990+0000 7fbedf7fe700 10 client.16215 handle_command_reply: tid=0 2023-10-18T09:31:28.993 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.990+0000 7fbf1a0e0b80 1 client.16215 shutdown 2023-10-18T09:31:28.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 msgr2=0x557444410a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:28.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbef000beb0 tx=0x7fbef000bf90 comp rx=0 tx=0).stop 2023-10-18T09:31:28.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 shutdown_connections 2023-10-18T09:31:28.994 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55744441c930 0x55744441ed50 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444075d0 0x557444410a20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/1174728791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444082d0 0x557444410f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:28.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 >> 172.21.15.61:0/1174728791 conn(0x55744437b720 msgr2=0x557444406080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:28.995 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.991+0000 7fbedf7fe700 0 client.16215 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:28.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.993+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 shutdown_connections 2023-10-18T09:31:28.996 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.993+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/1174728791 wait complete. 2023-10-18T09:31:28.998 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:28.995+0000 7fbf1a0e0b80 20 client.16215 trim_cache size 0 max 16384 2023-10-18T09:31:29.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.062+0000 7fbf1a0e0b80 1 Processor -- start 2023-10-18T09:31:29.065 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 -- start start 2023-10-18T09:31:29.066 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444418600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.066 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.066 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444079b0 0x55744440ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.066 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574441da920 con 0x5574444082d0 2023-10-18T09:31:29.067 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444419ce0 con 0x5574444079b0 2023-10-18T09:31:29.067 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.063+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444402ed0 con 0x557444408c30 2023-10-18T09:31:29.067 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.064+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.068 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.064+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:47508/0 (socket says 172.21.15.61:47508) 2023-10-18T09:31:29.068 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.064+0000 7fbeff7fe700 1 -- 172.21.15.61:0/2799627857 learned_addr learned my addr 172.21.15.61:0/2799627857 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:29.068 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.064+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444079b0 0x55744440ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.068 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.065+0000 7fbefffff700 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444418600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.069 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.067+0000 7fbefffff700 1 -- 172.21.15.61:0/2799627857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 msgr2=0x557444417f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:29.070 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.067+0000 7fbefffff700 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.070 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.067+0000 7fbefffff700 1 -- 172.21.15.61:0/2799627857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444079b0 msgr2=0x55744440ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:29.070 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.067+0000 7fbefffff700 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444079b0 0x55744440ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.070 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.067+0000 7fbefffff700 1 -- 172.21.15.61:0/2799627857 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55744416dc60 con 0x5574444082d0 2023-10-18T09:31:29.072 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.070+0000 7fbefffff700 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444418600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5574444026d0 tx=0x7fbef000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bb98a3fb58c4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:29.073 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.071+0000 7fbee6ffd700 1 -- 172.21.15.61:0/2799627857 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011640 con 0x5574444082d0 2023-10-18T09:31:29.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.074+0000 7fbee6ffd700 1 -- 172.21.15.61:0/2799627857 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef000f9d0 con 0x5574444082d0 2023-10-18T09:31:29.076 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.074+0000 7fbee6ffd700 1 -- 172.21.15.61:0/2799627857 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011640 con 0x5574444082d0 2023-10-18T09:31:29.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/2799627857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 msgr2=0x557444418600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:29.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444418600 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5574444026d0 tx=0x7fbef000ff50 comp rx=0 tx=0).stop 2023-10-18T09:31:29.077 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/2799627857 shutdown_connections 2023-10-18T09:31:29.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574444079b0 0x55744440ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444418600 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/2799627857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557444408c30 0x557444417f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.078 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/2799627857 >> 172.21.15.61:0/2799627857 conn(0x55744437b720 msgr2=0x557444406080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:29.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/2799627857 shutdown_connections 2023-10-18T09:31:29.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.075+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/2799627857 wait complete. 2023-10-18T09:31:29.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.076+0000 7fbf1a0e0b80 1 -- start start 2023-10-18T09:31:29.079 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.076+0000 7fbf1a0e0b80 1 Processor -- start 2023-10-18T09:31:29.080 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574444079b0 0x557444417790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444417cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574444163d0 con 0x5574444082d0 2023-10-18T09:31:29.081 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444416540 con 0x557444408c30 2023-10-18T09:31:29.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.078+0000 7fbf1a0e0b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x557444416700 con 0x5574444079b0 2023-10-18T09:31:29.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.079+0000 7fbf1a0e0b80 10 client.0 _ll_register_callbacks cb 0x557444240a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:31:29.082 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.079+0000 7fbeff7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444417cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.079+0000 7fbf04d9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60852/0 (socket says 172.21.15.61:60852) 2023-10-18T09:31:29.083 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 -- 172.21.15.61:0/3504190258 learned_addr learned my addr 172.21.15.61:0/3504190258 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:29.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 -- 172.21.15.61:0/3504190258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574444079b0 msgr2=0x557444417790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:29.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574444079b0 0x557444417790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 -- 172.21.15.61:0/3504190258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 msgr2=0x557444417cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:29.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444417cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.084 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.080+0000 7fbf04d9c700 1 -- 172.21.15.61:0/3504190258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557444184de0 con 0x557444408c30 2023-10-18T09:31:29.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.081+0000 7fbf04d9c700 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbef8006030 tx=0x7fbef8019d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5b5a32bd92438f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:29.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.081+0000 7fbedf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:31:29.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.083+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef80095e0 con 0x557444408c30 2023-10-18T09:31:29.085 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.083+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbef802a070 con 0x557444408c30 2023-10-18T09:31:29.086 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.083+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbef800f040 con 0x557444408c30 2023-10-18T09:31:29.088 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.086+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbef8004ea0 con 0x557444408c30 2023-10-18T09:31:29.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.086+0000 7fbf1a0e0b80 10 client.26432 fetch_fsmap learned FSMap version 43 2023-10-18T09:31:29.089 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.086+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5574441ab250 con 0x557444408c30 2023-10-18T09:31:29.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.092+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef800f040 con 0x557444408c30 2023-10-18T09:31:29.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fbef802c070 con 0x557444408c30 2023-10-18T09:31:29.095 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 10 client.26432 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:31:29.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 10 client.26432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:31:29.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 20 client.26432 populate_metadata read hostname 'smithi061' 2023-10-18T09:31:29.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557444471a60 0x557444473e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:29.096 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 4 client.26432 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:31:29.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.093+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557444352e50 con 0x557444471a60 2023-10-18T09:31:29.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.095+0000 7fbefffff700 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557444471a60 0x557444473e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:29.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.107+0000 7fbefffff700 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557444471a60 0x557444473e80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976aa996d99fb182 server_cookie=e18ad34dbc457663 in_seq=0 out_seq=0 2023-10-18T09:31:29.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.107+0000 7fbedf7fe700 10 client.26432 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:29.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.107+0000 7fbedf7fe700 1 -- 172.21.15.61:0/3504190258 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557444352e50 con 0x557444471a60 2023-10-18T09:31:29.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.107+0000 7fbedf7fe700 10 client.26432 handle_command_reply: tid=0 2023-10-18T09:31:29.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.107+0000 7fbf1a0e0b80 1 client.26432 shutdown 2023-10-18T09:31:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.110+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 msgr2=0x557444415df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.110+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbef8006030 tx=0x7fbef8019d00 comp rx=0 tx=0).stop 2023-10-18T09:31:29.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 shutdown_connections 2023-10-18T09:31:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x557444471a60 0x557444473e80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557444408c30 0x557444415df0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574444079b0 0x557444417790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 --2- 172.21.15.61:0/3504190258 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5574444082d0 0x557444417cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 >> 172.21.15.61:0/3504190258 conn(0x55744437b720 msgr2=0x55744440b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbedf7fe700 0 client.26432 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 shutdown_connections 2023-10-18T09:31:29.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 1 -- 172.21.15.61:0/3504190258 wait complete. 2023-10-18T09:31:29.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.111+0000 7fbf1a0e0b80 20 client.26432 trim_cache size 0 max 16384 2023-10-18T09:31:29.116 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:31:29.116 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:31:29.116 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:31:29.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:31:29.118 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:31:29.118 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.112+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 msgr2=0x7fbee80a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.112+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 0x7fbee80a39f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbef80175d0 tx=0x7fbef8011000 comp rx=0 tx=0).stop 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.112+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 msgr2=0x7fbf00114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.112+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbef4006010 tx=0x7fbef4003f90 comp rx=0 tx=0).stop 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 shutdown_connections 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbee80a1530 0x7fbee80a39f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.119 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf00101220 0x7fbf0010fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf000ff690 0x7fbf0010f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 --2- 172.21.15.61:0/1478388980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf00100880 0x7fbf00114c10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 >> 172.21.15.61:0/1478388980 conn(0x7fbf000fac00 msgr2=0x7fbf00101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 shutdown_connections 2023-10-18T09:31:29.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:29.113+0000 7fbf067ff700 1 -- 172.21.15.61:0/1478388980 wait complete. 2023-10-18T09:31:29.158 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:31:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 735.0 s, 547.0 tps, lat 176.344 ms stddev 233.459 2023-10-18T09:31:29.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:31:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:29.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:29 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:31:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:29.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:29 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:31:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:29 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:30.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:30 smithi170 ceph-mon[135150]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.1 MiB/s wr, 530 op/s 2023-10-18T09:31:30.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:30 smithi110 ceph-mon[134818]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.1 MiB/s wr, 530 op/s 2023-10-18T09:31:30.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:30 smithi061 ceph-mon[98573]: pgmap v746: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 8.1 MiB/s wr, 530 op/s 2023-10-18T09:31:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:31:31.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:31:31.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:31:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:31:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:31:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:31:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:31:32.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:32 smithi170 ceph-mon[135150]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-10-18T09:31:32.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:32 smithi110 ceph-mon[134818]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-10-18T09:31:32.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:32 smithi061 ceph-mon[98573]: pgmap v747: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 8.5 MiB/s wr, 532 op/s 2023-10-18T09:31:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:33 smithi170 ceph-mon[135150]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 498 op/s 2023-10-18T09:31:33.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:33 smithi110 ceph-mon[134818]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 498 op/s 2023-10-18T09:31:33.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:33 smithi061 ceph-mon[98573]: pgmap v748: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.3 MiB/s wr, 498 op/s 2023-10-18T09:31:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 740.0 s, 523.6 tps, lat 199.457 ms stddev 280.021 2023-10-18T09:31:35.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:35 smithi170 ceph-mon[135150]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.3 MiB/s wr, 479 op/s 2023-10-18T09:31:35.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:35 smithi110 ceph-mon[134818]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.3 MiB/s wr, 479 op/s 2023-10-18T09:31:35.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:35 smithi061 ceph-mon[98573]: pgmap v749: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.3 MiB/s wr, 479 op/s 2023-10-18T09:31:38.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:38 smithi170 ceph-mon[135150]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-10-18T09:31:38.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:38 smithi110 ceph-mon[134818]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-10-18T09:31:38.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:38 smithi061 ceph-mon[98573]: pgmap v750: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 7.5 MiB/s wr, 495 op/s 2023-10-18T09:31:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 745.0 s, 544.6 tps, lat 184.021 ms stddev 205.426 2023-10-18T09:31:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:39 smithi170 ceph-mon[135150]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.5 MiB/s wr, 470 op/s 2023-10-18T09:31:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:39 smithi110 ceph-mon[134818]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.5 MiB/s wr, 470 op/s 2023-10-18T09:31:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:39 smithi061 ceph-mon[98573]: pgmap v751: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 7.5 MiB/s wr, 470 op/s 2023-10-18T09:31:41.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:41 smithi170 ceph-mon[135150]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-18T09:31:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:41 smithi110 ceph-mon[134818]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-18T09:31:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:41 smithi061 ceph-mon[98573]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-18T09:31:43.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:43 smithi170 ceph-mon[135150]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2023-10-18T09:31:43.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:43 smithi110 ceph-mon[134818]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2023-10-18T09:31:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:43 smithi061 ceph-mon[98573]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 7.2 MiB/s wr, 499 op/s 2023-10-18T09:31:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 750.0 s, 468.6 tps, lat 214.287 ms stddev 272.183 2023-10-18T09:31:45.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:45 smithi170 ceph-mon[135150]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2023-10-18T09:31:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:45 smithi110 ceph-mon[134818]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2023-10-18T09:31:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:45 smithi061 ceph-mon[98573]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.8 MiB/s wr, 459 op/s 2023-10-18T09:31:48.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:48 smithi170 ceph-mon[135150]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 469 op/s 2023-10-18T09:31:48.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:48 smithi110 ceph-mon[134818]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 469 op/s 2023-10-18T09:31:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:48 smithi061 ceph-mon[98573]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.8 MiB/s wr, 469 op/s 2023-10-18T09:31:49.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 755.0 s, 484.8 tps, lat 201.013 ms stddev 222.566 2023-10-18T09:31:49.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:49 smithi170 ceph-mon[135150]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-10-18T09:31:49.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:49 smithi110 ceph-mon[134818]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-10-18T09:31:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:49 smithi061 ceph-mon[98573]: pgmap v756: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.8 MiB/s wr, 467 op/s 2023-10-18T09:31:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:51 smithi170 ceph-mon[135150]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 472 op/s 2023-10-18T09:31:51.721 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:51 smithi110 ceph-mon[134818]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 472 op/s 2023-10-18T09:31:51.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:51 smithi061 ceph-mon[98573]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.8 MiB/s wr, 472 op/s 2023-10-18T09:31:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:53 smithi170 ceph-mon[135150]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2023-10-18T09:31:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:53 smithi110 ceph-mon[134818]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2023-10-18T09:31:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:53 smithi061 ceph-mon[98573]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 6.5 MiB/s wr, 484 op/s 2023-10-18T09:31:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 760.0 s, 497.6 tps, lat 208.961 ms stddev 302.536 2023-10-18T09:31:55.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:55 smithi170 ceph-mon[135150]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2023-10-18T09:31:55.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:55 smithi110 ceph-mon[134818]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2023-10-18T09:31:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:55 smithi061 ceph-mon[98573]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.3 MiB/s wr, 451 op/s 2023-10-18T09:31:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:58 smithi170 ceph-mon[135150]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 470 op/s 2023-10-18T09:31:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:58 smithi110 ceph-mon[134818]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 470 op/s 2023-10-18T09:31:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:58 smithi061 ceph-mon[98573]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.7 MiB/s wr, 470 op/s 2023-10-18T09:31:59.159 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:31:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 765.0 s, 456.2 tps, lat 213.603 ms stddev 237.112 2023-10-18T09:31:59.622 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.620+0000 7ff242dff700 1 -- 172.21.15.61:0/3762543553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c1029d0 msgr2=0x7ff23c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.623 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.620+0000 7ff242dff700 1 --2- 172.21.15.61:0/3762543553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c1029d0 0x7ff23c102df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff230009990 tx=0x7ff230009ca0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 -- 172.21.15.61:0/3762543553 shutdown_connections 2023-10-18T09:31:59.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 --2- 172.21.15.61:0/3762543553 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c104560 0x7ff23c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 --2- 172.21.15.61:0/3762543553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c1029d0 0x7ff23c102df0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 --2- 172.21.15.61:0/3762543553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c103bc0 0x7ff23c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.624 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 -- 172.21.15.61:0/3762543553 >> 172.21.15.61:0/3762543553 conn(0x7ff23c0fdf40 msgr2=0x7ff23c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 -- 172.21.15.61:0/3762543553 shutdown_connections 2023-10-18T09:31:59.625 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.621+0000 7ff242dff700 1 -- 172.21.15.61:0/3762543553 wait complete. 2023-10-18T09:31:59.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.622+0000 7ff242dff700 1 Processor -- start 2023-10-18T09:31:59.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.623+0000 7ff242dff700 1 -- start start 2023-10-18T09:31:59.626 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.623+0000 7ff242dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.623+0000 7ff242dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c103bc0 0x7ff23c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.627 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.623+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:49768/0 (socket says 172.21.15.61:49768) 2023-10-18T09:31:59.628 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff242dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c104560 0x7ff23c19ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff240b9b700 1 -- 172.21.15.61:0/2943328451 learned_addr learned my addr 172.21.15.61:0/2943328451 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:59.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1a04e0 con 0x7ff23c103bc0 2023-10-18T09:31:59.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1a0620 con 0x7ff23c1029d0 2023-10-18T09:31:59.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c1a07b0 con 0x7ff23c104560 2023-10-18T09:31:59.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff24139c700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c104560 0x7ff23c19ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.624+0000 7ff23bfff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c103bc0 0x7ff23c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 -- 172.21.15.61:0/2943328451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c104560 msgr2=0x7ff23c19ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c104560 0x7ff23c19ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 -- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c103bc0 msgr2=0x7ff23c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.630 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c103bc0 0x7ff23c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300095f0 con 0x7ff23c1029d0 2023-10-18T09:31:59.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff240b9b700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b680 tx=0x7ff230004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.625+0000 7ff239ffb700 1 -- 172.21.15.61:0/2943328451 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23001d070 con 0x7ff23c1029d0 2023-10-18T09:31:59.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.626+0000 7ff239ffb700 1 -- 172.21.15.61:0/2943328451 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff230021410 con 0x7ff23c1029d0 2023-10-18T09:31:59.631 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.626+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c1a0a30 con 0x7ff23c1029d0 2023-10-18T09:31:59.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.626+0000 7ff239ffb700 1 -- 172.21.15.61:0/2943328451 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230017650 con 0x7ff23c1029d0 2023-10-18T09:31:59.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.626+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c1a0eb0 con 0x7ff23c1029d0 2023-10-18T09:31:59.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.630+0000 7ff239ffb700 1 -- 172.21.15.61:0/2943328451 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7ff230021580 con 0x7ff23c1029d0 2023-10-18T09:31:59.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.632+0000 7ff239ffb700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 0x7ff22c0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.632+0000 7ff239ffb700 1 -- 172.21.15.61:0/2943328451 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff2300ce420 con 0x7ff23c1029d0 2023-10-18T09:31:59.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.633+0000 7ff23bfff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 0x7ff22c0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.635+0000 7ff23bfff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 0x7ff22c0a3a40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff228009c60 tx=0x7ff228009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.636+0000 7ff256815b80 1 Processor -- start 2023-10-18T09:31:59.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.636+0000 7ff256815b80 1 -- start start 2023-10-18T09:31:59.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.636+0000 7ff256815b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f3c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.636+0000 7ff256815b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.636+0000 7ff256815b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f3ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff256815b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07eae6e0 con 0x558c07f3b5d0 2023-10-18T09:31:59.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff256815b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e94c60 con 0x558c07f3c2d0 2023-10-18T09:31:59.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff256815b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e92580 con 0x558c07f3cc30 2023-10-18T09:31:59.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff23bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff23bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:60222/0 (socket says 172.21.15.61:60222) 2023-10-18T09:31:59.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff23bfff700 1 -- 172.21.15.61:0/149111173 learned_addr learned my addr 172.21.15.61:0/149111173 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:59.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff240b9b700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f3c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff24139c700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f3ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.642 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff23bfff700 1 -- 172.21.15.61:0/149111173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 msgr2=0x558c07f3c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.637+0000 7ff23bfff700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f3c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff23bfff700 1 -- 172.21.15.61:0/149111173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 msgr2=0x558c07f3ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff23bfff700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f3ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff23bfff700 1 -- 172.21.15.61:0/149111173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c07e4b3f0 con 0x558c07f3cc30 2023-10-18T09:31:59.643 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff24139c700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:31:59.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff23bfff700 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e4e0 tx=0x7ff228014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68370a894a4ed9be server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff226ffd700 1 -- 172.21.15.61:0/149111173 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228026430 con 0x558c07f3cc30 2023-10-18T09:31:59.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff226ffd700 1 -- 172.21.15.61:0/149111173 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff228021070 con 0x558c07f3cc30 2023-10-18T09:31:59.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.638+0000 7ff226ffd700 1 -- 172.21.15.61:0/149111173 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22801d740 con 0x558c07f3cc30 2023-10-18T09:31:59.644 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.639+0000 7ff256815b80 1 -- 172.21.15.61:0/149111173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 msgr2=0x558c07f3b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.645 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.639+0000 7ff256815b80 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff22800e4e0 tx=0x7ff228014000 comp rx=0 tx=0).stop 2023-10-18T09:31:59.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:31:59 smithi170 ceph-mon[135150]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 483 op/s 2023-10-18T09:31:59.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.639+0000 7ff256815b80 1 -- 172.21.15.61:0/149111173 shutdown_connections 2023-10-18T09:31:59.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.639+0000 7ff256815b80 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f3ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f3c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 --2- 172.21.15.61:0/149111173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3b090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 -- 172.21.15.61:0/149111173 >> 172.21.15.61:0/149111173 conn(0x558c07eaf720 msgr2=0x558c07f35b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 -- 172.21.15.61:0/149111173 shutdown_connections 2023-10-18T09:31:59.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 -- 172.21.15.61:0/149111173 wait complete. 2023-10-18T09:31:59.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.640+0000 7ff256815b80 1 -- start start 2023-10-18T09:31:59.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 Processor -- start 2023-10-18T09:31:59.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e8de70 con 0x558c07f3c2d0 2023-10-18T09:31:59.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e2a710 con 0x558c07f3b5d0 2023-10-18T09:31:59.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e2a3b0 con 0x558c07f3cc30 2023-10-18T09:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff256815b80 10 client.0 _ll_register_callbacks cb 0x558c07d74a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:49780/0 (socket says 172.21.15.61:49780) 2023-10-18T09:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff240b9b700 1 -- 172.21.15.61:0/347761397 learned_addr learned my addr 172.21.15.61:0/347761397 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.641+0000 7ff23bfff700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff24139c700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 -- 172.21.15.61:0/347761397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 msgr2=0x558c07f4c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 -- 172.21.15.61:0/347761397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 msgr2=0x558c07f450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 -- 172.21.15.61:0/347761397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff228017070 con 0x558c07f3b5d0 2023-10-18T09:31:59.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff24139c700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:31:59.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff240b9b700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2300a31f0 tx=0x7ff230021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851103e1abe9d7b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff21bfff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:31:59.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23009b420 con 0x558c07f3b5d0 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff23009f570 con 0x558c07f3b5d0 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c07e92ef0 con 0x558c07f3b5d0 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.642+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2300a3720 con 0x558c07f3b5d0 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 10 client.26441 fetch_fsmap learned FSMap version 43 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x558c07cb8de0 con 0x558c07f3b5d0 2023-10-18T09:31:59.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558c07e92ef0 con 0x558c07f3b5d0 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff2300a1070 con 0x558c07f3b5d0 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 10 client.26441 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 20 client.26441 populate_metadata read hostname 'smithi061' 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07f50450 0x558c07f52870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 4 client.26441 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:31:59.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.643+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c07f0a8b0 con 0x558c07f50450 2023-10-18T09:31:59.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.644+0000 7ff23bfff700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07f50450 0x558c07f52870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.645+0000 7ff23bfff700 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07f50450 0x558c07f52870 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa34fd8cf114448b server_cookie=bc9bffe3d90c7290 in_seq=0 out_seq=0 2023-10-18T09:31:59.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.645+0000 7ff21bfff700 10 client.26441 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:59.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.645+0000 7ff21bfff700 1 -- 172.21.15.61:0/347761397 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558c07f0a8b0 con 0x558c07f50450 2023-10-18T09:31:59.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.645+0000 7ff21bfff700 10 client.26441 handle_command_reply: tid=0 2023-10-18T09:31:59.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.646+0000 7ff256815b80 1 client.26441 shutdown 2023-10-18T09:31:59.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.646+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 msgr2=0x558c07f44b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.646+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff2300a31f0 tx=0x7ff230021b60 comp rx=0 tx=0).stop 2023-10-18T09:31:59.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 shutdown_connections 2023-10-18T09:31:59.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07f50450 0x558c07f52870 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f44b60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 --2- 172.21.15.61:0/347761397 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 >> 172.21.15.61:0/347761397 conn(0x558c07eaf720 msgr2=0x558c07f3a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.648+0000 7ff21bfff700 0 client.26441 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:59.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.649+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 shutdown_connections 2023-10-18T09:31:59.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.650+0000 7ff256815b80 1 -- 172.21.15.61:0/347761397 wait complete. 2023-10-18T09:31:59.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.651+0000 7ff256815b80 20 client.26441 trim_cache size 0 max 16384 2023-10-18T09:31:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.676+0000 7ff256815b80 1 Processor -- start 2023-10-18T09:31:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.676+0000 7ff256815b80 1 -- start start 2023-10-18T09:31:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f4c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07d0e920 con 0x558c07f3c2d0 2023-10-18T09:31:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07f32550 con 0x558c07f3b5d0 2023-10-18T09:31:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff256815b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07e92ef0 con 0x558c07f3cc30 2023-10-18T09:31:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff240b9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:49788/0 (socket says 172.21.15.61:49788) 2023-10-18T09:31:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff240b9b700 1 -- 172.21.15.61:0/4066363214 learned_addr learned my addr 172.21.15.61:0/4066363214 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff24139c700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.677+0000 7ff23bfff700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f4c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 -- 172.21.15.61:0/4066363214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 msgr2=0x558c07f3f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 -- 172.21.15.61:0/4066363214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 msgr2=0x558c07f4c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f4c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 -- 172.21.15.61:0/4066363214 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c07ca1c60 con 0x558c07f3b5d0 2023-10-18T09:31:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff24139c700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:31:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff23bfff700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:31:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff240b9b700 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff23009b120 tx=0x7ff23009bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731ce5fc100c2d2a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff2277fe700 1 -- 172.21.15.61:0/4066363214 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23009f570 con 0x558c07f3b5d0 2023-10-18T09:31:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff2277fe700 1 -- 172.21.15.61:0/4066363214 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff23009b420 con 0x558c07f3b5d0 2023-10-18T09:31:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.678+0000 7ff2277fe700 1 -- 172.21.15.61:0/4066363214 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300b95b0 con 0x558c07f3b5d0 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 -- 172.21.15.61:0/4066363214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 msgr2=0x558c07f42ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff23009b120 tx=0x7ff23009bc60 comp rx=0 tx=0).stop 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 -- 172.21.15.61:0/4066363214 shutdown_connections 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f3f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3b5d0 0x558c07f42ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 --2- 172.21.15.61:0/4066363214 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3c2d0 0x558c07f4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 -- 172.21.15.61:0/4066363214 >> 172.21.15.61:0/4066363214 conn(0x558c07eaf720 msgr2=0x558c07f44070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 -- 172.21.15.61:0/4066363214 shutdown_connections 2023-10-18T09:31:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.679+0000 7ff256815b80 1 -- 172.21.15.61:0/4066363214 wait complete. 2023-10-18T09:31:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.680+0000 7ff256815b80 1 -- start start 2023-10-18T09:31:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.680+0000 7ff256815b80 1 Processor -- start 2023-10-18T09:31:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f4bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07f4a940 con 0x558c07f3b5d0 2023-10-18T09:31:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07f4aab0 con 0x558c07f3c2d0 2023-10-18T09:31:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c07f4ac70 con 0x558c07f3cc30 2023-10-18T09:31:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff256815b80 10 client.0 _ll_register_callbacks cb 0x558c07f4ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:31:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff23bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff23bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:49792/0 (socket says 172.21.15.61:49792) 2023-10-18T09:31:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff23bfff700 1 -- 172.21.15.61:0/450294865 learned_addr learned my addr 172.21.15.61:0/450294865 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:31:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff24139c700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.681+0000 7ff240b9b700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f4bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 -- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 msgr2=0x558c07f4c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 -- 172.21.15.61:0/450294865 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 msgr2=0x558c07f4bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:31:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f4bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 -- 172.21.15.61:0/450294865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c07cb8de0 con 0x558c07f3cc30 2023-10-18T09:31:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff23bfff700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:31:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff240b9b700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f4bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:31:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff24139c700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4a230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x558c07f4d2d0 tx=0x7ff2340124b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d886b8b78782ad4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff21bfff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234008e80 con 0x558c07f3cc30 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff228021070 con 0x558c07f3cc30 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.682+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff234004bb0 con 0x558c07f3cc30 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.683+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff234004d70 con 0x558c07f3cc30 2023-10-18T09:31:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.683+0000 7ff256815b80 10 client.26368 fetch_fsmap learned FSMap version 43 2023-10-18T09:31:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.683+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x558c07cdf250 con 0x558c07f3cc30 2023-10-18T09:31:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.683+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff228021070 con 0x558c07f3cc30 2023-10-18T09:31:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff234015070 con 0x558c07f3cc30 2023-10-18T09:31:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 10 client.26368 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:31:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 10 client.26368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 20 client.26368 populate_metadata read hostname 'smithi061' 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07fa58c0 0x558c07fa7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 4 client.26368 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.684+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c07e86e50 con 0x558c07fa58c0 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.685+0000 7ff240b9b700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07fa58c0 0x558c07fa7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:31:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.685+0000 7ff240b9b700 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07fa58c0 0x558c07fa7ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84900a3606dee921 server_cookie=ed877148f822dfe2 in_seq=0 out_seq=0 2023-10-18T09:31:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.686+0000 7ff21bfff700 10 client.26368 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.686+0000 7ff21bfff700 1 -- 172.21.15.61:0/450294865 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558c07e86e50 con 0x558c07fa58c0 2023-10-18T09:31:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.686+0000 7ff21bfff700 10 client.26368 handle_command_reply: tid=0 2023-10-18T09:31:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.686+0000 7ff256815b80 1 client.26368 shutdown 2023-10-18T09:31:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 msgr2=0x558c07f4a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4a230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x558c07f4d2d0 tx=0x7ff2340124b0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 shutdown_connections 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x558c07fa58c0 0x558c07fa7ce0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558c07f3cc30 0x558c07f4a230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x558c07f3b5d0 0x558c07f4bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff21bfff700 0 client.26368 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:31:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.687+0000 7ff256815b80 1 --2- 172.21.15.61:0/450294865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558c07f3c2d0 0x558c07f4c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.688+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 >> 172.21.15.61:0/450294865 conn(0x558c07eaf720 msgr2=0x558c07f426a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.688+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 shutdown_connections 2023-10-18T09:31:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.688+0000 7ff256815b80 1 -- 172.21.15.61:0/450294865 wait complete. 2023-10-18T09:31:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff256815b80 20 client.26368 trim_cache size 0 max 16384 2023-10-18T09:31:59.701 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:31:59.701 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:31:59.702 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:31:59.702 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:31:59.702 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:31:59.702 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:31:59.702 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:31:59.703 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:31:59.703 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:31:59.703 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:31:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 msgr2=0x7ff22c0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 0x7ff22c0a3a40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff228009c60 tx=0x7ff228009260 comp rx=0 tx=0).stop 2023-10-18T09:31:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 msgr2=0x7ff23c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:31:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b680 tx=0x7ff230004730 comp rx=0 tx=0).stop 2023-10-18T09:31:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 shutdown_connections 2023-10-18T09:31:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7ff22c0a1580 0x7ff22c0a3a40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff23c104560 0x7ff23c19ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff23c1029d0 0x7ff23c1989b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 --2- 172.21.15.61:0/2943328451 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7ff23c103bc0 0x7ff23c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:31:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.689+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 >> 172.21.15.61:0/2943328451 conn(0x7ff23c0fdf40 msgr2=0x7ff23c106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:31:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.690+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 shutdown_connections 2023-10-18T09:31:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:31:59.690+0000 7ff242dff700 1 -- 172.21.15.61:0/2943328451 wait complete. 2023-10-18T09:31:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:31:59 smithi110 ceph-mon[134818]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 483 op/s 2023-10-18T09:31:59.715 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:31:59 smithi061 ceph-mon[98573]: pgmap v761: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.6 MiB/s wr, 483 op/s 2023-10-18T09:31:59.718 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:32:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:01 smithi170 ceph-mon[135150]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2023-10-18T09:32:01.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:01 smithi110 ceph-mon[134818]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2023-10-18T09:32:01.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:01 smithi061 ceph-mon[98573]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.3 MiB/s wr, 450 op/s 2023-10-18T09:32:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:03 smithi170 ceph-mon[135150]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2023-10-18T09:32:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:03 smithi110 ceph-mon[134818]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2023-10-18T09:32:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:03 smithi061 ceph-mon[98573]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 499 op/s 2023-10-18T09:32:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 770.0 s, 426.4 tps, lat 234.445 ms stddev 259.789 2023-10-18T09:32:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:05 smithi170 ceph-mon[135150]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 458 op/s 2023-10-18T09:32:05.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:05 smithi110 ceph-mon[134818]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 458 op/s 2023-10-18T09:32:05.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:05 smithi061 ceph-mon[98573]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 458 op/s 2023-10-18T09:32:08.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:08 smithi170 ceph-mon[135150]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 467 op/s 2023-10-18T09:32:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:08 smithi110 ceph-mon[134818]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 467 op/s 2023-10-18T09:32:08.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:08 smithi061 ceph-mon[98573]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.0 MiB/s wr, 467 op/s 2023-10-18T09:32:09.341 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 775.0 s, 452.2 tps, lat 220.706 ms stddev 240.510 2023-10-18T09:32:09.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:09 smithi170 ceph-mon[135150]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-10-18T09:32:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:09 smithi110 ceph-mon[134818]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-10-18T09:32:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:09 smithi061 ceph-mon[98573]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.0 MiB/s wr, 473 op/s 2023-10-18T09:32:11.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:11 smithi170 ceph-mon[135150]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-18T09:32:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:11 smithi110 ceph-mon[134818]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-18T09:32:11.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:11 smithi061 ceph-mon[98573]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-18T09:32:13.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:13 smithi170 ceph-mon[135150]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2023-10-18T09:32:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:13 smithi110 ceph-mon[134818]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2023-10-18T09:32:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:13 smithi061 ceph-mon[98573]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2023-10-18T09:32:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 780.0 s, 493.6 tps, lat 208.916 ms stddev 249.047 2023-10-18T09:32:15.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:15 smithi170 ceph-mon[135150]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2023-10-18T09:32:15.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:15 smithi110 ceph-mon[134818]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2023-10-18T09:32:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:15 smithi061 ceph-mon[98573]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.0 MiB/s wr, 471 op/s 2023-10-18T09:32:18.397 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:18 smithi170 ceph-mon[135150]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 487 op/s 2023-10-18T09:32:18.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:18 smithi110 ceph-mon[134818]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 487 op/s 2023-10-18T09:32:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:18 smithi061 ceph-mon[98573]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.2 MiB/s wr, 487 op/s 2023-10-18T09:32:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 785.0 s, 512.4 tps, lat 190.911 ms stddev 235.395 2023-10-18T09:32:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:19 smithi170 ceph-mon[135150]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2023-10-18T09:32:19.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:19 smithi110 ceph-mon[134818]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2023-10-18T09:32:19.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:19 smithi061 ceph-mon[98573]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 510 op/s 2023-10-18T09:32:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:21 smithi170 ceph-mon[135150]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.6 MiB/s wr, 487 op/s 2023-10-18T09:32:21.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:21 smithi110 ceph-mon[134818]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.6 MiB/s wr, 487 op/s 2023-10-18T09:32:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:21 smithi061 ceph-mon[98573]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 6.6 MiB/s wr, 487 op/s 2023-10-18T09:32:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:23 smithi170 ceph-mon[135150]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.4 MiB/s wr, 557 op/s 2023-10-18T09:32:23.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:23 smithi110 ceph-mon[134818]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.4 MiB/s wr, 557 op/s 2023-10-18T09:32:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:23 smithi061 ceph-mon[98573]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.4 MiB/s wr, 557 op/s 2023-10-18T09:32:24.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 790.0 s, 442.6 tps, lat 228.321 ms stddev 280.278 2023-10-18T09:32:25.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:25 smithi170 ceph-mon[135150]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-18T09:32:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:25 smithi110 ceph-mon[134818]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-18T09:32:25.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:25 smithi061 ceph-mon[98573]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-18T09:32:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:28 smithi170 ceph-mon[135150]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2023-10-18T09:32:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:28 smithi110 ceph-mon[134818]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2023-10-18T09:32:28.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:28 smithi061 ceph-mon[98573]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 6.3 MiB/s wr, 478 op/s 2023-10-18T09:32:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 795.0 s, 464.2 tps, lat 213.762 ms stddev 243.388 2023-10-18T09:32:29.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:29 smithi170 ceph-mon[135150]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 6.6 MiB/s wr, 489 op/s 2023-10-18T09:32:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:29 smithi110 ceph-mon[134818]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 6.6 MiB/s wr, 489 op/s 2023-10-18T09:32:29.719 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:32:29.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:29 smithi061 ceph-mon[98573]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 6.6 MiB/s wr, 489 op/s 2023-10-18T09:32:30.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.099+0000 7f58be5ff700 1 -- 172.21.15.61:0/2092216784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b80747b0 msgr2=0x7f58b8072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.102 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.099+0000 7f58be5ff700 1 --2- 172.21.15.61:0/2092216784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b80747b0 0x7f58b8072c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 -- 172.21.15.61:0/2092216784 shutdown_connections 2023-10-18T09:32:30.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 --2- 172.21.15.61:0/2092216784 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b8108690 0x7f58b810ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.103 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 --2- 172.21.15.61:0/2092216784 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b80747b0 0x7f58b8072c00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 --2- 172.21.15.61:0/2092216784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80731d0 0x7f58b8073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 -- 172.21.15.61:0/2092216784 >> 172.21.15.61:0/2092216784 conn(0x7f58b80fbcf0 msgr2=0x7f58b80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 -- 172.21.15.61:0/2092216784 shutdown_connections 2023-10-18T09:32:30.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.100+0000 7f58be5ff700 1 -- 172.21.15.61:0/2092216784 wait complete. 2023-10-18T09:32:30.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.102+0000 7f58be5ff700 1 Processor -- start 2023-10-18T09:32:30.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.102+0000 7f58be5ff700 1 -- start start 2023-10-18T09:32:30.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.102+0000 7f58be5ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 0x7f58b8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58be5ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80747b0 0x7f58b8198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58be5ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58be5ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a04a0 con 0x7f58b80731d0 2023-10-18T09:32:30.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58be5ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a05e0 con 0x7f58b8108690 2023-10-18T09:32:30.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58be5ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b81a0770 con 0x7f58b80747b0 2023-10-18T09:32:30.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58bcb9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58bcb9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43392/0 (socket says 172.21.15.61:43392) 2023-10-18T09:32:30.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58bcb9c700 1 -- 172.21.15.61:0/1952836329 learned_addr learned my addr 172.21.15.61:0/1952836329 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:32:30.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58b77fe700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80747b0 0x7f58b8198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.103+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 0x7f58b8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 -- 172.21.15.61:0/1952836329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80747b0 msgr2=0x7f58b8198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80747b0 0x7f58b8198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 -- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 msgr2=0x7f58b8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 0x7f58b8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 -- 172.21.15.61:0/1952836329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58b8108690 2023-10-18T09:32:30.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 0x7f58b8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:32:30.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.104+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58b0013990 tx=0x7f58b0013d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.105+0000 7f58b57fa700 1 -- 172.21.15.61:0/1952836329 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00110d0 con 0x7f58b8108690 2023-10-18T09:32:30.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.105+0000 7f58b57fa700 1 -- 172.21.15.61:0/1952836329 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58b00116f0 con 0x7f58b8108690 2023-10-18T09:32:30.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.105+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b8100230 con 0x7f58b8108690 2023-10-18T09:32:30.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.106+0000 7f58b57fa700 1 -- 172.21.15.61:0/1952836329 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b001cba0 con 0x7f58b8108690 2023-10-18T09:32:30.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.106+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b8100740 con 0x7f58b8108690 2023-10-18T09:32:30.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.115+0000 7f58b57fa700 1 -- 172.21.15.61:0/1952836329 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f58b0011230 con 0x7f58b8108690 2023-10-18T09:32:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.117+0000 7f58b57fa700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 0x7f58a00a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.120 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.118+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 0x7f58a00a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.121+0000 7f58b57fa700 1 -- 172.21.15.61:0/1952836329 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f58b0006e40 con 0x7f58b8108690 2023-10-18T09:32:30.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.121+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 0x7f58a00a3a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f58a800b2e0 tx=0x7f58a8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 Processor -- start 2023-10-18T09:32:30.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 -- start start 2023-10-18T09:32:30.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e172fe76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 0x55e172fe6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe65d0 0x55e172fe6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172f596e0 con 0x55e172fe65d0 2023-10-18T09:32:30.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172f3fc60 con 0x55e172fe7c30 2023-10-18T09:32:30.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.124+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172f3d580 con 0x55e172fe72d0 2023-10-18T09:32:30.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.125+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e172fe76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.125+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e172fe76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:54964/0 (socket says 172.21.15.61:54964) 2023-10-18T09:32:30.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.125+0000 7f58b7fff700 1 -- 172.21.15.61:0/3763715924 learned_addr learned my addr 172.21.15.61:0/3763715924 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:32:30.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.125+0000 7f58b77fe700 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 0x55e172fe6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.125+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe65d0 0x55e172fe6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 -- 172.21.15.61:0/3763715924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 msgr2=0x55e172fe76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e172fe76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 -- 172.21.15.61:0/3763715924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe65d0 msgr2=0x55e172fe6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe65d0 0x55e172fe6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 -- 172.21.15.61:0/3763715924 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e172ef63f0 con 0x55e172fe7c30 2023-10-18T09:32:30.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f58b77fe700 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 0x55e172fe6090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c00 tx=0x7f58ac009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60ddfbad981f3d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.126+0000 7f589e7fc700 1 -- 172.21.15.61:0/3763715924 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011ab0 con 0x55e172fe7c30 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.127+0000 7f589e7fc700 1 -- 172.21.15.61:0/3763715924 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58ac011c10 con 0x55e172fe7c30 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.127+0000 7f589e7fc700 1 -- 172.21.15.61:0/3763715924 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00f6a0 con 0x55e172fe7c30 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/3763715924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 msgr2=0x55e172fe6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 0x55e172fe6090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009c00 tx=0x7f58ac009f10 comp rx=0 tx=0).stop 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/3763715924 shutdown_connections 2023-10-18T09:32:30.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe65d0 0x55e172fe6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e172fe76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/3763715924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe7c30 0x55e172fe6090 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/3763715924 >> 172.21.15.61:0/3763715924 conn(0x55e172f5a720 msgr2=0x55e172fe0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.128+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/3763715924 shutdown_connections 2023-10-18T09:32:30.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.129+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/3763715924 wait complete. 2023-10-18T09:32:30.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.129+0000 7f58d1ecfb80 1 -- start start 2023-10-18T09:32:30.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.130+0000 7f58d1ecfb80 1 Processor -- start 2023-10-18T09:32:30.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 0x55e173001040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173008080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172f38e70 con 0x55e172fe7c30 2023-10-18T09:32:30.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172ed5710 con 0x55e172fe72d0 2023-10-18T09:32:30.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172ed53b0 con 0x55e172fe65d0 2023-10-18T09:32:30.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:54970/0 (socket says 172.21.15.61:54970) 2023-10-18T09:32:30.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58b7fff700 1 -- 172.21.15.61:0/772567154 learned_addr learned my addr 172.21.15.61:0/772567154 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:32:30.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58d1ecfb80 10 client.0 _ll_register_callbacks cb 0x55e172e1fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:32:30.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.131+0000 7f58b77fe700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 0x55e173001040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.132+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173008080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.132+0000 7f58b7fff700 1 -- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 msgr2=0x55e173001040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58b7fff700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 0x55e173001040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58b7fff700 1 -- 172.21.15.61:0/772567154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 msgr2=0x55e173008080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58b7fff700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173008080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58b7fff700 1 -- 172.21.15.61:0/772567154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac009830 con 0x55e172fe65d0 2023-10-18T09:32:30.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58bcb9c700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173008080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:32:30.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.135+0000 7f58b77fe700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 0x55e173001040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:32:30.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.138+0000 7f58b7fff700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005590 tx=0x7f58a8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6db6540803dfe7d server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.138+0000 7f58977fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:32:30.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.138+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a802c040 con 0x55e172fe65d0 2023-10-18T09:32:30.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.138+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f58a8004d20 con 0x55e172fe65d0 2023-10-18T09:32:30.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.138+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e172f3def0 con 0x55e172fe65d0 2023-10-18T09:32:30.142 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58a801e070 con 0x55e172fe65d0 2023-10-18T09:32:30.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58d1ecfb80 10 client.26380 fetch_fsmap learned FSMap version 43 2023-10-18T09:32:30.143 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e172d63de0 con 0x55e172fe65d0 2023-10-18T09:32:30.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e172f3def0 con 0x55e172fe65d0 2023-10-18T09:32:30.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58a8030420 con 0x55e172fe65d0 2023-10-18T09:32:30.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58d1ecfb80 10 client.26380 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:32:30.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58d1ecfb80 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:32:30.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.139+0000 7f58d1ecfb80 20 client.26380 populate_metadata read hostname 'smithi061' 2023-10-18T09:32:30.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.140+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e17300c980 0x55e17300eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.140+0000 7f58d1ecfb80 4 client.26380 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:32:30.145 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.140+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e172fb58b0 con 0x55e17300c980 2023-10-18T09:32:30.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.145+0000 7f58b77fe700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e17300c980 0x55e17300eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.146+0000 7f58b77fe700 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e17300c980 0x55e17300eda0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96aeac14708ea188 server_cookie=3a87b6e9ddd55cfa in_seq=0 out_seq=0 2023-10-18T09:32:30.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58977fe700 10 client.26380 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:32:30.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58977fe700 1 -- 172.21.15.61:0/772567154 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e172fb58b0 con 0x55e17300c980 2023-10-18T09:32:30.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58977fe700 10 client.26380 handle_command_reply: tid=0 2023-10-18T09:32:30.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 client.26380 shutdown 2023-10-18T09:32:30.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 msgr2=0x55e173000b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58a8005590 tx=0x7f58a8009ca0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 shutdown_connections 2023-10-18T09:32:30.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e17300c980 0x55e17300eda0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173008080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe65d0 0x55e173000b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/772567154 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe72d0 0x55e173001040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58977fe700 0 client.26380 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:32:30.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.148+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 >> 172.21.15.61:0/772567154 conn(0x55e172f5a720 msgr2=0x55e172fe5080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.149+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 shutdown_connections 2023-10-18T09:32:30.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.149+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/772567154 wait complete. 2023-10-18T09:32:30.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.149+0000 7f58d1ecfb80 20 client.26380 trim_cache size 0 max 16384 2023-10-18T09:32:30.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 Processor -- start 2023-10-18T09:32:30.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 -- start start 2023-10-18T09:32:30.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e173008040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.211 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173000010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172db9920 con 0x55e172fe7c30 2023-10-18T09:32:30.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172fe10c0 con 0x55e172fe65d0 2023-10-18T09:32:30.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.208+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e172fe6090 con 0x55e172fe72d0 2023-10-18T09:32:30.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.209+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.209+0000 7f58bcb9c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173000010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.209+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43428/0 (socket says 172.21.15.61:43428) 2023-10-18T09:32:30.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.209+0000 7f58b7fff700 1 -- 172.21.15.61:0/1154150575 learned_addr learned my addr 172.21.15.61:0/1154150575 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:32:30.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.210+0000 7f58b7fff700 1 -- 172.21.15.61:0/1154150575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 msgr2=0x55e173008040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.210+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e173008040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.210+0000 7f58b7fff700 1 -- 172.21.15.61:0/1154150575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 msgr2=0x55e173000010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.210+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173000010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.210+0000 7f58b7fff700 1 -- 172.21.15.61:0/1154150575 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e172d4cc60 con 0x55e172fe65d0 2023-10-18T09:32:30.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.211+0000 7f58b7fff700 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a8030820 tx=0x7f58a8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eca5c08d3988d05 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.211+0000 7f589effd700 1 -- 172.21.15.61:0/1154150575 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a802c040 con 0x55e172fe65d0 2023-10-18T09:32:30.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.211+0000 7f589effd700 1 -- 172.21.15.61:0/1154150575 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58a8030a10 con 0x55e172fe65d0 2023-10-18T09:32:30.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.211+0000 7f589effd700 1 -- 172.21.15.61:0/1154150575 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800fb40 con 0x55e172fe65d0 2023-10-18T09:32:30.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.212+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/1154150575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 msgr2=0x55e172fdb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.212+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58a8030820 tx=0x7f58a8005e80 comp rx=0 tx=0).stop 2023-10-18T09:32:30.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.213+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/1154150575 shutdown_connections 2023-10-18T09:32:30.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.213+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe7c30 0x55e173000010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.213+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e172fdb5d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.213+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/1154150575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe72d0 0x55e173008040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.213+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/1154150575 >> 172.21.15.61:0/1154150575 conn(0x55e172f5a720 msgr2=0x55e172fe69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.215+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/1154150575 shutdown_connections 2023-10-18T09:32:30.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.215+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/1154150575 wait complete. 2023-10-18T09:32:30.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.215+0000 7f58d1ecfb80 1 -- start start 2023-10-18T09:32:30.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.216+0000 7f58d1ecfb80 1 Processor -- start 2023-10-18T09:32:30.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e173005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 0x55e1730060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe7c30 0x55e173007030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e173007570 con 0x55e172fe72d0 2023-10-18T09:32:30.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1730076e0 con 0x55e172fe65d0 2023-10-18T09:32:30.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1730078a0 con 0x55e172fe7c30 2023-10-18T09:32:30.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.217+0000 7f58d1ecfb80 10 client.0 _ll_register_callbacks cb 0x55e172e1fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:32:30.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.218+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e173005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.224 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.218+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e173005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43440/0 (socket says 172.21.15.61:43440) 2023-10-18T09:32:30.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.218+0000 7f58b7fff700 1 -- 172.21.15.61:0/706205463 learned_addr learned my addr 172.21.15.61:0/706205463 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:32:30.225 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.218+0000 7f58b77fe700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 0x55e1730060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.222+0000 7f58b77fe700 1 -- 172.21.15.61:0/706205463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe7c30 msgr2=0x55e173007030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.222+0000 7f58b77fe700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe7c30 0x55e173007030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.222+0000 7f58b77fe700 1 -- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 msgr2=0x55e173005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:32:30.226 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.222+0000 7f58b77fe700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e173005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.222+0000 7f58b77fe700 1 -- 172.21.15.61:0/706205463 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e172d63de0 con 0x55e172fe72d0 2023-10-18T09:32:30.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.223+0000 7f58b77fe700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 0x55e1730060c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009bd0 tx=0x7f58ac012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a227285d60f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:32:30.227 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.223+0000 7f58977fe700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:32:30.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.224+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011a60 con 0x55e172fe72d0 2023-10-18T09:32:30.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.224+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f58ac02a160 con 0x55e172fe72d0 2023-10-18T09:32:30.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.224+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ac011c10 con 0x55e172fe72d0 2023-10-18T09:32:30.228 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.225+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ac02a340 con 0x55e172fe72d0 2023-10-18T09:32:30.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.225+0000 7f58d1ecfb80 10 client.16245 fetch_fsmap learned FSMap version 43 2023-10-18T09:32:30.229 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.225+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e172d8a250 con 0x55e172fe72d0 2023-10-18T09:32:30.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.227+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58ac011c10 con 0x55e172fe72d0 2023-10-18T09:32:30.230 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.227+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f58ac009c30 con 0x55e172fe72d0 2023-10-18T09:32:30.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.227+0000 7f58d1ecfb80 10 client.16245 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:32:30.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.227+0000 7f58d1ecfb80 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:32:30.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.228+0000 7f58d1ecfb80 20 client.16245 populate_metadata read hostname 'smithi061' 2023-10-18T09:32:30.232 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.228+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e173061470 0x55e173063890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:32:30.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.228+0000 7f58d1ecfb80 4 client.16245 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:32:30.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.228+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e172f31e50 con 0x55e173061470 2023-10-18T09:32:30.233 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.230+0000 7f58b7fff700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e173061470 0x55e173063890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:32:30.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.231+0000 7f58b7fff700 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e173061470 0x55e173063890 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c53e09a3c68a18 server_cookie=8ad6b55c00e0da8d in_seq=0 out_seq=0 2023-10-18T09:32:30.234 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.231+0000 7f58977fe700 10 client.16245 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:32:30.236 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.234+0000 7f58977fe700 1 -- 172.21.15.61:0/706205463 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55e172f31e50 con 0x55e173061470 2023-10-18T09:32:30.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.234+0000 7f58977fe700 10 client.16245 handle_command_reply: tid=0 2023-10-18T09:32:30.237 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.234+0000 7f58d1ecfb80 1 client.16245 shutdown 2023-10-18T09:32:30.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.234+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 msgr2=0x55e1730060c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.234+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 0x55e1730060c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009bd0 tx=0x7f58ac012c50 comp rx=0 tx=0).stop 2023-10-18T09:32:30.238 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 shutdown_connections 2023-10-18T09:32:30.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55e173061470 0x55e173063890 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e172fe7c30 0x55e173007030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e172fe65d0 0x55e173005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 --2- 172.21.15.61:0/706205463 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55e172fe72d0 0x55e1730060c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.239 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 >> 172.21.15.61:0/706205463 conn(0x55e172f5a720 msgr2=0x55e172fed6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58977fe700 0 client.16245 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:32:30.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 shutdown_connections 2023-10-18T09:32:30.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.235+0000 7f58d1ecfb80 1 -- 172.21.15.61:0/706205463 wait complete. 2023-10-18T09:32:30.240 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.236+0000 7f58d1ecfb80 20 client.16245 trim_cache size 0 max 16384 2023-10-18T09:32:30.241 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:32:30.241 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:32:30.241 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:32:30.241 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:32:30.242 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:32:30.242 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:32:30.242 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:32:30.242 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:32:30.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:32:30.243 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:32:30.243 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.238+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 msgr2=0x7f58a00a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.238+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 0x7f58a00a3a40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f58a800b2e0 tx=0x7f58a8005fb0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.238+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 msgr2=0x7f58b819fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.238+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f58b0013990 tx=0x7f58b0013d50 comp rx=0 tx=0).stop 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 shutdown_connections 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f58a00a1580 0x7f58a00a3a40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.244 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58b8108690 0x7f58b819fed0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f58b80731d0 0x7f58b8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 --2- 172.21.15.61:0/1952836329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b80747b0 0x7f58b8198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:32:30.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 >> 172.21.15.61:0/1952836329 conn(0x7f58b80fbcf0 msgr2=0x7f58b81045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:32:30.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 shutdown_connections 2023-10-18T09:32:30.245 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:32:30.240+0000 7f58be5ff700 1 -- 172.21.15.61:0/1952836329 wait complete. 2023-10-18T09:32:30.279 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:32:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:31 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:32:31.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:31 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:32:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:31 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:32:32.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:32 smithi110 ceph-mon[134818]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.0 MiB/s wr, 446 op/s 2023-10-18T09:32:32.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:32 smithi061 ceph-mon[98573]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.0 MiB/s wr, 446 op/s 2023-10-18T09:32:32.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:32 smithi170 ceph-mon[135150]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 6.0 MiB/s wr, 446 op/s 2023-10-18T09:32:33.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:32:33.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:32:33.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:32:33.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:32:33.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:32:33.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:32:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:32:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:32:33.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:32:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 800.0 s, 476.0 tps, lat 205.292 ms stddev 225.476 2023-10-18T09:32:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:34 smithi061 ceph-mon[98573]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.1 MiB/s wr, 495 op/s 2023-10-18T09:32:34.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:34 smithi170 ceph-mon[135150]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.1 MiB/s wr, 495 op/s 2023-10-18T09:32:34.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:34 smithi110 ceph-mon[134818]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 6.1 MiB/s wr, 495 op/s 2023-10-18T09:32:35.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:35 smithi170 ceph-mon[135150]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 445 op/s 2023-10-18T09:32:35.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:35 smithi110 ceph-mon[134818]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 445 op/s 2023-10-18T09:32:35.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:35 smithi061 ceph-mon[98573]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 445 op/s 2023-10-18T09:32:38.401 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:38 smithi170 ceph-mon[135150]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.9 MiB/s wr, 468 op/s 2023-10-18T09:32:38.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:38 smithi110 ceph-mon[134818]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.9 MiB/s wr, 468 op/s 2023-10-18T09:32:38.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:38 smithi061 ceph-mon[98573]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.9 MiB/s wr, 468 op/s 2023-10-18T09:32:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 805.0 s, 535.6 tps, lat 194.618 ms stddev 210.945 2023-10-18T09:32:39.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:39 smithi170 ceph-mon[135150]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 458 op/s 2023-10-18T09:32:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:39 smithi110 ceph-mon[134818]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 458 op/s 2023-10-18T09:32:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:39 smithi061 ceph-mon[98573]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.2 MiB/s wr, 458 op/s 2023-10-18T09:32:41.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:41 smithi170 ceph-mon[135150]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2023-10-18T09:32:41.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:41 smithi110 ceph-mon[134818]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2023-10-18T09:32:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:41 smithi061 ceph-mon[98573]: pgmap v782: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 5.4 MiB/s wr, 452 op/s 2023-10-18T09:32:43.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:43 smithi170 ceph-mon[135150]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-10-18T09:32:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:43 smithi110 ceph-mon[134818]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-10-18T09:32:43.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:43 smithi061 ceph-mon[98573]: pgmap v783: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 529 op/s 2023-10-18T09:32:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 810.0 s, 467.2 tps, lat 208.036 ms stddev 258.014 2023-10-18T09:32:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:45 smithi170 ceph-mon[135150]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 485 op/s 2023-10-18T09:32:45.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:45 smithi110 ceph-mon[134818]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 485 op/s 2023-10-18T09:32:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:45 smithi061 ceph-mon[98573]: pgmap v784: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 5.6 MiB/s wr, 485 op/s 2023-10-18T09:32:48.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:48 smithi170 ceph-mon[135150]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2023-10-18T09:32:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:48 smithi110 ceph-mon[134818]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2023-10-18T09:32:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:48 smithi061 ceph-mon[98573]: pgmap v785: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 5.7 MiB/s wr, 483 op/s 2023-10-18T09:32:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 815.0 s, 480.8 tps, lat 211.197 ms stddev 281.803 2023-10-18T09:32:49.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:49 smithi170 ceph-mon[135150]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 484 op/s 2023-10-18T09:32:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:49 smithi110 ceph-mon[134818]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 484 op/s 2023-10-18T09:32:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:49 smithi061 ceph-mon[98573]: pgmap v786: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.7 MiB/s wr, 484 op/s 2023-10-18T09:32:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:51 smithi170 ceph-mon[135150]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2023-10-18T09:32:51.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:51 smithi110 ceph-mon[134818]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2023-10-18T09:32:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:51 smithi061 ceph-mon[98573]: pgmap v787: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.7 MiB/s wr, 482 op/s 2023-10-18T09:32:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:53 smithi170 ceph-mon[135150]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.9 MiB/s wr, 534 op/s 2023-10-18T09:32:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:53 smithi110 ceph-mon[134818]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.9 MiB/s wr, 534 op/s 2023-10-18T09:32:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:53 smithi061 ceph-mon[98573]: pgmap v788: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.9 MiB/s wr, 534 op/s 2023-10-18T09:32:54.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 820.0 s, 463.8 tps, lat 209.641 ms stddev 234.516 2023-10-18T09:32:55.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:55 smithi170 ceph-mon[135150]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2023-10-18T09:32:55.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:55 smithi110 ceph-mon[134818]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2023-10-18T09:32:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:55 smithi061 ceph-mon[98573]: pgmap v789: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 5.5 MiB/s wr, 463 op/s 2023-10-18T09:32:58.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:58 smithi170 ceph-mon[135150]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2023-10-18T09:32:58.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:58 smithi110 ceph-mon[134818]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2023-10-18T09:32:58.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:58 smithi061 ceph-mon[98573]: pgmap v790: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 5.7 MiB/s wr, 479 op/s 2023-10-18T09:32:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 825.0 s, 508.6 tps, lat 198.510 ms stddev 214.493 2023-10-18T09:32:59.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:32:59 smithi170 ceph-mon[135150]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 471 op/s 2023-10-18T09:32:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:32:59 smithi110 ceph-mon[134818]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 471 op/s 2023-10-18T09:32:59.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:32:59 smithi061 ceph-mon[98573]: pgmap v791: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.7 MiB/s wr, 471 op/s 2023-10-18T09:33:00.280 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:33:00.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.626+0000 7f7a9bbff700 1 -- 172.21.15.61:0/1862277077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a940ff690 msgr2=0x7f7a940ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.629 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.626+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/1862277077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a940ff690 0x7f7a940ffab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009990 tx=0x7f7a84009ca0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 -- 172.21.15.61:0/1862277077 shutdown_connections 2023-10-18T09:33:00.632 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/1862277077 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a94101220 0x7f7a94067830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/1862277077 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a940ff690 0x7f7a940ffab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/1862277077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94100880 0x7f7a94100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 -- 172.21.15.61:0/1862277077 >> 172.21.15.61:0/1862277077 conn(0x7f7a940fac00 msgr2=0x7f7a940fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 -- 172.21.15.61:0/1862277077 shutdown_connections 2023-10-18T09:33:00.633 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.630+0000 7f7a9bbff700 1 -- 172.21.15.61:0/1862277077 wait complete. 2023-10-18T09:33:00.634 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.631+0000 7f7a9bbff700 1 Processor -- start 2023-10-18T09:33:00.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 -- start start 2023-10-18T09:33:00.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a94100880 0x7f7a94198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.635 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 0x7f7a9419fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a0490 con 0x7f7a940ff690 2023-10-18T09:33:00.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a05d0 con 0x7f7a94100880 2023-10-18T09:33:00.636 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.632+0000 7f7a9bbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a941a0760 con 0x7f7a94101220 2023-10-18T09:33:00.637 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55370/0 (socket says 172.21.15.61:55370) 2023-10-18T09:33:00.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 -- 172.21.15.61:0/3474179687 learned_addr learned my addr 172.21.15.61:0/3474179687 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:00.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9919a700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a94100880 0x7f7a94198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.638 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 0x7f7a9419fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 -- 172.21.15.61:0/3474179687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 msgr2=0x7f7a9419fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 0x7f7a9419fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 -- 172.21.15.61:0/3474179687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a94100880 msgr2=0x7f7a94198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a94100880 0x7f7a94198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.639 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9999b700 1 -- 172.21.15.61:0/3474179687 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a840095f0 con 0x7f7a940ff690 2023-10-18T09:33:00.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.633+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 0x7f7a9419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:33:00.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.634+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400c010 tx=0x7f7a8400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.634+0000 7f7a8affd700 1 -- 172.21.15.61:0/3474179687 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8401d070 con 0x7f7a940ff690 2023-10-18T09:33:00.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.634+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a941a09e0 con 0x7f7a940ff690 2023-10-18T09:33:00.640 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.635+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a0e60 con 0x7f7a940ff690 2023-10-18T09:33:00.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.635+0000 7f7a8affd700 1 -- 172.21.15.61:0/3474179687 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a8400fc20 con 0x7f7a940ff690 2023-10-18T09:33:00.641 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.635+0000 7f7a8affd700 1 -- 172.21.15.61:0/3474179687 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a840173f0 con 0x7f7a940ff690 2023-10-18T09:33:00.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.645+0000 7f7a8affd700 1 -- 172.21.15.61:0/3474179687 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7a84017570 con 0x7f7a940ff690 2023-10-18T09:33:00.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.646+0000 7f7aaf474b80 1 Processor -- start 2023-10-18T09:33:00.649 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 -- start start 2023-10-18T09:33:00.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea6566e0 con 0x5599ea6ed440 2023-10-18T09:33:00.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea63cc60 con 0x5599ea6ecae0 2023-10-18T09:33:00.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.647+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea63a580 con 0x5599ea6ebde0 2023-10-18T09:33:00.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.648+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.648+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6ec260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:35530/0 (socket says 172.21.15.61:35530) 2023-10-18T09:33:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.648+0000 7f7a9919a700 1 -- 172.21.15.61:0/3351028996 learned_addr learned my addr 172.21.15.61:0/3351028996 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.648+0000 7f7a8affd700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 0x7f7a800a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.649+0000 7f7a8affd700 1 -- 172.21.15.61:0/3474179687 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7a840ce4b0 con 0x7f7a940ff690 2023-10-18T09:33:00.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.649+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.651+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 0x7f7a800a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.651+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.655 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 0x7f7a800a3a40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7a90014920 tx=0x7f7a9000d9f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9999b700 1 -- 172.21.15.61:0/3351028996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 msgr2=0x5599ea6ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.656 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9999b700 1 -- 172.21.15.61:0/3351028996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 msgr2=0x5599ea6ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6ecf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.652+0000 7f7a9999b700 1 -- 172.21.15.61:0/3351028996 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ea5f33f0 con 0x5599ea6ed440 2023-10-18T09:33:00.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.653+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6eb8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cbc00 tx=0x7f7a840a38a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618c967223a17f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.653+0000 7f7a7ffff700 1 -- 172.21.15.61:0/3351028996 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8409b2b0 con 0x5599ea6ed440 2023-10-18T09:33:00.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.654+0000 7f7a7ffff700 1 -- 172.21.15.61:0/3351028996 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a840ce2a0 con 0x5599ea6ed440 2023-10-18T09:33:00.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.654+0000 7f7a7ffff700 1 -- 172.21.15.61:0/3351028996 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a84017a80 con 0x5599ea6ed440 2023-10-18T09:33:00.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3351028996 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 msgr2=0x5599ea6eb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6eb8a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7a840cbc00 tx=0x7f7a840a38a0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3351028996 shutdown_connections 2023-10-18T09:33:00.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.655+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3351028996 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6eb8a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.660 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.656+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3351028996 >> 172.21.15.61:0/3351028996 conn(0x5599ea657720 msgr2=0x5599ea6ddb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.656+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3351028996 shutdown_connections 2023-10-18T09:33:00.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.656+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3351028996 wait complete. 2023-10-18T09:33:00.661 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.657+0000 7f7aaf474b80 1 -- start start 2023-10-18T09:33:00.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.657+0000 7f7aaf474b80 1 Processor -- start 2023-10-18T09:33:00.662 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.660+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.660+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.663 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.660+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6e9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.660+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.660+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45568/0 (socket says 172.21.15.61:45568) 2023-10-18T09:33:00.665 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.661+0000 7f7a9a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6e9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.661+0000 7f7a9a19c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6e9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:55388/0 (socket says 172.21.15.61:55388) 2023-10-18T09:33:00.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.662+0000 7f7a9919a700 1 -- 172.21.15.61:0/252959745 learned_addr learned my addr 172.21.15.61:0/252959745 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:00.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.663+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea635e70 con 0x5599ea6ed440 2023-10-18T09:33:00.666 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.663+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea5d2710 con 0x5599ea6ecae0 2023-10-18T09:33:00.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.663+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea5d23b0 con 0x5599ea6ebde0 2023-10-18T09:33:00.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.663+0000 7f7aaf474b80 10 client.0 _ll_register_callbacks cb 0x5599ea51ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:33:00.667 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.664+0000 7f7a9999b700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.665+0000 7f7a9919a700 1 -- 172.21.15.61:0/252959745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 msgr2=0x5599ea6e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.669 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.665+0000 7f7a9919a700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6e4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.665+0000 7f7a9919a700 1 -- 172.21.15.61:0/252959745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 msgr2=0x5599ea6e9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.665+0000 7f7a9919a700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6e9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.665+0000 7f7a9919a700 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a941a0e60 con 0x5599ea6ecae0 2023-10-18T09:33:00.670 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.666+0000 7f7a9999b700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:33:00.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.670+0000 7f7a9919a700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00abe0 tx=0x7f7a8c00acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90af08dde4248a20 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.670+0000 7f7a7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:33:00.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.675+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c011040 con 0x5599ea6ecae0 2023-10-18T09:33:00.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.675+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7a8c006e40 con 0x5599ea6ecae0 2023-10-18T09:33:00.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.676+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599ea63aef0 con 0x5599ea6ecae0 2023-10-18T09:33:00.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.677+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a8c00ae40 con 0x5599ea6ecae0 2023-10-18T09:33:00.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.677+0000 7f7aaf474b80 10 client.26477 fetch_fsmap learned FSMap version 43 2023-10-18T09:33:00.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.677+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5599ea460de0 con 0x5599ea6ecae0 2023-10-18T09:33:00.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599ea63aef0 con 0x5599ea6ecae0 2023-10-18T09:33:00.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a8c014340 con 0x5599ea6ecae0 2023-10-18T09:33:00.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 10 client.26477 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:33:00.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:33:00.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 20 client.26477 populate_metadata read hostname 'smithi061' 2023-10-18T09:33:00.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea7096f0 0x5599ea70bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 4 client.26477 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:33:00.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599ea6b28b0 con 0x5599ea7096f0 2023-10-18T09:33:00.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.678+0000 7f7a9999b700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea7096f0 0x5599ea70bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.680+0000 7f7a7d7fa700 10 client.26477 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:00.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.681+0000 7f7a9999b700 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea7096f0 0x5599ea70bb10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3eba506379bd74 server_cookie=1208aa6ea5b82397 in_seq=0 out_seq=0 2023-10-18T09:33:00.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.682+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/252959745 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599ea6b28b0 con 0x5599ea7096f0 2023-10-18T09:33:00.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.682+0000 7f7a7d7fa700 10 client.26477 handle_command_reply: tid=0 2023-10-18T09:33:00.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.682+0000 7f7aaf474b80 1 client.26477 shutdown 2023-10-18T09:33:00.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 msgr2=0x5599ea6e4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00abe0 tx=0x7f7a8c00acc0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 shutdown_connections 2023-10-18T09:33:00.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea7096f0 0x5599ea70bb10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ed440 0x5599ea6e9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ebde0 0x5599ea6e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7a7d7fa700 0 client.26477 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:00.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/252959745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4f80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.687 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.683+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 >> 172.21.15.61:0/252959745 conn(0x5599ea657720 msgr2=0x5599ea6efae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.685+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 shutdown_connections 2023-10-18T09:33:00.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.685+0000 7f7aaf474b80 1 -- 172.21.15.61:0/252959745 wait complete. 2023-10-18T09:33:00.688 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.685+0000 7f7aaf474b80 20 client.26477 trim_cache size 0 max 16384 2023-10-18T09:33:00.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.721+0000 7f7aaf474b80 1 Processor -- start 2023-10-18T09:33:00.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.721+0000 7f7aaf474b80 1 -- start start 2023-10-18T09:33:00.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ebde0 0x5599ea6e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea4b6920 con 0x5599ea6ebde0 2023-10-18T09:33:00.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea6dfb80 con 0x5599ea6ecae0 2023-10-18T09:33:00.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7aaf474b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea6e5910 con 0x5599ea6ed440 2023-10-18T09:33:00.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7a9a19c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7a9919a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45582/0 (socket says 172.21.15.61:45582) 2023-10-18T09:33:00.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7a9919a700 1 -- 172.21.15.61:0/1766628596 learned_addr learned my addr 172.21.15.61:0/1766628596 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:00.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.722+0000 7f7a9a19c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:35544/0 (socket says 172.21.15.61:35544) 2023-10-18T09:33:00.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.723+0000 7f7a9a19c700 1 -- 172.21.15.61:0/1766628596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 msgr2=0x5599ea6e4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.723+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.723+0000 7f7a9a19c700 1 -- 172.21.15.61:0/1766628596 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ebde0 msgr2=0x5599ea6e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.723+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ebde0 0x5599ea6e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.723+0000 7f7a9a19c700 1 -- 172.21.15.61:0/1766628596 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599ea449c60 con 0x5599ea6ed440 2023-10-18T09:33:00.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.724+0000 7f7a9a19c700 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ddd0 tx=0x7f7a9001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7013874e640e951 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.725+0000 7f7a88bff700 1 -- 172.21.15.61:0/1766628596 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90019070 con 0x5599ea6ed440 2023-10-18T09:33:00.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.726+0000 7f7a88bff700 1 -- 172.21.15.61:0/1766628596 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a90024070 con 0x5599ea6ed440 2023-10-18T09:33:00.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.726+0000 7f7a88bff700 1 -- 172.21.15.61:0/1766628596 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90019070 con 0x5599ea6ed440 2023-10-18T09:33:00.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.727+0000 7f7aaf474b80 1 -- 172.21.15.61:0/1766628596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 msgr2=0x5599ea6e2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.727+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000ddd0 tx=0x7f7a9001d000 comp rx=0 tx=0).stop 2023-10-18T09:33:00.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 -- 172.21.15.61:0/1766628596 shutdown_connections 2023-10-18T09:33:00.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea6e2f90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ebde0 0x5599ea6e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/1766628596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ecae0 0x5599ea6e4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 -- 172.21.15.61:0/1766628596 >> 172.21.15.61:0/1766628596 conn(0x5599ea657720 msgr2=0x5599ea6e4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 -- 172.21.15.61:0/1766628596 shutdown_connections 2023-10-18T09:33:00.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.728+0000 7f7aaf474b80 1 -- 172.21.15.61:0/1766628596 wait complete. 2023-10-18T09:33:00.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.729+0000 7f7aaf474b80 1 -- start start 2023-10-18T09:33:00.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.729+0000 7f7aaf474b80 1 Processor -- start 2023-10-18T09:33:00.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.730+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.730+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ecae0 0x5599ea756050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.730+0000 7f7aaf474b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea75d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.731+0000 7f7a9999b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.731+0000 7f7a9999b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:45594/0 (socket says 172.21.15.61:45594) 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 -- 172.21.15.61:0/3347335824 learned_addr learned my addr 172.21.15.61:0/3347335824 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea75d580 con 0x5599ea6ecae0 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea75d6c0 con 0x5599ea6ebde0 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599ea75d850 con 0x5599ea6ed440 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7aaf474b80 10 client.0 _ll_register_callbacks cb 0x5599ea51ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:33:00.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.731+0000 7f7a9a19c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea75d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9919a700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ecae0 0x5599ea756050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 -- 172.21.15.61:0/3347335824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 msgr2=0x5599ea75d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea75d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 -- 172.21.15.61:0/3347335824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ecae0 msgr2=0x5599ea756050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:00.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ecae0 0x5599ea756050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.732+0000 7f7a9999b700 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a90014df0 con 0x5599ea6ebde0 2023-10-18T09:33:00.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.733+0000 7f7a9999b700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a8409b9e0 tx=0x7f7a84009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d8538f9b4e6030 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:00.739 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.733+0000 7f7a7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.734+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a840b9ea0 con 0x5599ea6ebde0 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7a840b9cc0 con 0x5599ea6ebde0 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a840ce2a0 con 0x5599ea6ebde0 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a8409b4c0 con 0x5599ea6ebde0 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7aaf474b80 10 client.26483 fetch_fsmap learned FSMap version 43 2023-10-18T09:33:00.740 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5599ea487250 con 0x5599ea6ebde0 2023-10-18T09:33:00.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.735+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a840ce2a0 con 0x5599ea6ebde0 2023-10-18T09:33:00.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7a840b4b60 con 0x5599ea6ebde0 2023-10-18T09:33:00.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 10 client.26483 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:33:00.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:33:00.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 20 client.26483 populate_metadata read hostname 'smithi061' 2023-10-18T09:33:00.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea75eea0 0x5599ea7612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:00.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 4 client.26483 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:33:00.742 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599ea62ee50 con 0x5599ea75eea0 2023-10-18T09:33:00.743 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.736+0000 7f7a9919a700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea75eea0 0x5599ea7612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:00.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.737+0000 7f7a7d7fa700 10 client.26483 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:00.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.737+0000 7f7a9919a700 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea75eea0 0x5599ea7612c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eabb9ffbe07bdb server_cookie=9ed46ca31d01803a in_seq=0 out_seq=0 2023-10-18T09:33:00.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.737+0000 7f7a7d7fa700 1 -- 172.21.15.61:0/3347335824 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5599ea62ee50 con 0x5599ea75eea0 2023-10-18T09:33:00.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.737+0000 7f7a7d7fa700 10 client.26483 handle_command_reply: tid=0 2023-10-18T09:33:00.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.737+0000 7f7aaf474b80 1 client.26483 shutdown 2023-10-18T09:33:00.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.738+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 msgr2=0x5599ea755b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.738+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7a8409b9e0 tx=0x7f7a84009380 comp rx=0 tx=0).stop 2023-10-18T09:33:00.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 shutdown_connections 2023-10-18T09:33:00.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x5599ea75eea0 0x5599ea7612c0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5599ea6ed440 0x5599ea75d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5599ea6ebde0 0x5599ea755b10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 --2- 172.21.15.61:0/3347335824 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5599ea6ecae0 0x5599ea756050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.739+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 >> 172.21.15.61:0/3347335824 conn(0x5599ea657720 msgr2=0x5599ea6e49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.740+0000 7f7a7d7fa700 0 client.26483 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:00.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.740+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 shutdown_connections 2023-10-18T09:33:00.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.740+0000 7f7aaf474b80 1 -- 172.21.15.61:0/3347335824 wait complete. 2023-10-18T09:33:00.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.742+0000 7f7aaf474b80 20 client.26483 trim_cache size 0 max 16384 2023-10-18T09:33:00.748 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:33:00.748 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:33:00.748 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:33:00.748 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:33:00.748 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:33:00.749 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:33:00.749 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:33:00.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:33:00.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:33:00.749 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:33:00.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.743+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 msgr2=0x7f7a800a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.743+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 0x7f7a800a3a40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7a90014920 tx=0x7f7a9000d9f0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.743+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 msgr2=0x7f7a94198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:00.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.743+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400c010 tx=0x7f7a8400f740 comp rx=0 tx=0).stop 2023-10-18T09:33:00.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 shutdown_connections 2023-10-18T09:33:00.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7a800a1580 0x7f7a800a3a40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a94101220 0x7f7a9419fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7a940ff690 0x7f7a94198960 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 --2- 172.21.15.61:0/3474179687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a94100880 0x7f7a94198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:00.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 >> 172.21.15.61:0/3474179687 conn(0x7f7a940fac00 msgr2=0x7f7a940fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:00.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 shutdown_connections 2023-10-18T09:33:00.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:00.744+0000 7f7a9bbff700 1 -- 172.21.15.61:0/3474179687 wait complete. 2023-10-18T09:33:00.778 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:33:01.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:01 smithi170 ceph-mon[135150]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-10-18T09:33:01.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:01 smithi110 ceph-mon[134818]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-10-18T09:33:01.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:01 smithi061 ceph-mon[98573]: pgmap v792: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.9 MiB/s wr, 453 op/s 2023-10-18T09:33:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:03 smithi170 ceph-mon[135150]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2023-10-18T09:33:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:03 smithi110 ceph-mon[134818]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2023-10-18T09:33:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:03 smithi061 ceph-mon[98573]: pgmap v793: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.9 MiB/s wr, 508 op/s 2023-10-18T09:33:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 830.0 s, 469.6 tps, lat 212.922 ms stddev 231.469 2023-10-18T09:33:05.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:05 smithi170 ceph-mon[135150]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 464 op/s 2023-10-18T09:33:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:05 smithi110 ceph-mon[134818]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 464 op/s 2023-10-18T09:33:05.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:05 smithi061 ceph-mon[98573]: pgmap v794: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 464 op/s 2023-10-18T09:33:08.395 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:08 smithi170 ceph-mon[135150]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2023-10-18T09:33:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:08 smithi110 ceph-mon[134818]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2023-10-18T09:33:08.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:08 smithi061 ceph-mon[98573]: pgmap v795: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.6 MiB/s wr, 475 op/s 2023-10-18T09:33:09.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 835.0 s, 448.4 tps, lat 223.119 ms stddev 302.003 2023-10-18T09:33:09.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:09 smithi170 ceph-mon[135150]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2023-10-18T09:33:09.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:09 smithi110 ceph-mon[134818]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2023-10-18T09:33:09.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:09 smithi061 ceph-mon[98573]: pgmap v796: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 4.8 MiB/s wr, 453 op/s 2023-10-18T09:33:11.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:11 smithi170 ceph-mon[135150]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 457 op/s 2023-10-18T09:33:11.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:11 smithi110 ceph-mon[134818]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 457 op/s 2023-10-18T09:33:11.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:11 smithi061 ceph-mon[98573]: pgmap v797: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.2 MiB/s wr, 457 op/s 2023-10-18T09:33:13.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:13 smithi170 ceph-mon[135150]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2023-10-18T09:33:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:13 smithi110 ceph-mon[134818]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2023-10-18T09:33:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:13 smithi061 ceph-mon[98573]: pgmap v798: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 5.8 MiB/s wr, 508 op/s 2023-10-18T09:33:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 840.0 s, 475.2 tps, lat 218.602 ms stddev 243.552 2023-10-18T09:33:15.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:15 smithi170 ceph-mon[135150]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 466 op/s 2023-10-18T09:33:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:15 smithi110 ceph-mon[134818]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 466 op/s 2023-10-18T09:33:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:15 smithi061 ceph-mon[98573]: pgmap v799: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.0 MiB/s wr, 466 op/s 2023-10-18T09:33:18.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:18 smithi170 ceph-mon[135150]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-10-18T09:33:18.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:18 smithi110 ceph-mon[134818]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-10-18T09:33:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:18 smithi061 ceph-mon[98573]: pgmap v800: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.6 MiB/s wr, 457 op/s 2023-10-18T09:33:19.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 845.0 s, 565.0 tps, lat 170.457 ms stddev 235.508 2023-10-18T09:33:19.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:19 smithi110 ceph-mon[134818]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2023-10-18T09:33:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:19 smithi061 ceph-mon[98573]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2023-10-18T09:33:19.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:19 smithi170 ceph-mon[135150]: pgmap v801: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.9 MiB/s wr, 472 op/s 2023-10-18T09:33:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:21 smithi170 ceph-mon[135150]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-18T09:33:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:21 smithi110 ceph-mon[134818]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-18T09:33:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:21 smithi061 ceph-mon[98573]: pgmap v802: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 5.4 MiB/s wr, 491 op/s 2023-10-18T09:33:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:23 smithi170 ceph-mon[135150]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:33:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:23 smithi110 ceph-mon[134818]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:33:23.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:23 smithi061 ceph-mon[98573]: pgmap v803: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.7 MiB/s wr, 526 op/s 2023-10-18T09:33:24.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 850.0 s, 580.6 tps, lat 177.763 ms stddev 268.070 2023-10-18T09:33:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:25 smithi110 ceph-mon[134818]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.3 MiB/s wr, 510 op/s 2023-10-18T09:33:25.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:25 smithi061 ceph-mon[98573]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.3 MiB/s wr, 510 op/s 2023-10-18T09:33:25.898 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:25 smithi170 ceph-mon[135150]: pgmap v804: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.3 MiB/s wr, 510 op/s 2023-10-18T09:33:28.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:28 smithi170 ceph-mon[135150]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2023-10-18T09:33:28.464 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:28 smithi110 ceph-mon[134818]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2023-10-18T09:33:28.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:28 smithi061 ceph-mon[98573]: pgmap v805: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 5.2 MiB/s wr, 493 op/s 2023-10-18T09:33:29.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 855.0 s, 467.0 tps, lat 208.251 ms stddev 240.384 2023-10-18T09:33:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:29 smithi110 ceph-mon[134818]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.8 MiB/s wr, 532 op/s 2023-10-18T09:33:29.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:29 smithi061 ceph-mon[98573]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.8 MiB/s wr, 532 op/s 2023-10-18T09:33:29.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:29 smithi170 ceph-mon[135150]: pgmap v806: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.8 MiB/s wr, 532 op/s 2023-10-18T09:33:30.779 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:33:31.178 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 -- 172.21.15.61:0/2575086029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c101220 msgr2=0x7fa75c107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 --2- 172.21.15.61:0/2575086029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c101220 0x7fa75c107ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa754009fc0 tx=0x7fa75400b330 comp rx=0 tx=0).stop 2023-10-18T09:33:31.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 -- 172.21.15.61:0/2575086029 shutdown_connections 2023-10-18T09:33:31.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 --2- 172.21.15.61:0/2575086029 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c101220 0x7fa75c107ad0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.179 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 --2- 172.21.15.61:0/2575086029 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c0ff690 0x7fa75c0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 --2- 172.21.15.61:0/2575086029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.176+0000 7fa7617ff700 1 -- 172.21.15.61:0/2575086029 >> 172.21.15.61:0/2575086029 conn(0x7fa75c0fac00 msgr2=0x7fa75c0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.179+0000 7fa7617ff700 1 -- 172.21.15.61:0/2575086029 shutdown_connections 2023-10-18T09:33:31.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.179+0000 7fa7617ff700 1 -- 172.21.15.61:0/2575086029 wait complete. 2023-10-18T09:33:31.182 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.180+0000 7fa7617ff700 1 Processor -- start 2023-10-18T09:33:31.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 -- start start 2023-10-18T09:33:31.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c0ff690 0x7fa75c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 0x7fa75c19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1a0570 con 0x7fa75c101220 2023-10-18T09:33:31.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06a210 con 0x7fa75c0ff690 2023-10-18T09:33:31.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.181+0000 7fa7617ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c06a3a0 con 0x7fa75c100880 2023-10-18T09:33:31.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.182+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.182+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46924/0 (socket says 172.21.15.61:46924) 2023-10-18T09:33:31.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.182+0000 7fa75a7fc700 1 -- 172.21.15.61:0/666282041 learned_addr learned my addr 172.21.15.61:0/666282041 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:31.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 -- 172.21.15.61:0/666282041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c0ff690 msgr2=0x7fa75c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.187 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c0ff690 0x7fa75c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 -- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 msgr2=0x7fa75c19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 0x7fa75c19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 0x7fa75c19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.188 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 -- 172.21.15.61:0/666282041 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754009c40 con 0x7fa75c100880 2023-10-18T09:33:31.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 0x7fa75c19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:33:31.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.183+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9c0 tx=0x7fa75000ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.189 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.184+0000 7fa743fff700 1 -- 172.21.15.61:0/666282041 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75000cbd0 con 0x7fa75c100880 2023-10-18T09:33:31.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.184+0000 7fa743fff700 1 -- 172.21.15.61:0/666282041 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75000cd30 con 0x7fa75c100880 2023-10-18T09:33:31.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.184+0000 7fa743fff700 1 -- 172.21.15.61:0/666282041 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7500055a0 con 0x7fa75c100880 2023-10-18T09:33:31.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.184+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c06a680 con 0x7fa75c100880 2023-10-18T09:33:31.190 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.184+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c06ab40 con 0x7fa75c100880 2023-10-18T09:33:31.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.192+0000 7fa743fff700 1 -- 172.21.15.61:0/666282041 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fa750004750 con 0x7fa75c100880 2023-10-18T09:33:31.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.193+0000 7fa77509eb80 1 Processor -- start 2023-10-18T09:33:31.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.194+0000 7fa77509eb80 1 -- start start 2023-10-18T09:33:31.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.194+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 0x5559040fe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 0x5559040fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa77509eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559040716e0 con 0x5559040ffc30 2023-10-18T09:33:31.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa77509eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904057c60 con 0x5559040ff2d0 2023-10-18T09:33:31.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa77509eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904055580 con 0x5559040fe5d0 2023-10-18T09:33:31.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa75b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa75affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 0x5559040fe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa75b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50964/0 (socket says 172.21.15.61:50964) 2023-10-18T09:33:31.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.195+0000 7fa75b7fe700 1 -- 172.21.15.61:0/3892099350 learned_addr learned my addr 172.21.15.61:0/3892099350 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:31.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.196+0000 7fa75b7fe700 1 -- 172.21.15.61:0/3892099350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 msgr2=0x5559040fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 0x5559040fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 0x5559040fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75b7fe700 1 -- 172.21.15.61:0/3892099350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 msgr2=0x5559040fe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 0x5559040fe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75b7fe700 1 -- 172.21.15.61:0/3892099350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55590400e3f0 con 0x5559040ff2d0 2023-10-18T09:33:31.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.197+0000 7fa75affd700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 0x5559040fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:33:31.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 0x5559040fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:33:31.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa75400b6c0 tx=0x7fa754009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ebb54ec9e9defd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.202 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa743fff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 0x7fa7440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa743fff700 1 -- 172.21.15.61:0/666282041 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa750014070 con 0x7fa75c100880 2023-10-18T09:33:31.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa7417fa700 1 -- 172.21.15.61:0/3892099350 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400e040 con 0x5559040ff2d0 2023-10-18T09:33:31.203 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa7417fa700 1 -- 172.21.15.61:0/3892099350 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa754007420 con 0x5559040ff2d0 2023-10-18T09:33:31.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.198+0000 7fa7417fa700 1 -- 172.21.15.61:0/3892099350 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400bc50 con 0x5559040ff2d0 2023-10-18T09:33:31.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.199+0000 7fa75affd700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 0x7fa7440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.200+0000 7fa77509eb80 1 -- 172.21.15.61:0/3892099350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 msgr2=0x5559040ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.200+0000 7fa77509eb80 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa75400b6c0 tx=0x7fa754009bc0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.204 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.201+0000 7fa75affd700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 0x7fa7440a39c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005d90 tx=0x7fa74c00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.201+0000 7fa77509eb80 1 -- 172.21.15.61:0/3892099350 shutdown_connections 2023-10-18T09:33:31.205 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.201+0000 7fa77509eb80 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040fe5d0 0x5559040fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.201+0000 7fa77509eb80 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559040ff6f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.202+0000 7fa77509eb80 1 --2- 172.21.15.61:0/3892099350 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ffc30 0x5559040fe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.206 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.202+0000 7fa77509eb80 1 -- 172.21.15.61:0/3892099350 >> 172.21.15.61:0/3892099350 conn(0x555904072720 msgr2=0x5559040f8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.205+0000 7fa77509eb80 1 -- 172.21.15.61:0/3892099350 shutdown_connections 2023-10-18T09:33:31.207 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.205+0000 7fa77509eb80 1 -- 172.21.15.61:0/3892099350 wait complete. 2023-10-18T09:33:31.208 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.206+0000 7fa77509eb80 1 -- start start 2023-10-18T09:33:31.209 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.207+0000 7fa77509eb80 1 Processor -- start 2023-10-18T09:33:31.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.210 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904117830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904050e70 con 0x5559040fe5d0 2023-10-18T09:33:31.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555903fed710 con 0x5559040ff2d0 2023-10-18T09:33:31.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555903fed3b0 con 0x5559040ffc30 2023-10-18T09:33:31.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa77509eb80 10 client.0 _ll_register_callbacks cb 0x555903f37a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:33:31.212 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50976/0 (socket says 172.21.15.61:50976) 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa75a7fc700 1 -- 172.21.15.61:0/177365283 learned_addr learned my addr 172.21.15.61:0/177365283 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.208+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904117830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75affd700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 -- 172.21.15.61:0/177365283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 msgr2=0x555904117830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.213 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904117830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 -- 172.21.15.61:0/177365283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 msgr2=0x5559041102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041102b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 -- 172.21.15.61:0/177365283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c108420 con 0x5559040ff2d0 2023-10-18T09:33:31.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75affd700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:33:31.214 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa750096050 tx=0x7fa750009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ce22c44964061a server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.215 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.209+0000 7fa73eff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:33:31.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.210+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75009e420 con 0x5559040ff2d0 2023-10-18T09:33:31.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.210+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa7500a2660 con 0x5559040ff2d0 2023-10-18T09:33:31.216 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.210+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555904055ef0 con 0x5559040ff2d0 2023-10-18T09:33:31.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.210+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa75009e130 con 0x5559040ff2d0 2023-10-18T09:33:31.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 10 client.26492 fetch_fsmap learned FSMap version 43 2023-10-18T09:33:31.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555903e7bde0 con 0x5559040ff2d0 2023-10-18T09:33:31.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555904055ef0 con 0x5559040ff2d0 2023-10-18T09:33:31.217 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa7500a21d0 con 0x5559040ff2d0 2023-10-18T09:33:31.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 10 client.26492 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:33:31.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:33:31.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 20 client.26492 populate_metadata read hostname 'smithi061' 2023-10-18T09:33:31.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55590411c0a0 0x55590411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.218 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 4 client.26492 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:33:31.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.211+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559040cd8b0 con 0x55590411c0a0 2023-10-18T09:33:31.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.212+0000 7fa75affd700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55590411c0a0 0x55590411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.213+0000 7fa75affd700 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55590411c0a0 0x55590411e4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd29dbbf9992ea3 server_cookie=e83c29846aa68e27 in_seq=0 out_seq=0 2023-10-18T09:33:31.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.213+0000 7fa73eff5700 10 client.26492 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:31.219 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.214+0000 7fa73eff5700 1 -- 172.21.15.61:0/177365283 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5559040cd8b0 con 0x55590411c0a0 2023-10-18T09:33:31.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.214+0000 7fa73eff5700 10 client.26492 handle_command_reply: tid=0 2023-10-18T09:33:31.220 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.214+0000 7fa77509eb80 1 client.26492 shutdown 2023-10-18T09:33:31.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 msgr2=0x5559041107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa750096050 tx=0x7fa750009740 comp rx=0 tx=0).stop 2023-10-18T09:33:31.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 shutdown_connections 2023-10-18T09:33:31.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55590411c0a0 0x55590411e4c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.221 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904117830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 --2- 172.21.15.61:0/177365283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041107f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa73eff5700 0 client.26492 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.216+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 >> 172.21.15.61:0/177365283 conn(0x555904072720 msgr2=0x5559040fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.218+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 shutdown_connections 2023-10-18T09:33:31.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.218+0000 7fa77509eb80 1 -- 172.21.15.61:0/177365283 wait complete. 2023-10-18T09:33:31.223 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.219+0000 7fa77509eb80 20 client.26492 trim_cache size 0 max 16384 2023-10-18T09:33:31.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.264+0000 7fa77509eb80 1 Processor -- start 2023-10-18T09:33:31.267 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 -- start start 2023-10-18T09:33:31.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 0x5559040f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ff2d0 0x555904117750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x5559040f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555903ed1920 con 0x5559040ff2d0 2023-10-18T09:33:31.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904117c90 con 0x5559040fe5d0 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.265+0000 7fa77509eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559041189d0 con 0x5559040ffc30 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.266+0000 7fa75affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 0x5559040f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.266+0000 7fa75affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 0x5559040f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50980/0 (socket says 172.21.15.61:50980) 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.266+0000 7fa75affd700 1 -- 172.21.15.61:0/2687307419 learned_addr learned my addr 172.21.15.61:0/2687307419 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.266+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x5559040f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.266+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ff2d0 0x555904117750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 -- 172.21.15.61:0/2687307419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 msgr2=0x5559040f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 0x5559040f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 -- 172.21.15.61:0/2687307419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ff2d0 msgr2=0x555904117750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ff2d0 0x555904117750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.272 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 -- 172.21.15.61:0/2687307419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555903e64c60 con 0x5559040ffc30 2023-10-18T09:33:31.273 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.267+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x5559040f8b20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5559040f9f60 tx=0x7fa754009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58f5acbb6ea1eee server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.271+0000 7fa741ffb700 1 -- 172.21.15.61:0/2687307419 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75400e040 con 0x5559040ffc30 2023-10-18T09:33:31.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.271+0000 7fa741ffb700 1 -- 172.21.15.61:0/2687307419 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa754007650 con 0x5559040ffc30 2023-10-18T09:33:31.274 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.271+0000 7fa741ffb700 1 -- 172.21.15.61:0/2687307419 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75401d6d0 con 0x5559040ffc30 2023-10-18T09:33:31.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.272+0000 7fa77509eb80 1 -- 172.21.15.61:0/2687307419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 msgr2=0x5559040f8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.275 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.272+0000 7fa77509eb80 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x5559040f8b20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5559040f9f60 tx=0x7fa754009920 comp rx=0 tx=0).stop 2023-10-18T09:33:31.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 -- 172.21.15.61:0/2687307419 shutdown_connections 2023-10-18T09:33:31.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x5559040f8b20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.279 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040fe5d0 0x5559040f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 --2- 172.21.15.61:0/2687307419 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040ff2d0 0x555904117750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 -- 172.21.15.61:0/2687307419 >> 172.21.15.61:0/2687307419 conn(0x555904072720 msgr2=0x5559040fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.280 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.277+0000 7fa77509eb80 1 -- 172.21.15.61:0/2687307419 shutdown_connections 2023-10-18T09:33:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.278+0000 7fa77509eb80 1 -- 172.21.15.61:0/2687307419 wait complete. 2023-10-18T09:33:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.278+0000 7fa77509eb80 1 -- start start 2023-10-18T09:33:31.281 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.278+0000 7fa77509eb80 1 Processor -- start 2023-10-18T09:33:31.283 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.279+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa77509eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904115f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa75affd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.284 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa75a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:50996/0 (socket says 172.21.15.61:50996) 2023-10-18T09:33:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa75a7fc700 1 -- 172.21.15.61:0/1798339526 learned_addr learned my addr 172.21.15.61:0/1798339526 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:33:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904117b10 con 0x5559040fe5d0 2023-10-18T09:33:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904116470 con 0x5559040ff2d0 2023-10-18T09:33:31.285 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.280+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x555904116630 con 0x5559040ffc30 2023-10-18T09:33:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa77509eb80 10 client.0 _ll_register_callbacks cb 0x555903f37a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:33:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 -- 172.21.15.61:0/1798339526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 msgr2=0x555904115f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904115f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 -- 172.21.15.61:0/1798339526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 msgr2=0x5559041172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:33:31.286 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55590400e3f0 con 0x5559040ff2d0 2023-10-18T09:33:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75b7fe700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:33:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75affd700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:33:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa75a7fc700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa7500a2130 tx=0x7fa750096740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d74ce3b2953cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:33:31.287 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.281+0000 7fa73eff5700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:33:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.282+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7500a2660 con 0x5559040ff2d0 2023-10-18T09:33:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.282+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa7500b1030 con 0x5559040ff2d0 2023-10-18T09:33:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.282+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa754007420 con 0x5559040ff2d0 2023-10-18T09:33:31.289 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.284+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa75009e420 con 0x5559040ff2d0 2023-10-18T09:33:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.284+0000 7fa77509eb80 10 client.26498 fetch_fsmap learned FSMap version 43 2023-10-18T09:33:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.284+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555903ea2250 con 0x5559040ff2d0 2023-10-18T09:33:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa754007420 con 0x5559040ff2d0 2023-10-18T09:33:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa750099230 con 0x5559040ff2d0 2023-10-18T09:33:31.290 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 10 client.26498 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 10 client.26498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 20 client.26498 populate_metadata read hostname 'smithi061' 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555904171080 0x555904173470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 4 client.26498 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555904049e50 con 0x555904171080 2023-10-18T09:33:31.291 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.286+0000 7fa75affd700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555904171080 0x555904173470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:33:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.287+0000 7fa75affd700 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555904171080 0x555904173470 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454d7b2a2e6d65d3 server_cookie=e86c7fdfb4d015a6 in_seq=0 out_seq=0 2023-10-18T09:33:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.287+0000 7fa73eff5700 10 client.26498 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa73eff5700 1 -- 172.21.15.61:0/1798339526 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555904049e50 con 0x555904171080 2023-10-18T09:33:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa73eff5700 10 client.26498 handle_command_reply: tid=0 2023-10-18T09:33:31.293 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 client.26498 shutdown 2023-10-18T09:33:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 msgr2=0x5559041159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa7500a2130 tx=0x7fa750096740 comp rx=0 tx=0).stop 2023-10-18T09:33:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 shutdown_connections 2023-10-18T09:33:31.294 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x555904171080 0x555904173470 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5559040ffc30 0x555904115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x5559040fe5d0 0x5559041172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 --2- 172.21.15.61:0/1798339526 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559040ff2d0 0x5559041159f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 >> 172.21.15.61:0/1798339526 conn(0x555904072720 msgr2=0x5559040f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa73eff5700 0 client.26498 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:33:31.295 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 shutdown_connections 2023-10-18T09:33:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.288+0000 7fa77509eb80 1 -- 172.21.15.61:0/1798339526 wait complete. 2023-10-18T09:33:31.296 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.289+0000 7fa77509eb80 20 client.26498 trim_cache size 0 max 16384 2023-10-18T09:33:31.296 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:33:31.296 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:33:31.297 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:33:31.297 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:33:31.297 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:33:31.297 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:33:31.297 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:33:31.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:33:31.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:33:31.298 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:33:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.289+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 msgr2=0x7fa7440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.289+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 0x7fa7440a39c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005d90 tx=0x7fa74c00b2e0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.299 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.289+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 msgr2=0x7fa75c198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:33:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.289+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa75000e9c0 tx=0x7fa75000ed80 comp rx=0 tx=0).stop 2023-10-18T09:33:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 shutdown_connections 2023-10-18T09:33:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fa7440a1500 0x7fa7440a39c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fa75c101220 0x7fa75c19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.300 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa75c0ff690 0x7fa75c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 --2- 172.21.15.61:0/666282041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa75c100880 0x7fa75c198f80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:33:31.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 >> 172.21.15.61:0/666282041 conn(0x7fa75c0fac00 msgr2=0x7fa75c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:33:31.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 shutdown_connections 2023-10-18T09:33:31.301 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:33:31.290+0000 7fa7617ff700 1 -- 172.21.15.61:0/666282041 wait complete. 2023-10-18T09:33:31.327 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:33:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:31 smithi170 ceph-mon[135150]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-18T09:33:31.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:31 smithi110 ceph-mon[134818]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-18T09:33:31.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:31 smithi061 ceph-mon[98573]: pgmap v807: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 5.7 MiB/s wr, 528 op/s 2023-10-18T09:33:33.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:33 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:33:33.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:33 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:33:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:33 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:33:34.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 860.0 s, 420.8 tps, lat 238.792 ms stddev 335.985 2023-10-18T09:33:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:34 smithi061 ceph-mon[98573]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2023-10-18T09:33:34.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:34 smithi170 ceph-mon[135150]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2023-10-18T09:33:34.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:34 smithi110 ceph-mon[134818]: pgmap v808: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.5 MiB/s wr, 539 op/s 2023-10-18T09:33:35.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:33:35.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:33:35.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:33:35.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:33:35.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:33:35.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:33:35.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:33:35.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:33:35.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:33:36.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:36 smithi170 ceph-mon[135150]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2023-10-18T09:33:36.722 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:36 smithi110 ceph-mon[134818]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2023-10-18T09:33:36.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:36 smithi061 ceph-mon[98573]: pgmap v809: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 5.1 MiB/s wr, 531 op/s 2023-10-18T09:33:38.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:38 smithi170 ceph-mon[135150]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2023-10-18T09:33:38.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:38 smithi110 ceph-mon[134818]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2023-10-18T09:33:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:38 smithi061 ceph-mon[98573]: pgmap v810: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2023-10-18T09:33:39.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 865.0 s, 570.6 tps, lat 174.050 ms stddev 214.519 2023-10-18T09:33:39.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:39 smithi170 ceph-mon[135150]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:33:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:39 smithi110 ceph-mon[134818]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:33:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:39 smithi061 ceph-mon[98573]: pgmap v811: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 5.0 MiB/s wr, 516 op/s 2023-10-18T09:33:41.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:41 smithi170 ceph-mon[135150]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 485 op/s 2023-10-18T09:33:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:41 smithi110 ceph-mon[134818]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 485 op/s 2023-10-18T09:33:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:41 smithi061 ceph-mon[98573]: pgmap v812: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.2 MiB/s wr, 485 op/s 2023-10-18T09:33:43.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:43 smithi170 ceph-mon[135150]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2023-10-18T09:33:43.716 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:43 smithi110 ceph-mon[134818]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2023-10-18T09:33:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:43 smithi061 ceph-mon[98573]: pgmap v813: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 4.8 MiB/s wr, 492 op/s 2023-10-18T09:33:44.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 870.0 s, 448.2 tps, lat 219.622 ms stddev 259.328 2023-10-18T09:33:45.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:45 smithi170 ceph-mon[135150]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2023-10-18T09:33:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:45 smithi110 ceph-mon[134818]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2023-10-18T09:33:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:45 smithi061 ceph-mon[98573]: pgmap v814: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 515 op/s 2023-10-18T09:33:48.393 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:48 smithi170 ceph-mon[135150]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-10-18T09:33:48.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:48 smithi110 ceph-mon[134818]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-10-18T09:33:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:48 smithi061 ceph-mon[98573]: pgmap v815: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 4.6 MiB/s wr, 466 op/s 2023-10-18T09:33:49.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 875.0 s, 517.8 tps, lat 196.998 ms stddev 224.271 2023-10-18T09:33:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:49 smithi170 ceph-mon[135150]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.1 MiB/s wr, 515 op/s 2023-10-18T09:33:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:49 smithi110 ceph-mon[134818]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.1 MiB/s wr, 515 op/s 2023-10-18T09:33:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:49 smithi061 ceph-mon[98573]: pgmap v816: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.1 MiB/s wr, 515 op/s 2023-10-18T09:33:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:51 smithi170 ceph-mon[135150]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2023-10-18T09:33:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:51 smithi110 ceph-mon[134818]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2023-10-18T09:33:51.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:51 smithi061 ceph-mon[98573]: pgmap v817: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 5.4 MiB/s wr, 509 op/s 2023-10-18T09:33:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:53 smithi110 ceph-mon[134818]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:33:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:53 smithi061 ceph-mon[98573]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:33:53.901 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:53 smithi170 ceph-mon[135150]: pgmap v818: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:33:54.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 880.0 s, 478.7 tps, lat 210.313 ms stddev 236.675 2023-10-18T09:33:55.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:55 smithi170 ceph-mon[135150]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 5.2 MiB/s wr, 529 op/s 2023-10-18T09:33:55.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:55 smithi110 ceph-mon[134818]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 5.2 MiB/s wr, 529 op/s 2023-10-18T09:33:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:55 smithi061 ceph-mon[98573]: pgmap v819: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 5.2 MiB/s wr, 529 op/s 2023-10-18T09:33:58.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:58 smithi170 ceph-mon[135150]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-10-18T09:33:58.465 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:58 smithi110 ceph-mon[134818]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-10-18T09:33:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:58 smithi061 ceph-mon[98573]: pgmap v820: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 4.4 MiB/s wr, 461 op/s 2023-10-18T09:33:59.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 885.0 s, 459.7 tps, lat 215.044 ms stddev 239.228 2023-10-18T09:33:59.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:33:59 smithi170 ceph-mon[135150]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.2 MiB/s wr, 523 op/s 2023-10-18T09:33:59.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:33:59 smithi110 ceph-mon[134818]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.2 MiB/s wr, 523 op/s 2023-10-18T09:33:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:33:59 smithi061 ceph-mon[98573]: pgmap v821: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 5.2 MiB/s wr, 523 op/s 2023-10-18T09:34:01.328 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:34:01.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:01 smithi110 ceph-mon[134818]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:34:01.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.718+0000 7fd6051ff700 1 -- 172.21.15.61:0/1621970341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 msgr2=0x7fd600108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.718+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1621970341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd600108900 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80098f0 tx=0x7fd5f8009c00 comp rx=0 tx=0).stop 2023-10-18T09:34:01.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.722+0000 7fd6051ff700 1 -- 172.21.15.61:0/1621970341 shutdown_connections 2023-10-18T09:34:01.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.722+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1621970341 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd600108900 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.722+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1621970341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd6001036a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.722+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1621970341 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 0x7fd600105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.722+0000 7fd6051ff700 1 -- 172.21.15.61:0/1621970341 >> 172.21.15.61:0/1621970341 conn(0x7fd6000fac00 msgr2=0x7fd6000fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.725+0000 7fd6051ff700 1 -- 172.21.15.61:0/1621970341 shutdown_connections 2023-10-18T09:34:01.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.726+0000 7fd6051ff700 1 -- 172.21.15.61:0/1621970341 wait complete. 2023-10-18T09:34:01.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.727+0000 7fd6051ff700 1 Processor -- start 2023-10-18T09:34:01.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.728+0000 7fd6051ff700 1 -- start start 2023-10-18T09:34:01.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.729+0000 7fd6051ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd60006b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.729+0000 7fd6051ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 0x7fd60006b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd6051ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd6051ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6000726d0 con 0x7fd600103be0 2023-10-18T09:34:01.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd6051ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd600072840 con 0x7fd600106510 2023-10-18T09:34:01.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd6051ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6001ad3d0 con 0x7fd6001012b0 2023-10-18T09:34:01.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.729+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd60006b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd5ff59e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd5ff59e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:55160/0 (socket says 172.21.15.61:55160) 2023-10-18T09:34:01.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd5ff59e700 1 -- 172.21.15.61:0/1170377652 learned_addr learned my addr 172.21.15.61:0/1170377652 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:01.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd60006b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48254/0 (socket says 172.21.15.61:48254) 2023-10-18T09:34:01.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.730+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 0x7fd60006b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 -- 172.21.15.61:0/1170377652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 msgr2=0x7fd60006b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd60006b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 -- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 msgr2=0x7fd60006b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 0x7fd60006b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 -- 172.21.15.61:0/1170377652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f8009520 con 0x7fd600106510 2023-10-18T09:34:01.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009230 tx=0x7fd5f800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5f7fff700 1 -- 172.21.15.61:0/1170377652 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f801c070 con 0x7fd600106510 2023-10-18T09:34:01.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5f7fff700 1 -- 172.21.15.61:0/1170377652 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f8005530 con 0x7fd600106510 2023-10-18T09:34:01.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd5f7fff700 1 -- 172.21.15.61:0/1170377652 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8003e00 con 0x7fd600106510 2023-10-18T09:34:01.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.731+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6001ad600 con 0x7fd600106510 2023-10-18T09:34:01.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.732+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6001ad9d0 con 0x7fd600106510 2023-10-18T09:34:01.741 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.737+0000 7fd5f7fff700 1 -- 172.21.15.61:0/1170377652 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd5f802a440 con 0x7fd600106510 2023-10-18T09:34:01.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.743+0000 7fd618ac3b80 1 Processor -- start 2023-10-18T09:34:01.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.743+0000 7fd618ac3b80 1 -- start start 2023-10-18T09:34:01.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 0x55f593bcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593b406e0 con 0x55f593bce2d0 2023-10-18T09:34:01.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593b26c60 con 0x55f593bcd5d0 2023-10-18T09:34:01.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.744+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593b24580 con 0x55f593bcec30 2023-10-18T09:34:01.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 0x55f593bcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 0x55f593bcd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48270/0 (socket says 172.21.15.61:48270) 2023-10-18T09:34:01.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:55162/0 (socket says 172.21.15.61:55162) 2023-10-18T09:34:01.749 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fed9d700 1 -- 172.21.15.61:0/261986885 learned_addr learned my addr 172.21.15.61:0/261986885 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:01.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 -- 172.21.15.61:0/261986885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 msgr2=0x55f593bcd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 0x55f593bcd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.750 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 -- 172.21.15.61:0/261986885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 msgr2=0x55f593bce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.745+0000 7fd5fe59c700 1 -- 172.21.15.61:0/261986885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f593add3f0 con 0x55f593bcd5d0 2023-10-18T09:34:01.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009d20 tx=0x7fd5f00078a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119a8f21adebd7e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5f7fff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 0x7fd5ec0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.751 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5f57fa700 1 -- 172.21.15.61:0/261986885 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000e030 con 0x55f593bcd5d0 2023-10-18T09:34:01.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5f57fa700 1 -- 172.21.15.61:0/261986885 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f00158b0 con 0x55f593bcd5d0 2023-10-18T09:34:01.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5f57fa700 1 -- 172.21.15.61:0/261986885 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00146d0 con 0x55f593bcd5d0 2023-10-18T09:34:01.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.746+0000 7fd5f7fff700 1 -- 172.21.15.61:0/1170377652 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd5f80ce2d0 con 0x7fd600106510 2023-10-18T09:34:01.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 -- 172.21.15.61:0/261986885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 msgr2=0x55f593bcda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.752 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009d20 tx=0x7fd5f00078a0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 0x7fd5ec0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 -- 172.21.15.61:0/261986885 shutdown_connections 2023-10-18T09:34:01.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bcda50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.753 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/261986885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcec30 0x55f593bcd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.747+0000 7fd618ac3b80 1 -- 172.21.15.61:0/261986885 >> 172.21.15.61:0/261986885 conn(0x55f593b41720 msgr2=0x55f593bc7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.754 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.748+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 0x7fd5ec0a3a10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005fd0 tx=0x7fd5e800c5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.749+0000 7fd618ac3b80 1 -- 172.21.15.61:0/261986885 shutdown_connections 2023-10-18T09:34:01.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.750+0000 7fd618ac3b80 1 -- 172.21.15.61:0/261986885 wait complete. 2023-10-18T09:34:01.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.750+0000 7fd618ac3b80 1 -- start start 2023-10-18T09:34:01.755 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.751+0000 7fd618ac3b80 1 Processor -- start 2023-10-18T09:34:01.756 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:01 smithi061 ceph-mon[98573]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:34:01.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.751+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.751+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593be80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.757 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.751+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.752+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593b1fe70 con 0x55f593bce2d0 2023-10-18T09:34:01.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.752+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593abc710 con 0x55f593bcec30 2023-10-18T09:34:01.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.752+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593abc3b0 con 0x55f593bcd5d0 2023-10-18T09:34:01.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.752+0000 7fd618ac3b80 10 client.0 _ll_register_callbacks cb 0x55f593a06a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:34:01.758 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.753+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.753+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48284/0 (socket says 172.21.15.61:48284) 2023-10-18T09:34:01.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.753+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1779037581 learned_addr learned my addr 172.21.15.61:0/1779037581 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:01.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.753+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 msgr2=0x55f593bef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.753+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.759 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.754+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1779037581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 msgr2=0x55f593be80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.754+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593be80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.754+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1779037581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f0009a00 con 0x55f593bcd5d0 2023-10-18T09:34:01.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.754+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801ce40 tx=0x7fd5e8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b64dcda682ec63 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.754+0000 7fd5e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-18T09:34:01.760 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.755+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e801adb0 con 0x55f593bcd5d0 2023-10-18T09:34:01.761 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.755+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd5e801c020 con 0x55f593bcd5d0 2023-10-18T09:34:01.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.755+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f593b24ef0 con 0x55f593bcd5d0 2023-10-18T09:34:01.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.756+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e801c1e0 con 0x55f593bcd5d0 2023-10-18T09:34:01.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.756+0000 7fd618ac3b80 10 client.26413 fetch_fsmap learned FSMap version 43 2023-10-18T09:34:01.762 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.756+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55f59394ade0 con 0x55f593bcd5d0 2023-10-18T09:34:01.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.756+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f593b24ef0 con 0x55f593bcd5d0 2023-10-18T09:34:01.763 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.758+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd5e802b050 con 0x55f593bcd5d0 2023-10-18T09:34:01.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 10 client.26413 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:34:01.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:34:01.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 20 client.26413 populate_metadata read hostname 'smithi061' 2023-10-18T09:34:01.764 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593bf39e0 0x55f593bf5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 4 client.26413 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:34:01.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f593b9c8b0 con 0x55f593bf39e0 2023-10-18T09:34:01.765 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.761+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593bf39e0 0x55f593bf5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.762+0000 7fd5e67fc700 10 client.26413 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:01.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.762+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593bf39e0 0x55f593bf5e40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c959e497369db8 server_cookie=e12e8ff5eb23f694 in_seq=0 out_seq=0 2023-10-18T09:34:01.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.763+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1779037581 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f593b9c8b0 con 0x55f593bf39e0 2023-10-18T09:34:01.766 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.763+0000 7fd5e67fc700 10 client.26413 handle_command_reply: tid=0 2023-10-18T09:34:01.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.763+0000 7fd618ac3b80 1 client.26413 shutdown 2023-10-18T09:34:01.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.763+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 msgr2=0x55f593be7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.767 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.763+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801ce40 tx=0x7fd5e8004750 comp rx=0 tx=0).stop 2023-10-18T09:34:01.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 shutdown_connections 2023-10-18T09:34:01.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593bf39e0 0x55f593bf5e40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.768 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593be7b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd5e67fc700 0 client.26413 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1779037581 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593be80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.769 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 >> 172.21.15.61:0/1779037581 conn(0x55f593b41720 msgr2=0x55f593bcc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 shutdown_connections 2023-10-18T09:34:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1779037581 wait complete. 2023-10-18T09:34:01.770 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.764+0000 7fd618ac3b80 20 client.26413 trim_cache size 0 max 16384 2023-10-18T09:34:01.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.799+0000 7fd618ac3b80 1 Processor -- start 2023-10-18T09:34:01.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 -- start start 2023-10-18T09:34:01.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bef070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bd4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5939a0920 con 0x55f593bce2d0 2023-10-18T09:34:01.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593bcd090 con 0x55f593bcec30 2023-10-18T09:34:01.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.800+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593be8800 con 0x55f593bcd5d0 2023-10-18T09:34:01.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.801+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.805 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.801+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:48290/0 (socket says 172.21.15.61:48290) 2023-10-18T09:34:01.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.801+0000 7fd5fed9d700 1 -- 172.21.15.61:0/690977686 learned_addr learned my addr 172.21.15.61:0/690977686 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:01.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.801+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bd4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 -- 172.21.15.61:0/690977686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 msgr2=0x55f593bc1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.806 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 -- 172.21.15.61:0/690977686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 msgr2=0x55f593bef070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 -- 172.21.15.61:0/690977686 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f593933c60 con 0x55f593bcec30 2023-10-18T09:34:01.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:34:01.807 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5ff59e700 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bd4b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80cbad0 tx=0x7fd5f80976f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ee34c987e0b19e server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.802+0000 7fd5f5ffb700 1 -- 172.21.15.61:0/690977686 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f809ae70 con 0x55f593bcec30 2023-10-18T09:34:01.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.803+0000 7fd5f5ffb700 1 -- 172.21.15.61:0/690977686 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f809ac90 con 0x55f593bcec30 2023-10-18T09:34:01.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.803+0000 7fd5f5ffb700 1 -- 172.21.15.61:0/690977686 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80a3630 con 0x55f593bcec30 2023-10-18T09:34:01.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.805+0000 7fd618ac3b80 1 -- 172.21.15.61:0/690977686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 msgr2=0x55f593bd4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.805+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bd4b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80cbad0 tx=0x7fd5f80976f0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 -- 172.21.15.61:0/690977686 shutdown_connections 2023-10-18T09:34:01.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcec30 0x55f593bd4b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bcd5d0 0x55f593bc1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.811 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/690977686 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bce2d0 0x55f593bef070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 -- 172.21.15.61:0/690977686 >> 172.21.15.61:0/690977686 conn(0x55f593b41720 msgr2=0x55f593bc7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 -- 172.21.15.61:0/690977686 shutdown_connections 2023-10-18T09:34:01.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.806+0000 7fd618ac3b80 1 -- 172.21.15.61:0/690977686 wait complete. 2023-10-18T09:34:01.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.807+0000 7fd618ac3b80 1 -- start start 2023-10-18T09:34:01.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.807+0000 7fd618ac3b80 1 Processor -- start 2023-10-18T09:34:01.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 0x55f593bed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bcec30 0x55f593bedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593bee430 con 0x55f593bcec30 2023-10-18T09:34:01.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593bee5a0 con 0x55f593bcd5d0 2023-10-18T09:34:01.815 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f593c47900 con 0x55f593bce2d0 2023-10-18T09:34:01.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.809+0000 7fd618ac3b80 10 client.0 _ll_register_callbacks cb 0x55f593a06a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:34:01.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:55186/0 (socket says 172.21.15.61:55186) 2023-10-18T09:34:01.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1129526082 learned_addr learned my addr 172.21.15.61:0/1129526082 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:01.816 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 0x55f593bed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1129526082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 msgr2=0x55f593bed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 0x55f593bed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1129526082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bcec30 msgr2=0x55f593bedcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:01.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bcec30 0x55f593bedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.817 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 -- 172.21.15.61:0/1129526082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f59394ade0 con 0x55f593bcd5d0 2023-10-18T09:34:01.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 0x55f593bed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:34:01.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5fed9d700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8016040 tx=0x7fd5e802ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85bc9f012de5ce03 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:01.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.810+0000 7fd5e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:34:01.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.811+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e801c1e0 con 0x55f593bcd5d0 2023-10-18T09:34:01.818 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.811+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd5e802c780 con 0x55f593bcd5d0 2023-10-18T09:34:01.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.811+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5f00158b0 con 0x55f593bcd5d0 2023-10-18T09:34:01.819 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.811+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e8011ab0 con 0x55f593bcd5d0 2023-10-18T09:34:01.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.812+0000 7fd618ac3b80 10 client.26513 fetch_fsmap learned FSMap version 43 2023-10-18T09:34:01.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.812+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fd6001ad9d0 con 0x55f593bcd5d0 2023-10-18T09:34:01.820 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5f00158b0 con 0x55f593bcd5d0 2023-10-18T09:34:01.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 43) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd5e802b050 con 0x55f593bcd5d0 2023-10-18T09:34:01.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 10 client.26513 fetch_fsmap finished waiting for FSMap version 43 2023-10-18T09:34:01.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:34:01.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 20 client.26513 populate_metadata read hostname 'smithi061' 2023-10-18T09:34:01.821 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593c49330 0x55f593c4b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:01.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 4 client.26513 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:34:01.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f593b18e50 con 0x55f593c49330 2023-10-18T09:34:01.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.813+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593c49330 0x55f593c4b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:01.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.817+0000 7fd5fe59c700 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593c49330 0x55f593c4b750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b549872f87712960 server_cookie=dfc7bc5fe52509c8 in_seq=0 out_seq=0 2023-10-18T09:34:01.822 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.817+0000 7fd5e67fc700 10 client.26513 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.847+0000 7fd5e67fc700 1 -- 172.21.15.61:0/1129526082 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f593b18e50 con 0x55f593c49330 2023-10-18T09:34:01.849 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.847+0000 7fd5e67fc700 10 client.26513 handle_command_reply: tid=0 2023-10-18T09:34:01.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.847+0000 7fd618ac3b80 1 client.26513 shutdown 2023-10-18T09:34:01.850 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 msgr2=0x55f593bef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8016040 tx=0x7fd5e802ef20 comp rx=0 tx=0).stop 2023-10-18T09:34:01.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 shutdown_connections 2023-10-18T09:34:01.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55f593c49330 0x55f593c4b750 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.851 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55f593bcec30 0x55f593bedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f593bcd5d0 0x55f593bef030 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd5e67fc700 0 client.26513 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 --2- 172.21.15.61:0/1129526082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f593bce2d0 0x55f593bed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 >> 172.21.15.61:0/1129526082 conn(0x55f593b41720 msgr2=0x55f593bc14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.852 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 shutdown_connections 2023-10-18T09:34:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.848+0000 7fd618ac3b80 1 -- 172.21.15.61:0/1129526082 wait complete. 2023-10-18T09:34:01.853 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.850+0000 7fd618ac3b80 20 client.26513 trim_cache size 0 max 16384 2023-10-18T09:34:01.853 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:34:01.854 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:34:01.854 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:34:01.854 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:34:01.854 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:34:01.855 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:34:01.855 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:34:01.855 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:34:01.855 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:34:01.855 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:34:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 msgr2=0x7fd5ec0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 0x7fd5ec0a3a10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005fd0 tx=0x7fd5e800c5b0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 msgr2=0x7fd6001ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:01.856 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009230 tx=0x7fd5f800b8f0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 shutdown_connections 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd5ec0a1550 0x7fd5ec0a3a10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd600106510 0x7fd6001ace90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd6001012b0 0x7fd60006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 --2- 172.21.15.61:0/1170377652 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd600103be0 0x7fd60006b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:01.857 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 >> 172.21.15.61:0/1170377652 conn(0x7fd6000fac00 msgr2=0x7fd600101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:01.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 shutdown_connections 2023-10-18T09:34:01.858 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:01.851+0000 7fd6051ff700 1 -- 172.21.15.61:0/1170377652 wait complete. 2023-10-18T09:34:01.888 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:34:01.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:01 smithi170 ceph-mon[135150]: pgmap v822: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.7 MiB/s wr, 476 op/s 2023-10-18T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:03 smithi110 ceph-mon[134818]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2023-10-18T09:34:03.755 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:03 smithi061 ceph-mon[98573]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2023-10-18T09:34:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:03 smithi170 ceph-mon[135150]: pgmap v823: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.6 MiB/s wr, 505 op/s 2023-10-18T09:34:04.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 890.0 s, 503.6 tps, lat 202.456 ms stddev 209.286 2023-10-18T09:34:05.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:05 smithi170 ceph-mon[135150]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2023-10-18T09:34:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:05 smithi110 ceph-mon[134818]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2023-10-18T09:34:05.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:05 smithi061 ceph-mon[98573]: pgmap v824: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 5.0 MiB/s wr, 528 op/s 2023-10-18T09:34:08.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:08 smithi170 ceph-mon[135150]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-10-18T09:34:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:08 smithi110 ceph-mon[134818]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-10-18T09:34:08.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:08 smithi061 ceph-mon[98573]: pgmap v825: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 4.2 MiB/s wr, 465 op/s 2023-10-18T09:34:09.339 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 895.0 s, 397.2 tps, lat 238.360 ms stddev 244.794 2023-10-18T09:34:09.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:09 smithi170 ceph-mon[135150]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2023-10-18T09:34:09.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:09 smithi110 ceph-mon[134818]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2023-10-18T09:34:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:09 smithi061 ceph-mon[98573]: pgmap v826: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 5.1 MiB/s wr, 525 op/s 2023-10-18T09:34:11.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:11 smithi170 ceph-mon[135150]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.3 MiB/s wr, 472 op/s 2023-10-18T09:34:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:11 smithi110 ceph-mon[134818]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.3 MiB/s wr, 472 op/s 2023-10-18T09:34:11.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:11 smithi061 ceph-mon[98573]: pgmap v827: 129 pgs: 129 active+clean; 1.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.3 MiB/s wr, 472 op/s 2023-10-18T09:34:13.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:13 smithi110 ceph-mon[134818]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2023-10-18T09:34:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:13 smithi061 ceph-mon[98573]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2023-10-18T09:34:13.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:13 smithi170 ceph-mon[135150]: pgmap v828: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 4.5 MiB/s wr, 512 op/s 2023-10-18T09:34:14.338 INFO:teuthology.orchestra.run.smithi061.stderr:progress: 900.0 s, 466.8 tps, lat 221.952 ms stddev 269.966 2023-10-18T09:34:14.596 INFO:teuthology.orchestra.run.smithi061.stdout:transaction type: 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:scaling factor: 32 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:query mode: simple 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:number of clients: 100 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:number of threads: 4 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:duration: 900 s 2023-10-18T09:34:14.597 INFO:teuthology.orchestra.run.smithi061.stdout:number of transactions actually processed: 687590 2023-10-18T09:34:14.598 INFO:teuthology.orchestra.run.smithi061.stdout:latency average = 130.886 ms 2023-10-18T09:34:14.598 INFO:teuthology.orchestra.run.smithi061.stdout:latency stddev = 190.900 ms 2023-10-18T09:34:14.598 INFO:teuthology.orchestra.run.smithi061.stdout:tps = 763.769732 (including connections establishing) 2023-10-18T09:34:14.598 INFO:teuthology.orchestra.run.smithi061.stdout:tps = 763.776837 (excluding connections establishing) 2023-10-18T09:34:14.604 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-10-18T09:34:15.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:15 smithi110 ceph-mon[134818]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2023-10-18T09:34:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:15 smithi061 ceph-mon[98573]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2023-10-18T09:34:15.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:15 smithi170 ceph-mon[135150]: pgmap v829: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 4.3 MiB/s wr, 463 op/s 2023-10-18T09:34:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:18 smithi061 ceph-mon[98573]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-10-18T09:34:18.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:18 smithi170 ceph-mon[135150]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-10-18T09:34:18.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:18 smithi110 ceph-mon[134818]: pgmap v830: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.2 MiB/s wr, 452 op/s 2023-10-18T09:34:19.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:19 smithi110 ceph-mon[134818]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 621 op/s 2023-10-18T09:34:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:19 smithi061 ceph-mon[98573]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 621 op/s 2023-10-18T09:34:19.894 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:19 smithi170 ceph-mon[135150]: pgmap v831: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 5.8 MiB/s wr, 621 op/s 2023-10-18T09:34:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:21 smithi110 ceph-mon[134818]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 578 op/s 2023-10-18T09:34:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:21 smithi061 ceph-mon[98573]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 578 op/s 2023-10-18T09:34:21.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:21 smithi170 ceph-mon[135150]: pgmap v832: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.3 MiB/s wr, 578 op/s 2023-10-18T09:34:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:23 smithi110 ceph-mon[134818]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2023-10-18T09:34:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:23 smithi061 ceph-mon[98573]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2023-10-18T09:34:23.893 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:23 smithi170 ceph-mon[135150]: pgmap v833: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 6.3 MiB/s wr, 658 op/s 2023-10-18T09:34:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:25 smithi110 ceph-mon[134818]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-10-18T09:34:25.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:25 smithi061 ceph-mon[98573]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-10-18T09:34:25.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:25 smithi170 ceph-mon[135150]: pgmap v834: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-10-18T09:34:26.761 DEBUG:teuthology.orchestra.run.smithi061:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-10-18T09:34:26.990 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/: 2023-10-18T09:34:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:34:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:34:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 data 2023-10-18T09:34:26.992 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:26.992 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data: 2023-10-18T09:34:26.992 INFO:teuthology.orchestra.run.smithi061.stdout:total 31 2023-10-18T09:34:26.992 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 . 2023-10-18T09:34:26.992 INFO:teuthology.orchestra.run.smithi061.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 .. 2023-10-18T09:34:26.993 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 base 2023-10-18T09:34:26.993 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Oct 18 09:19 current_logfiles 2023-10-18T09:34:26.993 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 18 09:19 global 2023-10-18T09:34:26.993 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:19 log 2023-10-18T09:34:26.993 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_commit_ts 2023-10-18T09:34:26.994 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_dynshmem 2023-10-18T09:34:26.994 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Oct 18 09:18 pg_hba.conf 2023-10-18T09:34:26.994 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Oct 18 09:18 pg_ident.conf 2023-10-18T09:34:26.994 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 pg_logical 2023-10-18T09:34:26.994 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 pg_multixact 2023-10-18T09:34:26.995 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:19 pg_notify 2023-10-18T09:34:26.995 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_replslot 2023-10-18T09:34:26.995 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_serial 2023-10-18T09:34:26.995 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_snapshots 2023-10-18T09:34:26.996 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 pg_stat 2023-10-18T09:34:26.996 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:34 pg_stat_tmp 2023-10-18T09:34:26.996 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:34 pg_subtrans 2023-10-18T09:34:26.996 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_tblspc 2023-10-18T09:34:26.996 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 pg_twophase 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 18 09:34 pg_wal 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 pg_xact 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Oct 18 09:18 postgresql.auto.conf 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Oct 18 09:18 postgresql.conf 2023-10-18T09:34:26.997 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Oct 18 09:19 postmaster.opts 2023-10-18T09:34:26.998 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:26.998 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base: 2023-10-18T09:34:26.998 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:26.998 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 . 2023-10-18T09:34:26.998 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:26.999 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 1 2023-10-18T09:34:26.999 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 12346 2023-10-18T09:34:26.999 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 18 09:19 12347 2023-10-18T09:34:26.999 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 pgsql_tmp 2023-10-18T09:34:26.999 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.000 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/1: 2023-10-18T09:34:27.000 INFO:teuthology.orchestra.run.smithi061.stdout:total 7193 2023-10-18T09:34:27.000 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 . 2023-10-18T09:34:27.000 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 .. 2023-10-18T09:34:27.000 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:34:27.001 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 113 2023-10-18T09:34:27.001 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:34:27.001 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12182_fsm 2023-10-18T09:34:27.001 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:34:27.002 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12184 2023-10-18T09:34:27.002 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:34:27.002 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:34:27.002 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12187_fsm 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187_vm 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12189 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12191 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12192_fsm 2023-10-18T09:34:27.003 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:34:27.004 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:34:27.004 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:34:27.004 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:34:27.004 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12197_fsm 2023-10-18T09:34:27.004 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197_vm 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12199 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:34:27.005 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:34:27.006 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12206 2023-10-18T09:34:27.006 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207 2023-10-18T09:34:27.006 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12207_fsm 2023-10-18T09:34:27.006 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:34:27.006 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12216 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:18 1247 2023-10-18T09:34:27.007 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1247_fsm 2023-10-18T09:34:27.008 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1247_vm 2023-10-18T09:34:27.008 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:18 1249 2023-10-18T09:34:27.008 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1249_fsm 2023-10-18T09:34:27.008 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1249_vm 2023-10-18T09:34:27.009 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:34:27.009 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1255_fsm 2023-10-18T09:34:27.009 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:34:27.009 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:18 1259 2023-10-18T09:34:27.009 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1259_fsm 2023-10-18T09:34:27.010 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1259_vm 2023-10-18T09:34:27.010 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:34:27.010 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417_vm 2023-10-18T09:34:27.013 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418 2023-10-18T09:34:27.013 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418_vm 2023-10-18T09:34:27.013 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 175 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2187 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328 2023-10-18T09:34:27.014 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:34:27.015 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:34:27.015 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:34:27.015 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2337 2023-10-18T09:34:27.015 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:34:27.015 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2600_fsm 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2601_fsm 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:34:27.016 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2602_fsm 2023-10-18T09:34:27.017 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2602_vm 2023-10-18T09:34:27.017 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2603 2023-10-18T09:34:27.017 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2603_fsm 2023-10-18T09:34:27.017 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:34:27.017 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:34:27.018 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:34:27.018 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:34:27.018 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2605_fsm 2023-10-18T09:34:27.018 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2605_vm 2023-10-18T09:34:27.018 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606 2023-10-18T09:34:27.019 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:34:27.019 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606_vm 2023-10-18T09:34:27.019 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:34:27.019 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:34:27.019 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:34:27.020 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 18 09:18 2608 2023-10-18T09:34:27.020 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2608_fsm 2023-10-18T09:34:27.020 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2608_vm 2023-10-18T09:34:27.020 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:34:27.020 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610_fsm 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2610_vm 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:34:27.021 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611_vm 2023-10-18T09:34:27.022 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:34:27.022 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:34:27.022 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:34:27.022 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613 2023-10-18T09:34:27.022 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613_vm 2023-10-18T09:34:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:34:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:34:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:34:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:34:27.023 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2617_fsm 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:18 2618 2023-10-18T09:34:27.024 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2618_fsm 2023-10-18T09:34:27.025 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2618_vm 2023-10-18T09:34:27.025 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:18 2619 2023-10-18T09:34:27.025 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2619_fsm 2023-10-18T09:34:27.025 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2619_vm 2023-10-18T09:34:27.025 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:34:27.026 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620_vm 2023-10-18T09:34:27.026 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:34:27.026 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2651 2023-10-18T09:34:27.026 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2652 2023-10-18T09:34:27.027 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2653 2023-10-18T09:34:27.027 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2654 2023-10-18T09:34:27.027 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2655 2023-10-18T09:34:27.027 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:34:27.027 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:34:27.028 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 18 09:18 2658 2023-10-18T09:34:27.028 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2659 2023-10-18T09:34:27.028 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:34:27.028 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:34:27.028 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2662 2023-10-18T09:34:27.029 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2663 2023-10-18T09:34:27.029 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2664 2023-10-18T09:34:27.029 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2665 2023-10-18T09:34:27.029 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2666 2023-10-18T09:34:27.029 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2667 2023-10-18T09:34:27.030 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2668 2023-10-18T09:34:27.030 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:34:27.030 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:34:27.030 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:18 2673 2023-10-18T09:34:27.030 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:18 2674 2023-10-18T09:34:27.031 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:34:27.031 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2678 2023-10-18T09:34:27.031 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2679 2023-10-18T09:34:27.031 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2680 2023-10-18T09:34:27.031 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2682 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2684 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2685 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2686 2023-10-18T09:34:27.032 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:34:27.033 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:34:27.033 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:34:27.033 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2690 2023-10-18T09:34:27.033 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:18 2691 2023-10-18T09:34:27.033 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2692 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2693 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2696 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2701 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2702 2023-10-18T09:34:27.034 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2703 2023-10-18T09:34:27.035 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2704 2023-10-18T09:34:27.035 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:34:27.035 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2753_fsm 2023-10-18T09:34:27.035 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:34:27.035 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2754 2023-10-18T09:34:27.036 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2755 2023-10-18T09:34:27.036 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:34:27.036 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:34:27.036 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830 2023-10-18T09:34:27.036 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830_vm 2023-10-18T09:34:27.037 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2831 2023-10-18T09:34:27.037 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:34:27.037 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832_vm 2023-10-18T09:34:27.037 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:34:27.037 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834 2023-10-18T09:34:27.038 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:34:27.038 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:34:27.038 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836 2023-10-18T09:34:27.038 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:34:27.038 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:34:27.039 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:18 2838 2023-10-18T09:34:27.039 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:34:27.039 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:34:27.039 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:34:27.039 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840_fsm 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2841 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995_vm 2023-10-18T09:34:27.040 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:34:27.041 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079 2023-10-18T09:34:27.041 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:34:27.041 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079_vm 2023-10-18T09:34:27.041 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3080 2023-10-18T09:34:27.041 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3081 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3085 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3119 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3164 2023-10-18T09:34:27.042 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256_vm 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350_vm 2023-10-18T09:34:27.043 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3380 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3394 2023-10-18T09:34:27.044 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3394_vm 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3440 2023-10-18T09:34:27.045 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 3455 2023-10-18T09:34:27.046 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:34:27.046 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:34:27.046 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:34:27.046 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466 2023-10-18T09:34:27.046 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3467 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501_vm 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3502 2023-10-18T09:34:27.047 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3534 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3541_fsm 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541_vm 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3542 2023-10-18T09:34:27.048 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3574 2023-10-18T09:34:27.049 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3575 2023-10-18T09:34:27.049 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576 2023-10-18T09:34:27.049 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:34:27.049 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:34:27.049 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:34:27.050 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3597 2023-10-18T09:34:27.050 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:34:27.050 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:34:27.050 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:34:27.050 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:34:27.051 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:34:27.051 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:34:27.051 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:34:27.051 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:34:27.051 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601_vm 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3602_fsm 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602_vm 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3603 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3603_fsm 2023-10-18T09:34:27.052 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3603_vm 2023-10-18T09:34:27.055 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3604 2023-10-18T09:34:27.055 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:34:27.055 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3606 2023-10-18T09:34:27.055 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3607 2023-10-18T09:34:27.056 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:34:27.056 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:34:27.056 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3712 2023-10-18T09:34:27.056 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764 2023-10-18T09:34:27.056 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764_vm 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3997 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:34:27.057 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 548 2023-10-18T09:34:27.058 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 549 2023-10-18T09:34:27.058 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:34:27.058 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102_vm 2023-10-18T09:34:27.058 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:34:27.058 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104_vm 2023-10-18T09:34:27.059 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106 2023-10-18T09:34:27.059 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106_vm 2023-10-18T09:34:27.059 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6110 2023-10-18T09:34:27.059 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6111 2023-10-18T09:34:27.059 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6112 2023-10-18T09:34:27.060 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:34:27.060 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:34:27.060 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:34:27.060 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826_vm 2023-10-18T09:34:27.060 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:34:27.061 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:34:27.061 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:34:27.061 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:34:27.061 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.061 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-10-18T09:34:27.062 INFO:teuthology.orchestra.run.smithi061.stdout:total 7193 2023-10-18T09:34:27.062 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 293 Oct 18 09:18 . 2023-10-18T09:34:27.062 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 .. 2023-10-18T09:34:27.062 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:34:27.062 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 113 2023-10-18T09:34:27.063 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:34:27.063 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12182_fsm 2023-10-18T09:34:27.063 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:34:27.063 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12184 2023-10-18T09:34:27.063 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:34:27.064 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:34:27.064 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12187_fsm 2023-10-18T09:34:27.064 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187_vm 2023-10-18T09:34:27.064 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12189 2023-10-18T09:34:27.064 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12191 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12192_fsm 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:34:27.065 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:34:27.066 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12197_fsm 2023-10-18T09:34:27.066 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197_vm 2023-10-18T09:34:27.066 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12199 2023-10-18T09:34:27.066 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:34:27.066 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:34:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:34:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:34:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:34:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12206 2023-10-18T09:34:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12207_fsm 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:34:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:34:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12216 2023-10-18T09:34:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:18 1247 2023-10-18T09:34:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1247_fsm 2023-10-18T09:34:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1247_vm 2023-10-18T09:34:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:18 1249 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1249_fsm 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1249_vm 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1255_fsm 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:34:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:18 1259 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1259_fsm 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1259_vm 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417_vm 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418 2023-10-18T09:34:27.071 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1418_vm 2023-10-18T09:34:27.072 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:34:27.072 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 175 2023-10-18T09:34:27.072 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2187 2023-10-18T09:34:27.072 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:34:27.072 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2337 2023-10-18T09:34:27.073 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2600_fsm 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2601_fsm 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:34:27.074 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2602_fsm 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2602_vm 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2603 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2603_fsm 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:34:27.075 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2605_fsm 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2605_vm 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606 2023-10-18T09:34:27.076 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:34:27.077 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2606_vm 2023-10-18T09:34:27.077 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:34:27.077 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:34:27.077 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:34:27.077 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Oct 18 09:18 2608 2023-10-18T09:34:27.078 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2608_fsm 2023-10-18T09:34:27.078 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2608_vm 2023-10-18T09:34:27.078 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:34:27.078 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:34:27.078 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:34:27.079 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610 2023-10-18T09:34:27.079 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2610_fsm 2023-10-18T09:34:27.079 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2610_vm 2023-10-18T09:34:27.079 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:34:27.079 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611_vm 2023-10-18T09:34:27.080 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:34:27.080 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:34:27.080 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:34:27.080 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613 2023-10-18T09:34:27.080 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2613_vm 2023-10-18T09:34:27.081 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:34:27.081 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:34:27.081 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:34:27.081 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:34:27.081 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2617_fsm 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:18 2618 2023-10-18T09:34:27.082 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2618_fsm 2023-10-18T09:34:27.083 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2618_vm 2023-10-18T09:34:27.083 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:18 2619 2023-10-18T09:34:27.083 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2619_fsm 2023-10-18T09:34:27.083 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2619_vm 2023-10-18T09:34:27.083 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620_vm 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2651 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2652 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2653 2023-10-18T09:34:27.084 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2654 2023-10-18T09:34:27.085 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2655 2023-10-18T09:34:27.085 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:34:27.085 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:34:27.085 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Oct 18 09:18 2658 2023-10-18T09:34:27.085 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2659 2023-10-18T09:34:27.086 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:34:27.086 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:34:27.086 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2662 2023-10-18T09:34:27.086 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2663 2023-10-18T09:34:27.086 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2664 2023-10-18T09:34:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2665 2023-10-18T09:34:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2666 2023-10-18T09:34:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2667 2023-10-18T09:34:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2668 2023-10-18T09:34:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:34:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:34:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:18 2673 2023-10-18T09:34:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:18 2674 2023-10-18T09:34:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:34:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2678 2023-10-18T09:34:27.089 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2679 2023-10-18T09:34:27.089 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2680 2023-10-18T09:34:27.089 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:34:27.089 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2682 2023-10-18T09:34:27.089 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:34:27.090 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2684 2023-10-18T09:34:27.090 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2685 2023-10-18T09:34:27.090 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2686 2023-10-18T09:34:27.090 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:34:27.090 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:34:27.091 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:34:27.091 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:18 2690 2023-10-18T09:34:27.091 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:18 2691 2023-10-18T09:34:27.091 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2692 2023-10-18T09:34:27.091 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2693 2023-10-18T09:34:27.092 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2696 2023-10-18T09:34:27.092 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:34:27.092 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2701 2023-10-18T09:34:27.092 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2702 2023-10-18T09:34:27.092 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2703 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2704 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2753_fsm 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2754 2023-10-18T09:34:27.093 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2755 2023-10-18T09:34:27.094 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:34:27.094 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:34:27.094 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830 2023-10-18T09:34:27.094 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2830_vm 2023-10-18T09:34:27.094 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2831 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832_vm 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:34:27.095 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:34:27.096 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836 2023-10-18T09:34:27.096 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:34:27.096 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:34:27.096 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:18 2838 2023-10-18T09:34:27.096 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:34:27.097 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:34:27.097 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:34:27.097 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840 2023-10-18T09:34:27.097 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2840_fsm 2023-10-18T09:34:27.097 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:34:27.098 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2841 2023-10-18T09:34:27.098 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:34:27.098 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995_vm 2023-10-18T09:34:27.098 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:34:27.098 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3079_vm 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3080 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3081 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3085 2023-10-18T09:34:27.099 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118 2023-10-18T09:34:27.100 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:34:27.100 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3119 2023-10-18T09:34:27.100 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3164 2023-10-18T09:34:27.100 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256 2023-10-18T09:34:27.100 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3256_vm 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350_vm 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:34:27.101 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:34:27.102 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3380 2023-10-18T09:34:27.102 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381 2023-10-18T09:34:27.102 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:34:27.102 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3394 2023-10-18T09:34:27.102 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3394_vm 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3440 2023-10-18T09:34:27.103 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 3455 2023-10-18T09:34:27.104 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:34:27.104 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:34:27.104 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:34:27.104 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466 2023-10-18T09:34:27.104 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:34:27.105 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3467 2023-10-18T09:34:27.105 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:34:27.105 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:34:27.105 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501_vm 2023-10-18T09:34:27.105 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3502 2023-10-18T09:34:27.106 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:34:27.106 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3534 2023-10-18T09:34:27.106 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:34:27.106 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3541_fsm 2023-10-18T09:34:27.106 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541_vm 2023-10-18T09:34:27.107 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3542 2023-10-18T09:34:27.107 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3574 2023-10-18T09:34:27.107 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3575 2023-10-18T09:34:27.107 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576 2023-10-18T09:34:27.107 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:34:27.108 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:34:27.108 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:34:27.108 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3597 2023-10-18T09:34:27.108 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:34:27.108 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:34:27.109 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:34:27.109 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:34:27.109 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:34:27.109 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:34:27.109 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:34:27.110 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:34:27.110 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601_vm 2023-10-18T09:34:27.110 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:34:27.110 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3602_fsm 2023-10-18T09:34:27.110 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602_vm 2023-10-18T09:34:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3603 2023-10-18T09:34:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3603_fsm 2023-10-18T09:34:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3603_vm 2023-10-18T09:34:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3604 2023-10-18T09:34:27.111 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3606 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3607 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3712 2023-10-18T09:34:27.112 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3764_vm 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3997 2023-10-18T09:34:27.113 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:34:27.114 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 548 2023-10-18T09:34:27.114 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 549 2023-10-18T09:34:27.114 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:34:27.114 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102_vm 2023-10-18T09:34:27.114 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:34:27.115 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104_vm 2023-10-18T09:34:27.115 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106 2023-10-18T09:34:27.115 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6106_vm 2023-10-18T09:34:27.115 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6110 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6111 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6112 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:34:27.116 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826_vm 2023-10-18T09:34:27.117 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:34:27.117 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:34:27.117 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:34:27.117 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:18 PG_VERSION 2023-10-18T09:34:27.117 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.118 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-10-18T09:34:27.118 INFO:teuthology.orchestra.run.smithi061.stdout:total 550504 2023-10-18T09:34:27.118 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 308 Oct 18 09:19 . 2023-10-18T09:34:27.118 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 .. 2023-10-18T09:34:27.118 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 112 2023-10-18T09:34:27.119 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 113 2023-10-18T09:34:27.119 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Oct 18 09:18 12182 2023-10-18T09:34:27.119 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12182_fsm 2023-10-18T09:34:27.119 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12182_vm 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12184 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12186 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12187 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12187_fsm 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12187_vm 2023-10-18T09:34:27.120 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12189 2023-10-18T09:34:27.121 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12191 2023-10-18T09:34:27.121 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192 2023-10-18T09:34:27.121 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12192_fsm 2023-10-18T09:34:27.121 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12192_vm 2023-10-18T09:34:27.121 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12194 2023-10-18T09:34:27.122 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12196 2023-10-18T09:34:27.122 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12197 2023-10-18T09:34:27.122 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12197_fsm 2023-10-18T09:34:27.122 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12197_vm 2023-10-18T09:34:27.122 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 12199 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12201 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 12202_fsm 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12202_vm 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12204 2023-10-18T09:34:27.123 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12206 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12207 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 12207_fsm 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12207_vm 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12209 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 12211 2023-10-18T09:34:27.124 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12212 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 12214 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 12216 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Oct 18 09:19 1247 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1247_fsm 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 1247_vm 2023-10-18T09:34:27.125 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Oct 18 09:19 1249 2023-10-18T09:34:27.126 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1249_fsm 2023-10-18T09:34:27.126 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 1249_vm 2023-10-18T09:34:27.126 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Oct 18 09:18 1255 2023-10-18T09:34:27.126 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1255_fsm 2023-10-18T09:34:27.126 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1255_vm 2023-10-18T09:34:27.127 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Oct 18 09:34 1259 2023-10-18T09:34:27.127 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 1259_fsm 2023-10-18T09:34:27.127 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 1259_vm 2023-10-18T09:34:27.127 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 1417 2023-10-18T09:34:27.127 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1417_vm 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1418 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 1418_vm 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3358720 Oct 18 09:34 16387 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:34 16387_fsm 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:34 16387_vm 2023-10-18T09:34:27.128 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2170880 Oct 18 09:34 16393 2023-10-18T09:34:27.129 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:34 16393_fsm 2023-10-18T09:34:27.129 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:34 16393_vm 2023-10-18T09:34:27.129 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 441221120 Oct 18 09:34 16396 2023-10-18T09:34:27.129 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Oct 18 09:34 16396_fsm 2023-10-18T09:34:27.129 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:28 16396_vm 2023-10-18T09:34:27.130 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:34 16397 2023-10-18T09:34:27.130 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:34 16399 2023-10-18T09:34:27.130 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Oct 18 09:34 16401 2023-10-18T09:34:27.130 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 37085184 Oct 18 09:34 16403 2023-10-18T09:34:27.130 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:34 16403_fsm 2023-10-18T09:34:27.131 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 174 2023-10-18T09:34:27.131 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 175 2023-10-18T09:34:27.131 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2187 2023-10-18T09:34:27.131 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224 2023-10-18T09:34:27.131 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2224_vm 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2328 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2328_vm 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2336_vm 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2337 2023-10-18T09:34:27.132 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2600 2023-10-18T09:34:27.133 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2600_fsm 2023-10-18T09:34:27.133 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2600_vm 2023-10-18T09:34:27.133 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601 2023-10-18T09:34:27.133 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2601_fsm 2023-10-18T09:34:27.133 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2601_vm 2023-10-18T09:34:27.134 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2602 2023-10-18T09:34:27.134 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2602_fsm 2023-10-18T09:34:27.134 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2602_vm 2023-10-18T09:34:27.134 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2603 2023-10-18T09:34:27.134 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2603_fsm 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2603_vm 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2604_vm 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2605 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2605_fsm 2023-10-18T09:34:27.135 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2605_vm 2023-10-18T09:34:27.136 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2606 2023-10-18T09:34:27.136 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2606_fsm 2023-10-18T09:34:27.136 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2606_vm 2023-10-18T09:34:27.136 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607 2023-10-18T09:34:27.136 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2607_fsm 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2607_vm 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Oct 18 09:19 2608 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2608_fsm 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2608_vm 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Oct 18 09:18 2609 2023-10-18T09:34:27.137 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2609_fsm 2023-10-18T09:34:27.138 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2609_vm 2023-10-18T09:34:27.138 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2610 2023-10-18T09:34:27.138 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2610_fsm 2023-10-18T09:34:27.138 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2610_vm 2023-10-18T09:34:27.138 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2611 2023-10-18T09:34:27.139 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2611_vm 2023-10-18T09:34:27.139 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612 2023-10-18T09:34:27.139 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2612_fsm 2023-10-18T09:34:27.139 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2612_vm 2023-10-18T09:34:27.139 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2613 2023-10-18T09:34:27.140 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2613_vm 2023-10-18T09:34:27.140 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615 2023-10-18T09:34:27.140 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2615_fsm 2023-10-18T09:34:27.140 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2615_vm 2023-10-18T09:34:27.140 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616 2023-10-18T09:34:27.141 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2616_fsm 2023-10-18T09:34:27.141 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2616_vm 2023-10-18T09:34:27.141 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Oct 18 09:18 2617 2023-10-18T09:34:27.141 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2617_fsm 2023-10-18T09:34:27.141 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2617_vm 2023-10-18T09:34:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Oct 18 09:19 2618 2023-10-18T09:34:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2618_fsm 2023-10-18T09:34:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2618_vm 2023-10-18T09:34:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Oct 18 09:34 2619 2023-10-18T09:34:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:25 2619_fsm 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:23 2619_vm 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2620 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2620_vm 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2650 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2651 2023-10-18T09:34:27.143 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2652 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2653 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2654 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2655 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2656 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2657 2023-10-18T09:34:27.144 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Oct 18 09:19 2658 2023-10-18T09:34:27.145 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:19 2659 2023-10-18T09:34:27.145 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2660 2023-10-18T09:34:27.145 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2661 2023-10-18T09:34:27.145 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:19 2662 2023-10-18T09:34:27.145 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:19 2663 2023-10-18T09:34:27.146 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2664 2023-10-18T09:34:27.146 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2665 2023-10-18T09:34:27.146 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2666 2023-10-18T09:34:27.146 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2667 2023-10-18T09:34:27.146 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2668 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2669 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2670 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 319488 Oct 18 09:19 2673 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Oct 18 09:19 2674 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:18 2675 2023-10-18T09:34:27.147 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2678 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2679 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2680 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2681 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2682 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2683 2023-10-18T09:34:27.148 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2684 2023-10-18T09:34:27.149 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2685 2023-10-18T09:34:27.149 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2686 2023-10-18T09:34:27.149 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2687 2023-10-18T09:34:27.149 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:18 2688 2023-10-18T09:34:27.149 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Oct 18 09:18 2689 2023-10-18T09:34:27.150 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Oct 18 09:19 2690 2023-10-18T09:34:27.150 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Oct 18 09:19 2691 2023-10-18T09:34:27.150 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2692 2023-10-18T09:34:27.150 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2693 2023-10-18T09:34:27.150 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:25 2696 2023-10-18T09:34:27.151 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2699 2023-10-18T09:34:27.151 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2701 2023-10-18T09:34:27.151 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2702 2023-10-18T09:34:27.151 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2703 2023-10-18T09:34:27.151 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 2704 2023-10-18T09:34:27.152 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2753 2023-10-18T09:34:27.152 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2753_fsm 2023-10-18T09:34:27.152 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2753_vm 2023-10-18T09:34:27.152 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2754 2023-10-18T09:34:27.152 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2755 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2756 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 2757 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2830 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2830_vm 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2831 2023-10-18T09:34:27.153 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2832 2023-10-18T09:34:27.154 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2832_vm 2023-10-18T09:34:27.154 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2833 2023-10-18T09:34:27.154 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2834 2023-10-18T09:34:27.154 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2834_vm 2023-10-18T09:34:27.154 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2835 2023-10-18T09:34:27.155 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2836 2023-10-18T09:34:27.155 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2836_vm 2023-10-18T09:34:27.155 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2837 2023-10-18T09:34:27.155 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Oct 18 09:19 2838 2023-10-18T09:34:27.155 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2838_fsm 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2838_vm 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2839 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2840 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 2840_fsm 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2840_vm 2023-10-18T09:34:27.156 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2841 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2995 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 2995_vm 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2996 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3079 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3079_fsm 2023-10-18T09:34:27.157 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3079_vm 2023-10-18T09:34:27.158 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3080 2023-10-18T09:34:27.158 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3081 2023-10-18T09:34:27.158 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3085 2023-10-18T09:34:27.158 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3118 2023-10-18T09:34:27.158 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3118_vm 2023-10-18T09:34:27.159 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3119 2023-10-18T09:34:27.159 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3164 2023-10-18T09:34:27.159 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3256 2023-10-18T09:34:27.159 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3256_vm 2023-10-18T09:34:27.159 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3257 2023-10-18T09:34:27.160 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3258 2023-10-18T09:34:27.160 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3350 2023-10-18T09:34:27.160 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3350_vm 2023-10-18T09:34:27.160 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3351 2023-10-18T09:34:27.160 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3379 2023-10-18T09:34:27.161 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3380 2023-10-18T09:34:27.161 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3381 2023-10-18T09:34:27.161 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3381_vm 2023-10-18T09:34:27.161 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3394 2023-10-18T09:34:27.161 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3394_fsm 2023-10-18T09:34:27.162 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3394_vm 2023-10-18T09:34:27.162 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3395 2023-10-18T09:34:27.162 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3439 2023-10-18T09:34:27.162 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3439_vm 2023-10-18T09:34:27.162 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3440 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Oct 18 09:19 3455 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3456 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3456_fsm 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3456_vm 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3466 2023-10-18T09:34:27.163 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3466_vm 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3467 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3468 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3501 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3501_vm 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3502 2023-10-18T09:34:27.164 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3503 2023-10-18T09:34:27.165 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3534 2023-10-18T09:34:27.165 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3541 2023-10-18T09:34:27.165 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3541_fsm 2023-10-18T09:34:27.165 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3541_vm 2023-10-18T09:34:27.165 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3542 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3574 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3575 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 3576 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3576_vm 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596 2023-10-18T09:34:27.166 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3596_vm 2023-10-18T09:34:27.167 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3597 2023-10-18T09:34:27.167 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598 2023-10-18T09:34:27.167 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3598_vm 2023-10-18T09:34:27.167 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3599 2023-10-18T09:34:27.167 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3600_fsm 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3600_vm 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3601 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3601_fsm 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3601_vm 2023-10-18T09:34:27.168 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3602 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3602_fsm 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3602_vm 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3603 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:19 3603_fsm 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3603_vm 2023-10-18T09:34:27.169 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3604 2023-10-18T09:34:27.170 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3605 2023-10-18T09:34:27.170 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3606 2023-10-18T09:34:27.170 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3607 2023-10-18T09:34:27.170 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3608 2023-10-18T09:34:27.170 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Oct 18 09:18 3609 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 3712 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3764 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 3764_fsm 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3764_vm 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3766 2023-10-18T09:34:27.171 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 3767 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 3997 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 5002 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 548 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 549 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6102 2023-10-18T09:34:27.172 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6102_vm 2023-10-18T09:34:27.173 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6104 2023-10-18T09:34:27.173 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6104_vm 2023-10-18T09:34:27.173 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6106 2023-10-18T09:34:27.173 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 6106_vm 2023-10-18T09:34:27.173 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6110 2023-10-18T09:34:27.174 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6111 2023-10-18T09:34:27.174 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 6112 2023-10-18T09:34:27.174 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6113 2023-10-18T09:34:27.174 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6117 2023-10-18T09:34:27.174 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 826 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 826_vm 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 827 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 828 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:19 pg_filenode.map 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Oct 18 09:19 pg_internal.init 2023-10-18T09:34:27.175 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:19 PG_VERSION 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:19 . 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Oct 18 09:19 .. 2023-10-18T09:34:27.176 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.177 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/global: 2023-10-18T09:34:27.177 INFO:teuthology.orchestra.run.smithi061.stdout:total 574 2023-10-18T09:34:27.177 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 59 Oct 18 09:19 . 2023-10-18T09:34:27.177 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.177 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1136 2023-10-18T09:34:27.178 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1136_fsm 2023-10-18T09:34:27.178 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1136_vm 2023-10-18T09:34:27.178 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1137 2023-10-18T09:34:27.178 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1213 2023-10-18T09:34:27.178 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1213_fsm 2023-10-18T09:34:27.179 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1213_vm 2023-10-18T09:34:27.179 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1214 2023-10-18T09:34:27.179 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1214_fsm 2023-10-18T09:34:27.179 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1214_vm 2023-10-18T09:34:27.179 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1232 2023-10-18T09:34:27.180 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 1233 2023-10-18T09:34:27.180 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1260 2023-10-18T09:34:27.180 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1260_fsm 2023-10-18T09:34:27.180 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1260_vm 2023-10-18T09:34:27.180 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1261 2023-10-18T09:34:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1261_fsm 2023-10-18T09:34:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1261_vm 2023-10-18T09:34:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 1262 2023-10-18T09:34:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 1262_fsm 2023-10-18T09:34:27.181 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 1262_vm 2023-10-18T09:34:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 2396 2023-10-18T09:34:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Oct 18 09:18 2396_fsm 2023-10-18T09:34:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2396_vm 2023-10-18T09:34:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:19 2397 2023-10-18T09:34:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2671 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2672 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2676 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2677 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2694 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2695 2023-10-18T09:34:27.183 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2697 2023-10-18T09:34:27.184 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Oct 18 09:18 2698 2023-10-18T09:34:27.184 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2846 2023-10-18T09:34:27.185 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2846_vm 2023-10-18T09:34:27.186 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2847 2023-10-18T09:34:27.186 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2964 2023-10-18T09:34:27.186 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2964_vm 2023-10-18T09:34:27.186 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2965 2023-10-18T09:34:27.186 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2966 2023-10-18T09:34:27.187 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 2966_vm 2023-10-18T09:34:27.187 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 2967 2023-10-18T09:34:27.187 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3592 2023-10-18T09:34:27.187 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 3592_vm 2023-10-18T09:34:27.187 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 3593 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 4060 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 4060_vm 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 4061 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6000 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6000_vm 2023-10-18T09:34:27.188 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6001 2023-10-18T09:34:27.189 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6002 2023-10-18T09:34:27.189 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6100 2023-10-18T09:34:27.189 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 6100_vm 2023-10-18T09:34:27.189 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6114 2023-10-18T09:34:27.189 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 6115 2023-10-18T09:34:27.190 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:34 pg_control 2023-10-18T09:34:27.190 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Oct 18 09:18 pg_filenode.map 2023-10-18T09:34:27.190 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Oct 18 09:19 pg_internal.init 2023-10-18T09:34:27.190 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.190 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/log: 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout:total 1 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:19 . 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 805 Oct 18 09:34 postgresql-Wed.log 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.191 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-10-18T09:34:27.192 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.192 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.192 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.192 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.192 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-10-18T09:34:27.193 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.193 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.193 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.193 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.193 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-10-18T09:34:27.194 INFO:teuthology.orchestra.run.smithi061.stdout:total 1 2023-10-18T09:34:27.194 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 . 2023-10-18T09:34:27.194 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.194 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 mappings 2023-10-18T09:34:27.194 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Oct 18 09:34 replorigin_checkpoint 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 snapshots 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.195 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 .. 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 .. 2023-10-18T09:34:27.196 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.197 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-10-18T09:34:27.197 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.197 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 . 2023-10-18T09:34:27.197 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.197 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 members 2023-10-18T09:34:27.198 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 offsets 2023-10-18T09:34:27.198 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.198 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-10-18T09:34:27.198 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:34:27.198 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:34:27.199 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 .. 2023-10-18T09:34:27.199 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:18 0000 2023-10-18T09:34:27.199 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.199 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-10-18T09:34:27.199 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Oct 18 09:18 .. 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 0000 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-10-18T09:34:27.200 INFO:teuthology.orchestra.run.smithi061.stdout:total 8 2023-10-18T09:34:27.201 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:19 . 2023-10-18T09:34:27.201 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.201 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Oct 18 09:19 0000 2023-10-18T09:34:27.201 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.201 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-10-18T09:34:27.202 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.202 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.202 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.202 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.202 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-10-18T09:34:27.203 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.203 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.203 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.203 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.203 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-10-18T09:34:27.204 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.204 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.204 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.204 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.204 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-10-18T09:34:27.205 INFO:teuthology.orchestra.run.smithi061.stdout:total 13 2023-10-18T09:34:27.205 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Oct 18 09:34 . 2023-10-18T09:34:27.205 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.205 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Oct 18 09:34 db_0.stat 2023-10-18T09:34:27.205 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Oct 18 09:34 db_12347.stat 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Oct 18 09:34 global.stat 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:34 . 2023-10-18T09:34:27.206 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.207 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.207 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-10-18T09:34:27.207 INFO:teuthology.orchestra.run.smithi061.stdout:total 136 2023-10-18T09:34:27.207 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:34 . 2023-10-18T09:34:27.207 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 139264 Oct 18 09:34 000A 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.208 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.209 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.209 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-10-18T09:34:27.209 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.209 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.209 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.210 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.210 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-10-18T09:34:27.210 INFO:teuthology.orchestra.run.smithi061.stdout:total 1032192 2023-10-18T09:34:27.210 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 18 09:34 . 2023-10-18T09:34:27.210 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000C9 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CA 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CB 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CC 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CD 2023-10-18T09:34:27.211 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CE 2023-10-18T09:34:27.212 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000CF 2023-10-18T09:34:27.212 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000D0 2023-10-18T09:34:27.212 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000D1 2023-10-18T09:34:27.212 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D2 2023-10-18T09:34:27.212 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D3 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D4 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D5 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D6 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D7 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D8 2023-10-18T09:34:27.213 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:32 0000000100000000000000D9 2023-10-18T09:34:27.214 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:33 0000000100000000000000DA 2023-10-18T09:34:27.214 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:33 0000000100000000000000DB 2023-10-18T09:34:27.214 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:33 0000000100000000000000DC 2023-10-18T09:34:27.214 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:33 0000000100000000000000DD 2023-10-18T09:34:27.214 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:33 0000000100000000000000DE 2023-10-18T09:34:27.217 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:34 0000000100000000000000DF 2023-10-18T09:34:27.217 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:34 0000000100000000000000E0 2023-10-18T09:34:27.217 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:27 0000000100000000000000E1 2023-10-18T09:34:27.218 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:26 0000000100000000000000E2 2023-10-18T09:34:27.218 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:25 0000000100000000000000E3 2023-10-18T09:34:27.218 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:25 0000000100000000000000E4 2023-10-18T09:34:27.218 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:27 0000000100000000000000E5 2023-10-18T09:34:27.218 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:26 0000000100000000000000E6 2023-10-18T09:34:27.219 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:27 0000000100000000000000E7 2023-10-18T09:34:27.219 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:26 0000000100000000000000E8 2023-10-18T09:34:27.219 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:25 0000000100000000000000E9 2023-10-18T09:34:27.219 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:25 0000000100000000000000EA 2023-10-18T09:34:27.219 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:26 0000000100000000000000EB 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:25 0000000100000000000000EC 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:27 0000000100000000000000ED 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:26 0000000100000000000000EE 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:30 0000000100000000000000EF 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:27 0000000100000000000000F0 2023-10-18T09:34:27.220 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 0000000100000000000000F1 2023-10-18T09:34:27.221 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000F2 2023-10-18T09:34:27.221 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000F3 2023-10-18T09:34:27.221 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 0000000100000000000000F4 2023-10-18T09:34:27.221 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000F5 2023-10-18T09:34:27.221 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:30 0000000100000000000000F6 2023-10-18T09:34:27.222 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000F7 2023-10-18T09:34:27.222 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000F8 2023-10-18T09:34:27.222 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:31 0000000100000000000000F9 2023-10-18T09:34:27.222 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000FA 2023-10-18T09:34:27.222 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000FB 2023-10-18T09:34:27.223 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 0000000100000000000000FC 2023-10-18T09:34:27.223 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000FD 2023-10-18T09:34:27.223 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:30 0000000100000000000000FE 2023-10-18T09:34:27.223 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 0000000100000000000000FF 2023-10-18T09:34:27.223 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:30 000000010000000100000000 2023-10-18T09:34:27.224 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 000000010000000100000001 2023-10-18T09:34:27.224 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 000000010000000100000002 2023-10-18T09:34:27.224 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:30 000000010000000100000003 2023-10-18T09:34:27.224 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 000000010000000100000004 2023-10-18T09:34:27.224 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 000000010000000100000005 2023-10-18T09:34:27.225 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:29 000000010000000100000006 2023-10-18T09:34:27.225 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Oct 18 09:28 000000010000000100000007 2023-10-18T09:34:27.225 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 archive_status 2023-10-18T09:34:27.225 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.225 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:total 0 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Oct 18 09:18 . 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 64 Oct 18 09:34 .. 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-10-18T09:34:27.226 INFO:teuthology.orchestra.run.smithi061.stdout:total 176 2023-10-18T09:34:27.227 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Oct 18 09:18 . 2023-10-18T09:34:27.227 INFO:teuthology.orchestra.run.smithi061.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Oct 18 09:34 .. 2023-10-18T09:34:27.227 INFO:teuthology.orchestra.run.smithi061.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Oct 18 09:34 0000 2023-10-18T09:34:27.227 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-18T09:34:27.237 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-18T09:34:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:28 smithi170 ceph-mon[135150]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2023-10-18T09:34:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:28 smithi110 ceph-mon[134818]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2023-10-18T09:34:28.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:28 smithi061 ceph-mon[98573]: pgmap v835: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.8 MiB/s wr, 723 op/s 2023-10-18T09:34:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:29 smithi110 ceph-mon[134818]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2023-10-18T09:34:29.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:29 smithi061 ceph-mon[98573]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2023-10-18T09:34:29.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:29 smithi170 ceph-mon[135150]: pgmap v836: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.7 MiB/s wr, 721 op/s 2023-10-18T09:34:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:31 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:31 smithi110 ceph-mon[134818]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:34:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:31 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:31 smithi061 ceph-mon[98573]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:34:31.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:34:31.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:31 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:31.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:31 smithi170 ceph-mon[135150]: pgmap v837: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 4.7 MiB/s wr, 509 op/s 2023-10-18T09:34:32.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 -- 172.21.15.61:0/1928089093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 msgr2=0x7f4554101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 --2- 172.21.15.61:0/1928089093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554101df0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45480099c0 tx=0x7f4548009cd0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 -- 172.21.15.61:0/1928089093 shutdown_connections 2023-10-18T09:34:32.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 --2- 172.21.15.61:0/1928089093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 0x7f4554108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.108 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 --2- 172.21.15.61:0/1928089093 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 0x7f4554100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 --2- 172.21.15.61:0/1928089093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554101df0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.104+0000 7f455a1ff700 1 -- 172.21.15.61:0/1928089093 >> 172.21.15.61:0/1928089093 conn(0x7f45540fbcf0 msgr2=0x7f45540fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.105+0000 7f455a1ff700 1 -- 172.21.15.61:0/1928089093 shutdown_connections 2023-10-18T09:34:32.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.105+0000 7f455a1ff700 1 -- 172.21.15.61:0/1928089093 wait complete. 2023-10-18T09:34:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.105+0000 7f455a1ff700 1 Processor -- start 2023-10-18T09:34:32.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 -- start start 2023-10-18T09:34:32.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 0x7f4554074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 0x7f4554073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540738c0 con 0x7f45541007a0 2023-10-18T09:34:32.113 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4554073a30 con 0x7f4554101990 2023-10-18T09:34:32.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f455a1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541a07a0 con 0x7f4554102330 2023-10-18T09:34:32.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43378/0 (socket says 172.21.15.61:43378) 2023-10-18T09:34:32.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f4552ffd700 1 -- 172.21.15.61:0/2651009827 learned_addr learned my addr 172.21.15.61:0/2651009827 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:32.114 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f45537fe700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 0x7f4554074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.106+0000 7f4553fff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 0x7f4554073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 -- 172.21.15.61:0/2651009827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 msgr2=0x7f4554073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 0x7f4554073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 -- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 msgr2=0x7f4554074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.115 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 0x7f4554074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 -- 172.21.15.61:0/2651009827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45480095f0 con 0x7f4554101990 2023-10-18T09:34:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45480058e0 tx=0x7f454800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4550ff9700 1 -- 172.21.15.61:0/2651009827 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454801d070 con 0x7f4554101990 2023-10-18T09:34:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4550ff9700 1 -- 172.21.15.61:0/2651009827 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454800bda0 con 0x7f4554101990 2023-10-18T09:34:32.116 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45541a09d0 con 0x7f4554101990 2023-10-18T09:34:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f4550ff9700 1 -- 172.21.15.61:0/2651009827 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548004b70 con 0x7f4554101990 2023-10-18T09:34:32.117 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.107+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45541a0df0 con 0x7f4554101990 2023-10-18T09:34:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.110+0000 7f4550ff9700 1 -- 172.21.15.61:0/2651009827 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f4548004cd0 con 0x7f4554101990 2023-10-18T09:34:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 Processor -- start 2023-10-18T09:34:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 -- start start 2023-10-18T09:34:32.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f4550ff9700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 0x7f453c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce9c30 0x56438cce8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438cce8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cc5b6e0 con 0x56438cce9c30 2023-10-18T09:34:32.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cc41c60 con 0x56438cce85d0 2023-10-18T09:34:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f456dc08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cc3f580 con 0x56438cce92d0 2023-10-18T09:34:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f4550ff9700 1 -- 172.21.15.61:0/2651009827 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f45480cde80 con 0x7f4554101990 2023-10-18T09:34:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f45537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f45537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:53600/0 (socket says 172.21.15.61:53600) 2023-10-18T09:34:32.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f45537fe700 1 -- 172.21.15.61:0/3400604 learned_addr learned my addr 172.21.15.61:0/3400604 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.112+0000 7f4553fff700 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438cce8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f4552ffd700 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce9c30 0x56438cce8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f4553fff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 0x7f453c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 -- 172.21.15.61:0/3400604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 msgr2=0x56438cce8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438cce8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 -- 172.21.15.61:0/3400604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce9c30 msgr2=0x56438cce8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce9c30 0x56438cce8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 -- 172.21.15.61:0/3400604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56438cbf83f0 con 0x56438cce92d0 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f45537fe700 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4544009d30 tx=0x7f4544011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dea3c4a4a8fabb8f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f4553fff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 0x7f453c0a39c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f454c009fd0 tx=0x7f454c009100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.113+0000 7f4539ffb700 1 -- 172.21.15.61:0/3400604 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544004220 con 0x56438cce92d0 2023-10-18T09:34:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f4539ffb700 1 -- 172.21.15.61:0/3400604 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4544004380 con 0x56438cce92d0 2023-10-18T09:34:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f4539ffb700 1 -- 172.21.15.61:0/3400604 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440198d0 con 0x56438cce92d0 2023-10-18T09:34:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f456dc08b80 1 -- 172.21.15.61:0/3400604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 msgr2=0x56438cce96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f456dc08b80 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4544009d30 tx=0x7f4544011720 comp rx=0 tx=0).stop 2023-10-18T09:34:32.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f456dc08b80 1 -- 172.21.15.61:0/3400604 shutdown_connections 2023-10-18T09:34:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f456dc08b80 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438cce8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.114+0000 7f456dc08b80 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce92d0 0x56438cce96f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.115+0000 7f456dc08b80 1 --2- 172.21.15.61:0/3400604 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce9c30 0x56438cce8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.115+0000 7f456dc08b80 1 -- 172.21.15.61:0/3400604 >> 172.21.15.61:0/3400604 conn(0x56438cc5c720 msgr2=0x56438cce2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.115+0000 7f456dc08b80 1 -- 172.21.15.61:0/3400604 shutdown_connections 2023-10-18T09:34:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.115+0000 7f456dc08b80 1 -- 172.21.15.61:0/3400604 wait complete. 2023-10-18T09:34:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.115+0000 7f456dc08b80 1 -- start start 2023-10-18T09:34:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 Processor -- start 2023-10-18T09:34:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cc3ae70 con 0x56438cce85d0 2023-10-18T09:34:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cbd7710 con 0x56438cce92d0 2023-10-18T09:34:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cbd73b0 con 0x56438cce9c30 2023-10-18T09:34:32.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.116+0000 7f456dc08b80 10 client.0 _ll_register_callbacks cb 0x56438cb21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:34:32.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43386/0 (socket says 172.21.15.61:43386) 2023-10-18T09:34:32.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 -- 172.21.15.61:0/2018358576 learned_addr learned my addr 172.21.15.61:0/2018358576 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:32.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f45537fe700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4553fff700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 -- 172.21.15.61:0/2018358576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 msgr2=0x56438ccf8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 -- 172.21.15.61:0/2018358576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 msgr2=0x56438ccf1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.132 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 -- 172.21.15.61:0/2018358576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544009960 con 0x56438cce92d0 2023-10-18T09:34:32.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45480cb5d0 tx=0x7f454809b3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97d5751a1ea28ecd server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.117+0000 7f4532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:34:32.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454809b580 con 0x56438cce92d0 2023-10-18T09:34:32.133 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f454809ae80 con 0x56438cce92d0 2023-10-18T09:34:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56438cc3fef0 con 0x56438cce92d0 2023-10-18T09:34:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454809efd0 con 0x56438cce92d0 2023-10-18T09:34:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 10 client.26525 fetch_fsmap learned FSMap version 44 2023-10-18T09:34:32.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56438ca65de0 con 0x56438cce92d0 2023-10-18T09:34:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f454807d080 con 0x56438cce92d0 2023-10-18T09:34:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 44) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f45480a2cf0 con 0x56438cce92d0 2023-10-18T09:34:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 10 client.26525 fetch_fsmap finished waiting for FSMap version 44 2023-10-18T09:34:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:34:32.135 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 20 client.26525 populate_metadata read hostname 'smithi061' 2023-10-18T09:34:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438ccfd4b0 0x56438ccff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 4 client.26525 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:34:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.118+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56438ccb78b0 con 0x56438ccfd4b0 2023-10-18T09:34:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.119+0000 7f45537fe700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438ccfd4b0 0x56438ccff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.136 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.119+0000 7f45537fe700 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438ccfd4b0 0x56438ccff8a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a262808f58686da server_cookie=582c78a63e9974ac in_seq=0 out_seq=0 2023-10-18T09:34:32.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.119+0000 7f4532ffd700 10 client.26525 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:32.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.120+0000 7f4532ffd700 1 -- 172.21.15.61:0/2018358576 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56438ccb78b0 con 0x56438ccfd4b0 2023-10-18T09:34:32.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.120+0000 7f4532ffd700 10 client.26525 handle_command_reply: tid=0 2023-10-18T09:34:32.138 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.120+0000 7f456dc08b80 1 client.26525 shutdown 2023-10-18T09:34:32.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.120+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 msgr2=0x56438ccf1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.120+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f45480cb5d0 tx=0x7f454809b3c0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 shutdown_connections 2023-10-18T09:34:32.139 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438ccfd4b0 0x56438ccff8a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2018358576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf1fb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.140 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 >> 172.21.15.61:0/2018358576 conn(0x56438cc5c720 msgr2=0x56438cce7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f4532ffd700 0 client.26525 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:32.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 shutdown_connections 2023-10-18T09:34:32.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 1 -- 172.21.15.61:0/2018358576 wait complete. 2023-10-18T09:34:32.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.121+0000 7f456dc08b80 20 client.26525 trim_cache size 0 max 16384 2023-10-18T09:34:32.147 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.138+0000 7f456dc08b80 1 Processor -- start 2023-10-18T09:34:32.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.138+0000 7f456dc08b80 1 -- start start 2023-10-18T09:34:32.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.138+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.138+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.148 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.138+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f456dc08b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438ccddaf0 con 0x56438cce92d0 2023-10-18T09:34:32.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f456dc08b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cabb920 con 0x56438cce85d0 2023-10-18T09:34:32.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f456dc08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cce30a0 con 0x56438cce9c30 2023-10-18T09:34:32.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.149 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43398/0 (socket says 172.21.15.61:43398) 2023-10-18T09:34:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 -- 172.21.15.61:0/1218772643 learned_addr learned my addr 172.21.15.61:0/1218772643 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:58412/0 (socket says 172.21.15.61:58412) 2023-10-18T09:34:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f4553fff700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 -- 172.21.15.61:0/1218772643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 msgr2=0x56438ccf1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 -- 172.21.15.61:0/1218772643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 msgr2=0x56438ccf8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 -- 172.21.15.61:0/1218772643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56438ca4ec60 con 0x56438cce85d0 2023-10-18T09:34:32.151 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f4553fff700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f4552ffd700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.139+0000 7f45537fe700 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4544009230 tx=0x7f4544011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3b82da472ab6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f453a7fc700 1 -- 172.21.15.61:0/1218772643 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544004180 con 0x56438cce85d0 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f453a7fc700 1 -- 172.21.15.61:0/1218772643 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4544005500 con 0x56438cce85d0 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f453a7fc700 1 -- 172.21.15.61:0/1218772643 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454402aa00 con 0x56438cce85d0 2023-10-18T09:34:32.152 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 -- 172.21.15.61:0/1218772643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 msgr2=0x56438ccec9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4544009230 tx=0x7f4544011720 comp rx=0 tx=0).stop 2023-10-18T09:34:32.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 -- 172.21.15.61:0/1218772643 shutdown_connections 2023-10-18T09:34:32.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce85d0 0x56438ccec9e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.153 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 --2- 172.21.15.61:0/1218772643 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce92d0 0x56438ccf8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 -- 172.21.15.61:0/1218772643 >> 172.21.15.61:0/1218772643 conn(0x56438cc5c720 msgr2=0x56438ccec2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 -- 172.21.15.61:0/1218772643 shutdown_connections 2023-10-18T09:34:32.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.140+0000 7f456dc08b80 1 -- 172.21.15.61:0/1218772643 wait complete. 2023-10-18T09:34:32.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 -- start start 2023-10-18T09:34:32.154 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 Processor -- start 2023-10-18T09:34:32.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438ccf83b0 con 0x56438cce85d0 2023-10-18T09:34:32.155 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438ccf8520 con 0x56438cce92d0 2023-10-18T09:34:32.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56438cd51310 con 0x56438cce9c30 2023-10-18T09:34:32.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.141+0000 7f456dc08b80 10 client.0 _ll_register_callbacks cb 0x56438cb21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:34:32.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:43400/0 (socket says 172.21.15.61:43400) 2023-10-18T09:34:32.156 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 -- 172.21.15.61:0/2855490212 learned_addr learned my addr 172.21.15.61:0/2855490212 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:34:32.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f45537fe700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4553fff700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 -- 172.21.15.61:0/2855490212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 msgr2=0x56438ccf7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.157 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 -- 172.21.15.61:0/2855490212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 msgr2=0x56438ccf9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:34:32.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 -- 172.21.15.61:0/2855490212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56438ca65de0 con 0x56438cce92d0 2023-10-18T09:34:32.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4552ffd700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45480a3050 tx=0x7f454809a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148a920ad8ca2ee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:34:32.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:34:32.158 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45480aedd0 con 0x56438cce92d0 2023-10-18T09:34:32.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4544004380 con 0x56438cce92d0 2023-10-18T09:34:32.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.142+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f454809b580 con 0x56438cce92d0 2023-10-18T09:34:32.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454809ae80 con 0x56438cce92d0 2023-10-18T09:34:32.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 10 client.26531 fetch_fsmap learned FSMap version 44 2023-10-18T09:34:32.159 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56438cbf83f0 con 0x56438cce92d0 2023-10-18T09:34:32.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4548083030 con 0x56438cce92d0 2023-10-18T09:34:32.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 44) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f45480cba90 con 0x56438cce92d0 2023-10-18T09:34:32.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 10 client.26531 fetch_fsmap finished waiting for FSMap version 44 2023-10-18T09:34:32.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 10 client.26531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:34:32.160 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 20 client.26531 populate_metadata read hostname 'smithi061' 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438cd529b0 0x56438cd54dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 4 client.26531 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56438cc33e50 con 0x56438cd529b0 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.143+0000 7f45537fe700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438cd529b0 0x56438cd54dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f45537fe700 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438cd529b0 0x56438cd54dd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5febb44f57a3f69 server_cookie=11d6ddf89bb2a748 in_seq=0 out_seq=0 2023-10-18T09:34:32.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f4532ffd700 10 client.26531 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:32.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f4532ffd700 1 -- 172.21.15.61:0/2855490212 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56438cc33e50 con 0x56438cd529b0 2023-10-18T09:34:32.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f4532ffd700 10 client.26531 handle_command_reply: tid=0 2023-10-18T09:34:32.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 client.26531 shutdown 2023-10-18T09:34:32.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 msgr2=0x56438ccf7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.162 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f45480a3050 tx=0x7f454809a6d0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 shutdown_connections 2023-10-18T09:34:32.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56438cd529b0 0x56438cd54dd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56438cce9c30 0x56438ccf7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56438cce85d0 0x56438ccf9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 --2- 172.21.15.61:0/2855490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56438cce92d0 0x56438ccf7650 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f4532ffd700 0 client.26531 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:34:32.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 >> 172.21.15.61:0/2855490212 conn(0x56438cc5c720 msgr2=0x56438ccf1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 shutdown_connections 2023-10-18T09:34:32.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.144+0000 7f456dc08b80 1 -- 172.21.15.61:0/2855490212 wait complete. 2023-10-18T09:34:32.164 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f456dc08b80 20 client.26531 trim_cache size 0 max 16384 2023-10-18T09:34:32.165 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:34:32.165 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:34:32.165 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:34:32.165 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:34:32.166 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:34:32.166 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:34:32.166 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:34:32.166 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:34:32.167 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:34:32.167 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:34:32.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 msgr2=0x7f453c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 0x7f453c0a39c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f454c009fd0 tx=0x7f454c009100 comp rx=0 tx=0).stop 2023-10-18T09:34:32.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 msgr2=0x7f4554072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:34:32.169 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45480058e0 tx=0x7f454800fa00 comp rx=0 tx=0).stop 2023-10-18T09:34:32.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 shutdown_connections 2023-10-18T09:34:32.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f453c0a1500 0x7f453c0a39c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4554102330 0x7f4554073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f45541007a0 0x7f4554074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.170 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 --2- 172.21.15.61:0/2651009827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4554101990 0x7f4554072be0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:34:32.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 >> 172.21.15.61:0/2651009827 conn(0x7f45540fbcf0 msgr2=0x7f4554104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:34:32.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 shutdown_connections 2023-10-18T09:34:32.171 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:34:32.145+0000 7f455a1ff700 1 -- 172.21.15.61:0/2651009827 wait complete. 2023-10-18T09:34:32.172 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:34:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:33 smithi110 ceph-mon[134818]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2023-10-18T09:34:33.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:33 smithi061 ceph-mon[98573]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2023-10-18T09:34:33.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:33 smithi170 ceph-mon[135150]: pgmap v838: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.2 MiB/s wr, 453 op/s 2023-10-18T09:34:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:35 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:34:35.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:35 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:34:35.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:35 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:34:36.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:36 smithi061 ceph-mon[98573]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.8 MiB/s wr, 323 op/s 2023-10-18T09:34:36.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:36 smithi170 ceph-mon[135150]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.8 MiB/s wr, 323 op/s 2023-10-18T09:34:36.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:36 smithi110 ceph-mon[134818]: pgmap v839: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.8 MiB/s wr, 323 op/s 2023-10-18T09:34:37.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:34:37.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:34:37.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:34:37.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:34:37.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:34:37.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:34:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:34:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:34:37.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:34:38.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:38 smithi061 ceph-mon[98573]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2023-10-18T09:34:38.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:38 smithi170 ceph-mon[135150]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2023-10-18T09:34:38.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:38 smithi110 ceph-mon[134818]: pgmap v840: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 153 op/s 2023-10-18T09:34:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:39 smithi110 ceph-mon[134818]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 821 KiB/s wr, 107 op/s 2023-10-18T09:34:39.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:39 smithi061 ceph-mon[98573]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 821 KiB/s wr, 107 op/s 2023-10-18T09:34:39.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:39 smithi170 ceph-mon[135150]: pgmap v841: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 821 KiB/s wr, 107 op/s 2023-10-18T09:34:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:41 smithi110 ceph-mon[134818]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-10-18T09:34:41.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:41 smithi061 ceph-mon[98573]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-10-18T09:34:41.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:41 smithi170 ceph-mon[135150]: pgmap v842: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-10-18T09:34:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:43 smithi110 ceph-mon[134818]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2023-10-18T09:34:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:43 smithi061 ceph-mon[98573]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2023-10-18T09:34:43.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:43 smithi170 ceph-mon[135150]: pgmap v843: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 16 op/s 2023-10-18T09:34:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:45 smithi110 ceph-mon[134818]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2023-10-18T09:34:45.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:45 smithi061 ceph-mon[98573]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2023-10-18T09:34:45.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:45 smithi170 ceph-mon[135150]: pgmap v844: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 15 op/s 2023-10-18T09:34:48.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:48 smithi170 ceph-mon[135150]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-10-18T09:34:48.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:48 smithi110 ceph-mon[134818]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-10-18T09:34:48.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:48 smithi061 ceph-mon[98573]: pgmap v845: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2023-10-18T09:34:49.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:49 smithi110 ceph-mon[134818]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-10-18T09:34:49.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:49 smithi061 ceph-mon[98573]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-10-18T09:34:49.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:49 smithi170 ceph-mon[135150]: pgmap v846: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2023-10-18T09:34:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:51 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:51.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:51 smithi110 ceph-mon[134818]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-10-18T09:34:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:51 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:51.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:51 smithi061 ceph-mon[98573]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-10-18T09:34:51.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:51 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:51.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:51 smithi170 ceph-mon[135150]: pgmap v847: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-10-18T09:34:53.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:53 smithi110 ceph-mon[134818]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-10-18T09:34:53.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:53 smithi061 ceph-mon[98573]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-10-18T09:34:53.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:53 smithi170 ceph-mon[135150]: pgmap v848: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 33 KiB/s wr, 14 op/s 2023-10-18T09:34:55.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:55 smithi110 ceph-mon[134818]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:34:55.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:55 smithi061 ceph-mon[98573]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:34:55.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:55 smithi170 ceph-mon[135150]: pgmap v849: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:34:56.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:56 smithi110 ceph-mon[134818]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:56.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:56 smithi061 ceph-mon[98573]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:56.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:56 smithi170 ceph-mon[135150]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-10-18T09:34:58.394 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:58 smithi170 ceph-mon[135150]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-10-18T09:34:58.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:58 smithi110 ceph-mon[134818]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-10-18T09:34:58.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:58 smithi061 ceph-mon[98573]: pgmap v850: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 14 op/s 2023-10-18T09:34:59.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:34:59 smithi110 ceph-mon[134818]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:34:59.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:34:59 smithi061 ceph-mon[98573]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:34:59.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:34:59 smithi170 ceph-mon[135150]: pgmap v851: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 38 KiB/s wr, 15 op/s 2023-10-18T09:35:01.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:01 smithi110 ceph-mon[134818]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-10-18T09:35:01.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:01 smithi061 ceph-mon[98573]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-10-18T09:35:01.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:01 smithi170 ceph-mon[135150]: pgmap v852: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 13 op/s 2023-10-18T09:35:02.173 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:35:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 -- 172.21.15.61:0/2676606597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 msgr2=0x7f027c108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 --2- 172.21.15.61:0/2676606597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c108be0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f027400b0d0 tx=0x7f027400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 -- 172.21.15.61:0/2676606597 shutdown_connections 2023-10-18T09:35:02.391 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 --2- 172.21.15.61:0/2676606597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c108be0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 --2- 172.21.15.61:0/2676606597 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c1007a0 0x7f027c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 --2- 172.21.15.61:0/2676606597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c101990 0x7f027c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 -- 172.21.15.61:0/2676606597 >> 172.21.15.61:0/2676606597 conn(0x7f027c0fbcf0 msgr2=0x7f027c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 -- 172.21.15.61:0/2676606597 shutdown_connections 2023-10-18T09:35:02.392 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.388+0000 7f0280bff700 1 -- 172.21.15.61:0/2676606597 wait complete. 2023-10-18T09:35:02.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 Processor -- start 2023-10-18T09:35:02.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 -- start start 2023-10-18T09:35:02.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c1007a0 0x7f027c196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c101990 0x7f027c196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19e370 con 0x7f027c101990 2023-10-18T09:35:02.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19e4b0 con 0x7f027c102330 2023-10-18T09:35:02.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.389+0000 7f0280bff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c19e640 con 0x7f027c1007a0 2023-10-18T09:35:02.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60278/0 (socket says 172.21.15.61:60278) 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f0273fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c101990 0x7f027c196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 -- 172.21.15.61:0/1726314680 learned_addr learned my addr 172.21.15.61:0/1726314680 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027a59c700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c1007a0 0x7f027c196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 -- 172.21.15.61:0/1726314680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c1007a0 msgr2=0x7f027c196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c1007a0 0x7f027c196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 -- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c101990 msgr2=0x7f027c196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c101990 0x7f027c196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 -- 172.21.15.61:0/1726314680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0274009d00 con 0x7f027c102330 2023-10-18T09:35:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0274007b60 tx=0x7f0274008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.390+0000 7f02737fe700 1 -- 172.21.15.61:0/1726314680 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400e040 con 0x7f027c102330 2023-10-18T09:35:02.398 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.391+0000 7f02737fe700 1 -- 172.21.15.61:0/1726314680 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027401d930 con 0x7f027c102330 2023-10-18T09:35:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.391+0000 7f02737fe700 1 -- 172.21.15.61:0/1726314680 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401ba40 con 0x7f027c102330 2023-10-18T09:35:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.391+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027c19e8c0 con 0x7f027c102330 2023-10-18T09:35:02.399 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.391+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c19ed40 con 0x7f027c102330 2023-10-18T09:35:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.394+0000 7f02737fe700 1 -- 172.21.15.61:0/1726314680 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f027401daa0 con 0x7f027c102330 2023-10-18T09:35:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 Processor -- start 2023-10-18T09:35:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 -- start start 2023-10-18T09:35:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02737fe700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 0x7f02680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02737fe700 1 -- 172.21.15.61:0/1726314680 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f02740ce4b0 con 0x7f027c102330 2023-10-18T09:35:02.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab006e0 con 0x56195ab8e2d0 2023-10-18T09:35:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195aae6c60 con 0x56195ab8d5d0 2023-10-18T09:35:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.396+0000 7f02945bfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195aae4580 con 0x56195ab8ec30 2023-10-18T09:35:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f0273fff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 0x7f02680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60282/0 (socket says 172.21.15.61:60282) 2023-10-18T09:35:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 -- 172.21.15.61:0/1776666881 learned_addr learned my addr 172.21.15.61:0/1776666881 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f0273fff700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027a59c700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 -- 172.21.15.61:0/1776666881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 msgr2=0x56195ab8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 -- 172.21.15.61:0/1776666881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 msgr2=0x56195ab8e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab8e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 -- 172.21.15.61:0/1776666881 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56195aa9d3f0 con 0x56195ab8d5d0 2023-10-18T09:35:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027a59c700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab8e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-18T09:35:02.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f027ad9d700 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02740060b0 tx=0x7f02740978c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118aac9488663412 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f0273fff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 0x7f02680a39c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0264011710 tx=0x7f02640099b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.397+0000 7f0270ff9700 1 -- 172.21.15.61:0/1776666881 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027409bbb0 con 0x56195ab8d5d0 2023-10-18T09:35:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f0270ff9700 1 -- 172.21.15.61:0/1776666881 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027409b4b0 con 0x56195ab8d5d0 2023-10-18T09:35:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f0270ff9700 1 -- 172.21.15.61:0/1776666881 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401de10 con 0x56195ab8d5d0 2023-10-18T09:35:02.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 -- 172.21.15.61:0/1776666881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 msgr2=0x56195ab8da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f02740060b0 tx=0x7f02740978c0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 -- 172.21.15.61:0/1776666881 shutdown_connections 2023-10-18T09:35:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8da50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab8e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 --2- 172.21.15.61:0/1776666881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 -- 172.21.15.61:0/1776666881 >> 172.21.15.61:0/1776666881 conn(0x56195ab01720 msgr2=0x56195ab87b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 -- 172.21.15.61:0/1776666881 shutdown_connections 2023-10-18T09:35:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.398+0000 7f02945bfb80 1 -- 172.21.15.61:0/1776666881 wait complete. 2023-10-18T09:35:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 -- start start 2023-10-18T09:35:02.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 Processor -- start 2023-10-18T09:35:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195aadfe70 con 0x56195ab8e2d0 2023-10-18T09:35:02.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195aa7c710 con 0x56195ab8d5d0 2023-10-18T09:35:02.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.399+0000 7f02945bfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195aa7c3b0 con 0x56195ab8ec30 2023-10-18T09:35:02.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f02945bfb80 10 client.0 _ll_register_callbacks cb 0x56195a9c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:35:02.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f0273fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.414 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60284/0 (socket says 172.21.15.61:60284) 2023-10-18T09:35:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 -- 172.21.15.61:0/2686076549 learned_addr learned my addr 172.21.15.61:0/2686076549 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f0273fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:50316/0 (socket says 172.21.15.61:50316) 2023-10-18T09:35:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027ad9d700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 -- 172.21.15.61:0/2686076549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 msgr2=0x56195ab9de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.415 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab9de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 -- 172.21.15.61:0/2686076549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 msgr2=0x56195ab96e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f027a59c700 1 -- 172.21.15.61:0/2686076549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027c19ed40 con 0x56195ab8d5d0 2023-10-18T09:35:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.400+0000 7f0273fff700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:35:02.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f027a59c700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f026c012700 tx=0x7f026c012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15dd2ebe80dbda0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:35:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c004020 con 0x56195ab8d5d0 2023-10-18T09:35:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f026c004180 con 0x56195ab8d5d0 2023-10-18T09:35:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56195aae4ef0 con 0x56195ab8d5d0 2023-10-18T09:35:02.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f026c004340 con 0x56195ab8d5d0 2023-10-18T09:35:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f02945bfb80 10 client.26543 fetch_fsmap learned FSMap version 46 2023-10-18T09:35:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56195a90ade0 con 0x56195ab8d5d0 2023-10-18T09:35:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56195aae4ef0 con 0x56195ab8d5d0 2023-10-18T09:35:02.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.401+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 46) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f026c0107f0 con 0x56195ab8d5d0 2023-10-18T09:35:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 10 client.26543 fetch_fsmap finished waiting for FSMap version 46 2023-10-18T09:35:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 10 client.26543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:35:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 20 client.26543 populate_metadata read hostname 'smithi061' 2023-10-18T09:35:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195aba2770 0x56195aba4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 4 client.26543 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:35:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56195ab5c8b0 con 0x56195aba2770 2023-10-18T09:35:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f0273fff700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195aba2770 0x56195aba4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f0261ffb700 10 client.26543 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:02.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.402+0000 7f0273fff700 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195aba2770 0x56195aba4b90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109e84f6ebdd2719 server_cookie=96bf33cf6aca8ea7 in_seq=0 out_seq=0 2023-10-18T09:35:02.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f0261ffb700 1 -- 172.21.15.61:0/2686076549 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56195ab5c8b0 con 0x56195aba2770 2023-10-18T09:35:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f0261ffb700 10 client.26543 handle_command_reply: tid=0 2023-10-18T09:35:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 client.26543 shutdown 2023-10-18T09:35:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 msgr2=0x56195ab968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f026c012700 tx=0x7f026c012a10 comp rx=0 tx=0).stop 2023-10-18T09:35:02.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 shutdown_connections 2023-10-18T09:35:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195aba2770 0x56195aba4b90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8ec30 0x56195ab9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab968c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 --2- 172.21.15.61:0/2686076549 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8e2d0 0x56195ab96e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f0261ffb700 0 client.26543 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 >> 172.21.15.61:0/2686076549 conn(0x56195ab01720 msgr2=0x56195ab8c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 shutdown_connections 2023-10-18T09:35:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.403+0000 7f02945bfb80 1 -- 172.21.15.61:0/2686076549 wait complete. 2023-10-18T09:35:02.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.404+0000 7f02945bfb80 20 client.26543 trim_cache size 0 max 16384 2023-10-18T09:35:02.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 Processor -- start 2023-10-18T09:35:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 -- start start 2023-10-18T09:35:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab94bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195a960920 con 0x56195ab8ec30 2023-10-18T09:35:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab892e0 con 0x56195ab8d5d0 2023-10-18T09:35:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.421+0000 7f02945bfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab89b80 con 0x56195ab8e2d0 2023-10-18T09:35:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab94bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60296/0 (socket says 172.21.15.61:60296) 2023-10-18T09:35:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027a59c700 1 -- 172.21.15.61:0/3584565637 learned_addr learned my addr 172.21.15.61:0/3584565637 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f0273fff700 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 -- 172.21.15.61:0/3584565637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 msgr2=0x56195ab9de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 -- 172.21.15.61:0/3584565637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 msgr2=0x56195ab8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 -- 172.21.15.61:0/3584565637 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56195a8f3c60 con 0x56195ab8ec30 2023-10-18T09:35:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f027ad9d700 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab94bd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0274086110 tx=0x7f0274006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385405af4733304f server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f02717fa700 1 -- 172.21.15.61:0/3584565637 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027409bbb0 con 0x56195ab8ec30 2023-10-18T09:35:02.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f02717fa700 1 -- 172.21.15.61:0/3584565637 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02740b7070 con 0x56195ab8ec30 2023-10-18T09:35:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.422+0000 7f02717fa700 1 -- 172.21.15.61:0/3584565637 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027409b710 con 0x56195ab8ec30 2023-10-18T09:35:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- 172.21.15.61:0/3584565637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 msgr2=0x56195ab94bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab94bd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0274086110 tx=0x7f0274006000 comp rx=0 tx=0).stop 2023-10-18T09:35:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- 172.21.15.61:0/3584565637 shutdown_connections 2023-10-18T09:35:02.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab94bd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 --2- 172.21.15.61:0/3584565637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- 172.21.15.61:0/3584565637 >> 172.21.15.61:0/3584565637 conn(0x56195ab01720 msgr2=0x56195aba94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- 172.21.15.61:0/3584565637 shutdown_connections 2023-10-18T09:35:02.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- 172.21.15.61:0/3584565637 wait complete. 2023-10-18T09:35:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.423+0000 7f02945bfb80 1 -- start start 2023-10-18T09:35:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 Processor -- start 2023-10-18T09:35:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab9d080 con 0x56195ab8ec30 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab9d1f0 con 0x56195ab8d5d0 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x56195ab9d3b0 con 0x56195ab8e2d0 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f02945bfb80 10 client.0 _ll_register_callbacks cb 0x56195a9c6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f027ad9d700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:50336/0 (socket says 172.21.15.61:50336) 2023-10-18T09:35:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f027a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:60304/0 (socket says 172.21.15.61:60304) 2023-10-18T09:35:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.424+0000 7f027ad9d700 1 -- 172.21.15.61:0/900676594 learned_addr learned my addr 172.21.15.61:0/900676594 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0273fff700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 -- 172.21.15.61:0/900676594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 msgr2=0x56195ab9bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 -- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 msgr2=0x56195ab9b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 -- 172.21.15.61:0/900676594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56195aa9d3f0 con 0x56195ab8ec30 2023-10-18T09:35:02.438 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0273fff700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027a59c700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:35:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f027ad9d700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f027409b440 tx=0x7f02740978f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af524599c769b43 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:35:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02740a37b0 con 0x56195ab8ec30 2023-10-18T09:35:02.439 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027409b4b0 con 0x56195ab8ec30 2023-10-18T09:35:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f027409f600 con 0x56195ab8ec30 2023-10-18T09:35:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02740ce200 con 0x56195ab8ec30 2023-10-18T09:35:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f02945bfb80 10 client.16308 fetch_fsmap learned FSMap version 46 2023-10-18T09:35:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.425+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x56195a931250 con 0x56195ab8ec30 2023-10-18T09:35:02.440 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027409b4b0 con 0x56195ab8ec30 2023-10-18T09:35:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 46) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f027401bba0 con 0x56195ab8ec30 2023-10-18T09:35:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 10 client.16308 fetch_fsmap finished waiting for FSMap version 46 2023-10-18T09:35:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:35:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 20 client.16308 populate_metadata read hostname 'smithi061' 2023-10-18T09:35:02.441 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195abf6bb0 0x56195abf8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:02.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 4 client.16308 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:35:02.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56195aad8e50 con 0x56195abf6bb0 2023-10-18T09:35:02.442 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.426+0000 7f027a59c700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195abf6bb0 0x56195abf8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:02.443 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f027a59c700 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195abf6bb0 0x56195abf8fd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94392a1599118e4c server_cookie=5ca169a3dfb8db05 in_seq=0 out_seq=0 2023-10-18T09:35:02.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f0261ffb700 10 client.16308 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:02.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f0261ffb700 1 -- 172.21.15.61:0/900676594 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56195aad8e50 con 0x56195abf6bb0 2023-10-18T09:35:02.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f0261ffb700 10 client.16308 handle_command_reply: tid=0 2023-10-18T09:35:02.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 client.16308 shutdown 2023-10-18T09:35:02.444 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 msgr2=0x56195ab9cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f027409b440 tx=0x7f02740978f0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 shutdown_connections 2023-10-18T09:35:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x56195abf6bb0 0x56195abf8fd0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x56195ab8ec30 0x56195ab9cb40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.445 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56195ab8d5d0 0x56195ab9b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f0261ffb700 0 client.16308 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 --2- 172.21.15.61:0/900676594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56195ab8e2d0 0x56195ab9bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 >> 172.21.15.61:0/900676594 conn(0x56195ab01720 msgr2=0x56195ab95e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 shutdown_connections 2023-10-18T09:35:02.446 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 1 -- 172.21.15.61:0/900676594 wait complete. 2023-10-18T09:35:02.447 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.427+0000 7f02945bfb80 20 client.16308 trim_cache size 0 max 16384 2023-10-18T09:35:02.447 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:35:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:35:02.448 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:35:02.448 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:35:02.448 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:35:02.448 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:35:02.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:02.449 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:02.449 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:02.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 msgr2=0x7f02680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 0x7f02680a39c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0264011710 tx=0x7f02640099b0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.450 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 msgr2=0x7f027c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:02.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0274007b60 tx=0x7f0274008e40 comp rx=0 tx=0).stop 2023-10-18T09:35:02.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 shutdown_connections 2023-10-18T09:35:02.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f02680a1500 0x7f02680a39c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f027c102330 0x7f027c19dda0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.451 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f027c1007a0 0x7f027c196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 --2- 172.21.15.61:0/1726314680 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f027c101990 0x7f027c196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:02.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 >> 172.21.15.61:0/1726314680 conn(0x7f027c0fbcf0 msgr2=0x7f027c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:02.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 shutdown_connections 2023-10-18T09:35:02.452 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:02.428+0000 7f0280bff700 1 -- 172.21.15.61:0/1726314680 wait complete. 2023-10-18T09:35:02.452 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:35:03.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:03 smithi110 ceph-mon[134818]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-10-18T09:35:03.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:03 smithi061 ceph-mon[98573]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-10-18T09:35:03.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:03 smithi170 ceph-mon[135150]: pgmap v853: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 15 op/s 2023-10-18T09:35:05.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:05 smithi110 ceph-mon[134818]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2023-10-18T09:35:05.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:05 smithi061 ceph-mon[98573]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2023-10-18T09:35:05.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:05 smithi170 ceph-mon[135150]: pgmap v854: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 13 op/s 2023-10-18T09:35:08.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:08 smithi170 ceph-mon[135150]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-10-18T09:35:08.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:08 smithi110 ceph-mon[134818]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-10-18T09:35:08.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:08 smithi061 ceph-mon[98573]: pgmap v855: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2023-10-18T09:35:09.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:09 smithi110 ceph-mon[134818]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-10-18T09:35:09.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:09 smithi061 ceph-mon[98573]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-10-18T09:35:09.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:09 smithi170 ceph-mon[135150]: pgmap v856: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2023-10-18T09:35:11.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:11 smithi110 ceph-mon[134818]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2023-10-18T09:35:11.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:11 smithi061 ceph-mon[98573]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2023-10-18T09:35:11.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:11 smithi170 ceph-mon[135150]: pgmap v857: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 11 op/s 2023-10-18T09:35:13.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:13 smithi110 ceph-mon[134818]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2023-10-18T09:35:13.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:13 smithi061 ceph-mon[98573]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2023-10-18T09:35:13.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:13 smithi170 ceph-mon[135150]: pgmap v858: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2023-10-18T09:35:15.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:15 smithi110 ceph-mon[134818]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T09:35:15.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:15 smithi061 ceph-mon[98573]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T09:35:15.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:15 smithi170 ceph-mon[135150]: pgmap v859: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2023-10-18T09:35:18.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:18 smithi170 ceph-mon[135150]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2023-10-18T09:35:18.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:18 smithi110 ceph-mon[134818]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2023-10-18T09:35:18.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:18 smithi061 ceph-mon[98573]: pgmap v860: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 12 op/s 2023-10-18T09:35:19.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:19 smithi110 ceph-mon[134818]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-10-18T09:35:19.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:19 smithi061 ceph-mon[98573]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-10-18T09:35:19.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:19 smithi170 ceph-mon[135150]: pgmap v861: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-10-18T09:35:21.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:21 smithi110 ceph-mon[134818]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-10-18T09:35:21.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:21 smithi061 ceph-mon[98573]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-10-18T09:35:21.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:21 smithi170 ceph-mon[135150]: pgmap v862: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2023-10-18T09:35:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:23 smithi110 ceph-mon[134818]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2023-10-18T09:35:23.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:23 smithi061 ceph-mon[98573]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2023-10-18T09:35:23.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:23 smithi170 ceph-mon[135150]: pgmap v863: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2023-10-18T09:35:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:25 smithi110 ceph-mon[134818]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-18T09:35:25.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:25 smithi061 ceph-mon[98573]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-18T09:35:25.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:25 smithi170 ceph-mon[135150]: pgmap v864: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2023-10-18T09:35:28.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:28 smithi170 ceph-mon[135150]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-18T09:35:28.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:28 smithi110 ceph-mon[134818]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-18T09:35:28.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:28 smithi061 ceph-mon[98573]: pgmap v865: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2023-10-18T09:35:29.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:29 smithi110 ceph-mon[134818]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:29.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:29 smithi061 ceph-mon[98573]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:29.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:29 smithi170 ceph-mon[135150]: pgmap v866: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:31.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:31 smithi110 ceph-mon[134818]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:35:31.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:31 smithi061 ceph-mon[98573]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:35:31.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:31 smithi170 ceph-mon[135150]: pgmap v867: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 853 B/s wr, 9 op/s 2023-10-18T09:35:32.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-18T09:35:32.676 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 -- 172.21.15.61:0/3445811546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf90101990 msgr2=0x7fdf90101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 --2- 172.21.15.61:0/3445811546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf90101990 0x7fdf90101df0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf840099c0 tx=0x7fdf84009cd0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 -- 172.21.15.61:0/3445811546 shutdown_connections 2023-10-18T09:35:32.677 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 --2- 172.21.15.61:0/3445811546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90102330 0x7fdf90108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 --2- 172.21.15.61:0/3445811546 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf901007a0 0x7fdf90100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 --2- 172.21.15.61:0/3445811546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf90101990 0x7fdf90101df0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.673+0000 7fdf969ff700 1 -- 172.21.15.61:0/3445811546 >> 172.21.15.61:0/3445811546 conn(0x7fdf900fbcf0 msgr2=0x7fdf900fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.674+0000 7fdf969ff700 1 -- 172.21.15.61:0/3445811546 shutdown_connections 2023-10-18T09:35:32.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.674+0000 7fdf969ff700 1 -- 172.21.15.61:0/3445811546 wait complete. 2023-10-18T09:35:32.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 Processor -- start 2023-10-18T09:35:32.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 -- start start 2023-10-18T09:35:32.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90101990 0x7fdf90198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf90102330 0x7fdf9019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a0450 con 0x7fdf90102330 2023-10-18T09:35:32.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a0590 con 0x7fdf901007a0 2023-10-18T09:35:32.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf969ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a0720 con 0x7fdf90101990 2023-10-18T09:35:32.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47654/0 (socket says 172.21.15.61:47654) 2023-10-18T09:35:32.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2372144221 learned_addr learned my addr 172.21.15.61:0/2372144221 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90101990 0x7fdf90198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.675+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf90102330 0x7fdf9019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2372144221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90101990 msgr2=0x7fdf90198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90101990 0x7fdf90198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf90102330 msgr2=0x7fdf9019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf90102330 0x7fdf9019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2372144221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840095f0 con 0x7fdf901007a0 2023-10-18T09:35:32.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e970 tx=0x7fdf8000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8d7fa700 1 -- 172.21.15.61:0/2372144221 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80009790 con 0x7fdf901007a0 2023-10-18T09:35:32.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8d7fa700 1 -- 172.21.15.61:0/2372144221 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8000c7e0 con 0x7fdf901007a0 2023-10-18T09:35:32.685 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901a0a00 con 0x7fdf901007a0 2023-10-18T09:35:32.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf8d7fa700 1 -- 172.21.15.61:0/2372144221 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80003c70 con 0x7fdf901007a0 2023-10-18T09:35:32.686 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.676+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf901a0f60 con 0x7fdf901007a0 2023-10-18T09:35:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.679+0000 7fdf8d7fa700 1 -- 172.21.15.61:0/2372144221 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdf8000cc50 con 0x7fdf901007a0 2023-10-18T09:35:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 Processor -- start 2023-10-18T09:35:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 -- start start 2023-10-18T09:35:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdf8d7fa700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 0x7fdf780a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.691 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdf8d7fa700 1 -- 172.21.15.61:0/2372144221 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdf80014070 con 0x7fdf901007a0 2023-10-18T09:35:32.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf24928c30 0x55cf24927090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf24927a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf2489a6e0 con 0x55cf249275d0 2023-10-18T09:35:32.692 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24880c60 con 0x55cf249282d0 2023-10-18T09:35:32.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.681+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf2487e580 con 0x55cf24928c30 2023-10-18T09:35:32.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 0x7fdf780a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47656/0 (socket says 172.21.15.61:47656) 2023-10-18T09:35:32.693 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2371486192 learned_addr learned my addr 172.21.15.61:0/2371486192 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf24927a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2371486192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf24928c30 msgr2=0x55cf24927090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf24928c30 0x55cf24927090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2371486192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 msgr2=0x55cf24927a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf24927a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2371486192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf248373f0 con 0x55cf249282d0 2023-10-18T09:35:32.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf24927a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:32.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf80096bc0 tx=0x7fdf8000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1803e69c70231906 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 0x7fdf780a39c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf88007950 tx=0x7fdf8800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf767fc700 1 -- 172.21.15.61:0/2371486192 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8009a150 con 0x55cf249282d0 2023-10-18T09:35:32.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf767fc700 1 -- 172.21.15.61:0/2371486192 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8009e2a0 con 0x55cf249282d0 2023-10-18T09:35:32.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.682+0000 7fdf767fc700 1 -- 172.21.15.61:0/2371486192 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8009a3b0 con 0x55cf249282d0 2023-10-18T09:35:32.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2371486192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 msgr2=0x55cf249286f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdf80096bc0 tx=0x7fdf8000cf50 comp rx=0 tx=0).stop 2023-10-18T09:35:32.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2371486192 shutdown_connections 2023-10-18T09:35:32.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf24927a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf249282d0 0x55cf249286f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2371486192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf24928c30 0x55cf24927090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2371486192 >> 172.21.15.61:0/2371486192 conn(0x55cf2489b720 msgr2=0x55cf24921b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2371486192 shutdown_connections 2023-10-18T09:35:32.700 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.683+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2371486192 wait complete. 2023-10-18T09:35:32.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 -- start start 2023-10-18T09:35:32.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 Processor -- start 2023-10-18T09:35:32.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24930b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf24931060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24879e70 con 0x55cf249282d0 2023-10-18T09:35:32.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24816710 con 0x55cf24928c30 2023-10-18T09:35:32.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.684+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf248163b0 con 0x55cf249275d0 2023-10-18T09:35:32.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdfaa293b80 10 client.0 _ll_register_callbacks cb 0x55cf24760a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:35:32.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24930b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47664/0 (socket says 172.21.15.61:47664) 2023-10-18T09:35:32.705 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 -- 172.21.15.61:0/4124062952 learned_addr learned my addr 172.21.15.61:0/4124062952 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf24931060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.706 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 -- 172.21.15.61:0/4124062952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 msgr2=0x55cf24930b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24930b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 -- 172.21.15.61:0/4124062952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 msgr2=0x55cf24931060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf24931060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.707 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 -- 172.21.15.61:0/4124062952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf901a0f60 con 0x55cf24928c30 2023-10-18T09:35:32.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009550 tx=0x7fdf88004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0926c1a77be229 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.708 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.685+0000 7fdf6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2023-10-18T09:35:32.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8801f070 con 0x55cf24928c30 2023-10-18T09:35:32.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf2487eef0 con 0x55cf24928c30 2023-10-18T09:35:32.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mon.2 v2:172.21.15.110:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdf88007500 con 0x55cf24928c30 2023-10-18T09:35:32.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf88017dd0 con 0x55cf24928c30 2023-10-18T09:35:32.709 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 10 client.26561 fetch_fsmap learned FSMap version 46 2023-10-18T09:35:32.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cf246a4de0 con 0x55cf24928c30 2023-10-18T09:35:32.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mon.2 v2:172.21.15.110:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf2487eef0 con 0x55cf24928c30 2023-10-18T09:35:32.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mon.2 v2:172.21.15.110:3300/0 5 ==== fsmap(e 46) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdf88025b90 con 0x55cf24928c30 2023-10-18T09:35:32.710 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 10 client.26561 fetch_fsmap finished waiting for FSMap version 46 2023-10-18T09:35:32.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:35:32.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 20 client.26561 populate_metadata read hostname 'smithi061' 2023-10-18T09:35:32.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf2493ca60 0x55cf2493ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 4 client.26561 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-18T09:35:32.711 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.686+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf248f68b0 con 0x55cf2493ca60 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.687+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf2493ca60 0x55cf2493ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.687+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf2493ca60 0x55cf2493ee80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5e00500157c72c server_cookie=37610cdeaba491a in_seq=0 out_seq=0 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.687+0000 7fdf6f7fe700 10 client.26561 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/4124062952 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55cf248f68b0 con 0x55cf2493ca60 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdf6f7fe700 10 client.26561 handle_command_reply: tid=0 2023-10-18T09:35:32.712 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 client.26561 shutdown 2023-10-18T09:35:32.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 msgr2=0x55cf249380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009550 tx=0x7fdf88004b40 comp rx=0 tx=0).stop 2023-10-18T09:35:32.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 shutdown_connections 2023-10-18T09:35:32.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf2493ca60 0x55cf2493ee80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.713 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf249380a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24930b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdf6f7fe700 0 client.26561 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:32.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/4124062952 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf24931060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 >> 172.21.15.61:0/4124062952 conn(0x55cf2489b720 msgr2=0x55cf24926080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.714 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 shutdown_connections 2023-10-18T09:35:32.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 1 -- 172.21.15.61:0/4124062952 wait complete. 2023-10-18T09:35:32.715 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.688+0000 7fdfaa293b80 20 client.26561 trim_cache size 0 max 16384 2023-10-18T09:35:32.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 Processor -- start 2023-10-18T09:35:32.720 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 -- start start 2023-10-18T09:35:32.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 0x55cf24938060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf2492b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf246fa920 con 0x55cf249275d0 2023-10-18T09:35:32.721 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf2491a0d0 con 0x55cf24928c30 2023-10-18T09:35:32.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24939220 con 0x55cf249282d0 2023-10-18T09:35:32.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdf8ffff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:60282/0 (socket says 172.21.15.61:60282) 2023-10-18T09:35:32.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2859898216 learned_addr learned my addr 172.21.15.61:0/2859898216 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf2492b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.706+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 0x55cf24938060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2859898216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 msgr2=0x55cf24938060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 0x55cf24938060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2859898216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 msgr2=0x55cf2492b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf2492b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 -- 172.21.15.61:0/2859898216 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf2468dc60 con 0x55cf249275d0 2023-10-18T09:35:32.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 0x55cf24938060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:32.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf2492b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:32.724 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdf80099e50 tx=0x7fdf800984f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca715a22861caf server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf76ffd700 1 -- 172.21.15.61:0/2859898216 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800985d0 con 0x55cf249275d0 2023-10-18T09:35:32.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf76ffd700 1 -- 172.21.15.61:0/2859898216 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf80098310 con 0x55cf249275d0 2023-10-18T09:35:32.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.707+0000 7fdf76ffd700 1 -- 172.21.15.61:0/2859898216 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8009a3b0 con 0x55cf249275d0 2023-10-18T09:35:32.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2859898216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 msgr2=0x55cf2492b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdf80099e50 tx=0x7fdf800984f0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2859898216 shutdown_connections 2023-10-18T09:35:32.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf2492b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249275d0 0x55cf2492b2d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/2859898216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249282d0 0x55cf24938060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2859898216 >> 172.21.15.61:0/2859898216 conn(0x55cf2489b720 msgr2=0x55cf2492eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2859898216 shutdown_connections 2023-10-18T09:35:32.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- 172.21.15.61:0/2859898216 wait complete. 2023-10-18T09:35:32.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.708+0000 7fdfaa293b80 1 -- start start 2023-10-18T09:35:32.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 Processor -- start 2023-10-18T09:35:32.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24937f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf249366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24937320 con 0x55cf249282d0 2023-10-18T09:35:32.728 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24937490 con 0x55cf24928c30 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf24990880 con 0x55cf249275d0 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.709+0000 7fdfaa293b80 10 client.0 _ll_register_callbacks cb 0x55cf24760a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf249366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf94f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf94f9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47674/0 (socket says 172.21.15.61:47674) 2023-10-18T09:35:32.729 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf94f9c700 1 -- 172.21.15.61:0/3605334634 learned_addr learned my addr 172.21.15.61:0/3605334634 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24937f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 -- 172.21.15.61:0/3605334634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 msgr2=0x55cf24937f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24937f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.730 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 -- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 msgr2=0x55cf24936c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-18T09:35:32.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 -- 172.21.15.61:0/3605334634 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf248373f0 con 0x55cf249282d0 2023-10-18T09:35:32.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24937f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:32.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf94f9c700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:35:32.731 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf8f7fe700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf249366f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098e0 tx=0x7fdf840057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e4ebbb0cd042ff server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.710+0000 7fdf6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.61:3300/0 2023-10-18T09:35:32.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400be10 con 0x55cf249282d0 2023-10-18T09:35:32.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf8009e2a0 con 0x55cf249282d0 2023-10-18T09:35:32.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mon.0 v2:172.21.15.61:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdf840037d0 con 0x55cf249282d0 2023-10-18T09:35:32.732 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf84003990 con 0x55cf249282d0 2023-10-18T09:35:32.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 10 client.16326 fetch_fsmap learned FSMap version 46 2023-10-18T09:35:32.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55cf246cb250 con 0x55cf249282d0 2023-10-18T09:35:32.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mon.0 v2:172.21.15.61:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8009e2a0 con 0x55cf249282d0 2023-10-18T09:35:32.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mon.0 v2:172.21.15.61:3300/0 5 ==== fsmap(e 46) v1 ==== 7910+0+0 (secure 0 0 0) 0x7fdf84024c80 con 0x55cf249282d0 2023-10-18T09:35:32.733 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 10 client.16326 fetch_fsmap finished waiting for FSMap version 46 2023-10-18T09:35:32.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-18T09:35:32.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 20 client.16326 populate_metadata read hostname 'smithi061' 2023-10-18T09:35:32.734 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf24992330 0x55cf24994750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 4 client.16326 mds_command: new command op to 24692 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-18T09:35:32.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.711+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 --> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cf24872e50 con 0x55cf24992330 2023-10-18T09:35:32.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf24992330 0x55cf24994750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf8ffff700 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf24992330 0x55cf24994750 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7618d603d8b3aa8a server_cookie=c83e2aaab1d0f2ac in_seq=0 out_seq=0 2023-10-18T09:35:32.735 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf6f7fe700 10 client.16326 ms_handle_connect on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:32.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf6f7fe700 1 -- 172.21.15.61:0/3605334634 <== mds.0 v2:172.21.15.110:6838/1483417844 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cf24872e50 con 0x55cf24992330 2023-10-18T09:35:32.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf6f7fe700 10 client.16326 handle_command_reply: tid=0 2023-10-18T09:35:32.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 client.16326 shutdown 2023-10-18T09:35:32.736 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 msgr2=0x55cf249366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf249366f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdf840098e0 tx=0x7fdf840057d0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 shutdown_connections 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:6838/1483417844,v1:172.21.15.110:6839/1483417844] conn(0x55cf24992330 0x55cf24994750 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cf24928c30 0x55cf24936c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf249275d0 0x55cf24937f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.737 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 --2- 172.21.15.61:0/3605334634 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x55cf249282d0 0x55cf249366f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdf6f7fe700 0 client.16326 ms_handle_reset on v2:172.21.15.110:6838/1483417844 2023-10-18T09:35:32.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.712+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 >> 172.21.15.61:0/3605334634 conn(0x55cf2489b720 msgr2=0x55cf2492b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 shutdown_connections 2023-10-18T09:35:32.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdfaa293b80 1 -- 172.21.15.61:0/3605334634 wait complete. 2023-10-18T09:35:32.738 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdfaa293b80 20 client.16326 trim_cache size 0 max 16384 2023-10-18T09:35:32.739 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:32.739 INFO:teuthology.orchestra.run.smithi061.stdout: "status": "scrub active (1 inodes in the stack)", 2023-10-18T09:35:32.739 INFO:teuthology.orchestra.run.smithi061.stdout: "scrubs": { 2023-10-18T09:35:32.740 INFO:teuthology.orchestra.run.smithi061.stdout: "d1454a57-d3e0-4614-9ecd-13610eb47bb9": { 2023-10-18T09:35:32.740 INFO:teuthology.orchestra.run.smithi061.stdout: "path": "/", 2023-10-18T09:35:32.740 INFO:teuthology.orchestra.run.smithi061.stdout: "tag": "d1454a57-d3e0-4614-9ecd-13610eb47bb9", 2023-10-18T09:35:32.740 INFO:teuthology.orchestra.run.smithi061.stdout: "options": "recursive,force" 2023-10-18T09:35:32.741 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:32.741 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:32.741 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:32.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 msgr2=0x7fdf780a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 0x7fdf780a39c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdf88007950 tx=0x7fdf8800b040 comp rx=0 tx=0).stop 2023-10-18T09:35:32.744 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 msgr2=0x7fdf90198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e970 tx=0x7fdf8000ec80 comp rx=0 tx=0).stop 2023-10-18T09:35:32.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 shutdown_connections 2023-10-18T09:35:32.745 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdf780a1500 0x7fdf780a39c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf90102330 0x7fdf9019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf901007a0 0x7fdf90198920 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.746 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 --2- 172.21.15.61:0/2372144221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf90101990 0x7fdf90198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.713+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 >> 172.21.15.61:0/2372144221 conn(0x7fdf900fbcf0 msgr2=0x7fdf90107180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.747 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.714+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 shutdown_connections 2023-10-18T09:35:32.748 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.714+0000 7fdf969ff700 1 -- 172.21.15.61:0/2372144221 wait complete. 2023-10-18T09:35:32.748 INFO:tasks.cephfs.filesystem:scrub status for tag:d1454a57-d3e0-4614-9ecd-13610eb47bb9 - {'path': '/', 'tag': 'd1454a57-d3e0-4614-9ecd-13610eb47bb9', 'options': 'recursive,force'} 2023-10-18T09:35:32.749 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/cephfs/filesystem.py", line 1730, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-10-18T09:35:32.749 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_f7420d7f8e55ceea786dcbf536c338ce920c21eb/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-10-18T09:35:32.750 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-18T09:35:32.763 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:35:32.971 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/4204624907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf041022d0 msgr2=0x7fdf04108b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/4204624907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf041022d0 0x7fdf04108b80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00b0d0 tx=0x7fdefc00b3e0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/4204624907 shutdown_connections 2023-10-18T09:35:32.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/4204624907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf041022d0 0x7fdf04108b80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/4204624907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf04100740 0x7fdf04100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.972 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/4204624907 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04101930 0x7fdf04101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/4204624907 >> 172.21.15.61:0/4204624907 conn(0x7fdf040fbc90 msgr2=0x7fdf040fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:32.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/4204624907 shutdown_connections 2023-10-18T09:35:32.973 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.969+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/4204624907 wait complete. 2023-10-18T09:35:32.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 Processor -- start 2023-10-18T09:35:32.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 -- start start 2023-10-18T09:35:32.975 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04100740 0x7fdf041989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 0x7fdf0419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a04f0 con 0x7fdf04100740 2023-10-18T09:35:32.976 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a0630 con 0x7fdf04101930 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0b3ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041a07c0 con 0x7fdf041022d0 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0899a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0899a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47706/0 (socket says 172.21.15.61:47706) 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0919b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04100740 0x7fdf041989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.970+0000 7fdf0899a700 1 -- 172.21.15.61:0/2687228063 learned_addr learned my addr 172.21.15.61:0/2687228063 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:32.977 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0999c700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 0x7fdf0419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 -- 172.21.15.61:0/2687228063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 msgr2=0x7fdf0419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 0x7fdf0419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 -- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04100740 msgr2=0x7fdf041989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:32.978 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04100740 0x7fdf041989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:32.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 -- 172.21.15.61:0/2687228063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009d00 con 0x7fdf04101930 2023-10-18T09:35:32.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0999c700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 0x7fdf0419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:35:32.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0899a700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9a0 tx=0x7fdef400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4009790 con 0x7fdf04101930 2023-10-18T09:35:32.979 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf041a0aa0 con 0x7fdf04101930 2023-10-18T09:35:32.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef400cbd0 con 0x7fdf04101930 2023-10-18T09:35:32.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4018680 con 0x7fdf04101930 2023-10-18T09:35:32.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.971+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf041a1000 con 0x7fdf04101930 2023-10-18T09:35:32.980 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.972+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04067530 con 0x7fdf04101930 2023-10-18T09:35:32.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.974+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fdef400cd40 con 0x7fdf04101930 2023-10-18T09:35:32.981 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.976+0000 7fdefa7fc700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 0x7fdef00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:32.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.976+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdef4014070 con 0x7fdf04101930 2023-10-18T09:35:32.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.976+0000 7fdf0919b700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 0x7fdef00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:32.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.976+0000 7fdf0919b700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 0x7fdef00a3940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf04101790 tx=0x7fdf00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:32.982 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:32.978+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdef40968b0 con 0x7fdf04101930 2023-10-18T09:35:33.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.191+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf0410bfb0 con 0x7fdf04101930 2023-10-18T09:35:33.195 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.193+0000 7fdefa7fc700 1 -- 172.21.15.61:0/2687228063 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fdef4099ed0 con 0x7fdf04101930 2023-10-18T09:35:33.196 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:35:33.196 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:35:33.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.195+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 msgr2=0x7fdef00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.195+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 0x7fdef00a3940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf04101790 tx=0x7fdf00009260 comp rx=0 tx=0).stop 2023-10-18T09:35:33.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.195+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 msgr2=0x7fdf04198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.195+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e9a0 tx=0x7fdef400ecb0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 shutdown_connections 2023-10-18T09:35:33.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fdef00a1480 0x7fdef00a3940 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf041022d0 0x7fdf0419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fdf04100740 0x7fdf041989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 --2- 172.21.15.61:0/2687228063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf04101930 0x7fdf04198f00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 >> 172.21.15.61:0/2687228063 conn(0x7fdf040fbc90 msgr2=0x7fdf04072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:33.200 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 shutdown_connections 2023-10-18T09:35:33.201 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.196+0000 7fdf0b3ff700 1 -- 172.21.15.61:0/2687228063 wait complete. 2023-10-18T09:35:33.218 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:35:33.218 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:35:33.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 -- 172.21.15.61:0/3356159489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14001007a0 msgr2=0x7f1400100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 --2- 172.21.15.61:0/3356159489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14001007a0 0x7f1400100bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0099c0 tx=0x7f13fc009cd0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 -- 172.21.15.61:0/3356159489 shutdown_connections 2023-10-18T09:35:33.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 --2- 172.21.15.61:0/3356159489 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1400102330 0x7f1400108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 --2- 172.21.15.61:0/3356159489 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14001007a0 0x7f1400100bc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 --2- 172.21.15.61:0/3356159489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 0x7f1400101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 -- 172.21.15.61:0/3356159489 >> 172.21.15.61:0/3356159489 conn(0x7f14000fbcf0 msgr2=0x7f14000fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:33.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 -- 172.21.15.61:0/3356159489 shutdown_connections 2023-10-18T09:35:33.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.424+0000 7f14073ff700 1 -- 172.21.15.61:0/3356159489 wait complete. 2023-10-18T09:35:33.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.425+0000 7f14073ff700 1 Processor -- start 2023-10-18T09:35:33.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 -- start start 2023-10-18T09:35:33.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 0x7f1400072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1400102330 0x7f1400073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000738c0 con 0x7f14001007a0 2023-10-18T09:35:33.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1400073a30 con 0x7f1400102330 2023-10-18T09:35:33.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f14073ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14001a0790 con 0x7f1400101990 2023-10-18T09:35:33.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f140519b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f140519b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:60336/0 (socket says 172.21.15.61:60336) 2023-10-18T09:35:33.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f140519b700 1 -- 172.21.15.61:0/1529232838 learned_addr learned my addr 172.21.15.61:0/1529232838 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:33.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f140499a700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 0x7f1400072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.426+0000 7f140599c700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1400102330 0x7f1400073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 -- 172.21.15.61:0/1529232838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 msgr2=0x7f1400072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 0x7f1400072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 -- 172.21.15.61:0/1529232838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1400102330 msgr2=0x7f1400073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1400102330 0x7f1400073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 -- 172.21.15.61:0/1529232838 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0095f0 con 0x7f14001007a0 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f140519b700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13fc003730 tx=0x7f13fc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:33.434 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc01d070 con 0x7f14001007a0 2023-10-18T09:35:33.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14001a09c0 con 0x7f14001007a0 2023-10-18T09:35:33.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13fc00bda0 con 0x7f14001007a0 2023-10-18T09:35:33.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0173f0 con 0x7f14001007a0 2023-10-18T09:35:33.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.427+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001a0de0 con 0x7f14001007a0 2023-10-18T09:35:33.435 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.428+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400107c70 con 0x7f14001007a0 2023-10-18T09:35:33.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.430+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f13fc022470 con 0x7f14001007a0 2023-10-18T09:35:33.436 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.431+0000 7f13f67fc700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 0x7f13ec0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.432+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f13fc0ce4d0 con 0x7f14001007a0 2023-10-18T09:35:33.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.432+0000 7f140499a700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 0x7f13ec0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.433+0000 7f140499a700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 0x7f13ec0a3940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f13f0005950 tx=0x7f13f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:33.437 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.435+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f13fc097810 con 0x7f14001007a0 2023-10-18T09:35:33.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:33 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2687228063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:35:33.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:33 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2687228063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:35:33.647 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.645+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f140010c110 con 0x7f14001007a0 2023-10-18T09:35:33.648 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.645+0000 7f13f67fc700 1 -- 172.21.15.61:0/1529232838 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13fc09ae30 con 0x7f14001007a0 2023-10-18T09:35:33.648 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:35:33.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 msgr2=0x7f13ec0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.650 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 0x7f13ec0a3940 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f13f0005950 tx=0x7f13f000b410 comp rx=0 tx=0).stop 2023-10-18T09:35:33.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 msgr2=0x7f1400074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f13fc003730 tx=0x7f13fc00fa00 comp rx=0 tx=0).stop 2023-10-18T09:35:33.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 shutdown_connections 2023-10-18T09:35:33.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f13ec0a1480 0x7f13ec0a3940 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.651 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1400102330 0x7f1400073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f14001007a0 0x7f1400074550 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 --2- 172.21.15.61:0/1529232838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1400101990 0x7f1400072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 >> 172.21.15.61:0/1529232838 conn(0x7f14000fbcf0 msgr2=0x7f1400104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 shutdown_connections 2023-10-18T09:35:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.648+0000 7f14073ff700 1 -- 172.21.15.61:0/1529232838 wait complete. 2023-10-18T09:35:33.669 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:35:33.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:33 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2687228063' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:35:33.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/4258387804 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d681007a0 msgr2=0x7f5d68100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/4258387804 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d681007a0 0x7f5d68100bc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009920 tx=0x7f5d58009c30 comp rx=0 tx=0).stop 2023-10-18T09:35:33.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/4258387804 shutdown_connections 2023-10-18T09:35:33.881 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/4258387804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d68102330 0x7f5d68108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/4258387804 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d681007a0 0x7f5d68100bc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/4258387804 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/4258387804 >> 172.21.15.61:0/4258387804 conn(0x7f5d680fbcf0 msgr2=0x7f5d680fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:33.882 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/4258387804 shutdown_connections 2023-10-18T09:35:33.883 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.878+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/4258387804 wait complete. 2023-10-18T09:35:33.885 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.879+0000 7f5d6c9ff700 1 Processor -- start 2023-10-18T09:35:33.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 -- start start 2023-10-18T09:35:33.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d681007a0 0x7f5d68074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d68102330 0x7f5d68073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.886 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68073820 con 0x7f5d68102330 2023-10-18T09:35:33.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68073990 con 0x7f5d68101990 2023-10-18T09:35:33.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d6c9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681a08d0 con 0x7f5d681007a0 2023-10-18T09:35:33.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47754/0 (socket says 172.21.15.61:47754) 2023-10-18T09:35:33.887 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d5ffff700 1 -- 172.21.15.61:0/3019370681 learned_addr learned my addr 172.21.15.61:0/3019370681 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:33.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.880+0000 7f5d66d9d700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d68102330 0x7f5d68073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d6659c700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d681007a0 0x7f5d68074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 -- 172.21.15.61:0/3019370681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d681007a0 msgr2=0x7f5d68074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d681007a0 0x7f5d68074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.888 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 -- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d68102330 msgr2=0x7f5d68073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:33.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d68102330 0x7f5d68073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:33.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 -- 172.21.15.61:0/3019370681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500095f0 con 0x7f5d68101990 2023-10-18T09:35:33.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5ffff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ec10 tx=0x7f5d5000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:33.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5000cc90 con 0x7f5d68101990 2023-10-18T09:35:33.889 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d58009520 con 0x7f5d68101990 2023-10-18T09:35:33.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d500044e0 con 0x7f5d68101990 2023-10-18T09:35:33.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50010430 con 0x7f5d68101990 2023-10-18T09:35:33.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.881+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681a0f00 con 0x7f5d68101990 2023-10-18T09:35:33.890 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.882+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d68107d10 con 0x7f5d68101990 2023-10-18T09:35:33.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.885+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f5d50003f90 con 0x7f5d68101990 2023-10-18T09:35:33.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.886+0000 7f5d5f7fe700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 0x7f5d540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:33.891 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.886+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5d50014070 con 0x7f5d68101990 2023-10-18T09:35:33.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.886+0000 7f5d6659c700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 0x7f5d540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:33.892 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.887+0000 7f5d6659c700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 0x7f5d540a3940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098f0 tx=0x7f5d58019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:33.893 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:33.890+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d50096a10 con 0x7f5d68101990 2023-10-18T09:35:34.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.104+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d6810c000 con 0x7f5d68101990 2023-10-18T09:35:34.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.104+0000 7f5d5f7fe700 1 -- 172.21.15.61:0/3019370681 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d50017e90 con 0x7f5d68101990 2023-10-18T09:35:34.107 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:35:34.109 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 msgr2=0x7f5d540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 0x7f5d540a3940 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5d580098f0 tx=0x7f5d58019040 comp rx=0 tx=0).stop 2023-10-18T09:35:34.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 msgr2=0x7f5d68072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000ec10 tx=0x7f5d5000ef20 comp rx=0 tx=0).stop 2023-10-18T09:35:34.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 shutdown_connections 2023-10-18T09:35:34.110 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f5d540a1480 0x7f5d540a3940 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f5d68102330 0x7f5d68073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d681007a0 0x7f5d68074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 --2- 172.21.15.61:0/3019370681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d68101990 0x7f5d68072be0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 >> 172.21.15.61:0/3019370681 conn(0x7f5d680fbcf0 msgr2=0x7f5d68104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:34.111 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 shutdown_connections 2023-10-18T09:35:34.112 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.107+0000 7f5d6c9ff700 1 -- 172.21.15.61:0/3019370681 wait complete. 2023-10-18T09:35:34.129 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:35:34.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 -- 172.21.15.61:0/2547569474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8101990 msgr2=0x7efeb8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.339 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 --2- 172.21.15.61:0/2547569474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8101990 0x7efeb8101df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7efea80099c0 tx=0x7efea8009cd0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 -- 172.21.15.61:0/2547569474 shutdown_connections 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 --2- 172.21.15.61:0/2547569474 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8102330 0x7efeb8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 --2- 172.21.15.61:0/2547569474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 0x7efeb8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 --2- 172.21.15.61:0/2547569474 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8101990 0x7efeb8101df0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 -- 172.21.15.61:0/2547569474 >> 172.21.15.61:0/2547569474 conn(0x7efeb80fbcf0 msgr2=0x7efeb80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:34.340 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 -- 172.21.15.61:0/2547569474 shutdown_connections 2023-10-18T09:35:34.341 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.336+0000 7efebfbff700 1 -- 172.21.15.61:0/2547569474 wait complete. 2023-10-18T09:35:34.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.337+0000 7efebfbff700 1 Processor -- start 2023-10-18T09:35:34.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.337+0000 7efebfbff700 1 -- start start 2023-10-18T09:35:34.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 0x7efeb8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.343 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 0x7efeb8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e240 con 0x7efeb8101990 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e380 con 0x7efeb8102330 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebfbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb819e510 con 0x7efeb81007a0 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebd19a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 0x7efeb8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.344 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47776/0 (socket says 172.21.15.61:47776) 2023-10-18T09:35:34.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 -- 172.21.15.61:0/1465377122 learned_addr learned my addr 172.21.15.61:0/1465377122 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:34.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebd99b700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 0x7efeb8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 -- 172.21.15.61:0/1465377122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 msgr2=0x7efeb8196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.345 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 0x7efeb8196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 -- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 msgr2=0x7efeb8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 0x7efeb8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.338+0000 7efebe19c700 1 -- 172.21.15.61:0/1465377122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea80095f0 con 0x7efeb8102330 2023-10-18T09:35:34.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efebd19a700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 0x7efeb8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:35:34.346 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efebe19c700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef30 tx=0x7efeb000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:34.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0018070 con 0x7efeb8102330 2023-10-18T09:35:34.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeb000f040 con 0x7efeb8102330 2023-10-18T09:35:34.347 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb819e790 con 0x7efeb8102330 2023-10-18T09:35:34.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb000cd00 con 0x7efeb8102330 2023-10-18T09:35:34.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.339+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb819ec50 con 0x7efeb8102330 2023-10-18T09:35:34.348 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.340+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb8107c70 con 0x7efeb8102330 2023-10-18T09:35:34.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.342+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7efeb0003a00 con 0x7efeb8102330 2023-10-18T09:35:34.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.343+0000 7efeaeffd700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 0x7efea40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.349 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.344+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efeb00cd550 con 0x7efeb8102330 2023-10-18T09:35:34.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.344+0000 7efebd99b700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 0x7efea40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.344+0000 7efebd99b700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 0x7efea40a3940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeb81017f0 tx=0x7efeb4007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:34.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.346+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7efeb0096890 con 0x7efeb8102330 2023-10-18T09:35:34.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:34 smithi061 ceph-mon[98573]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-18T09:35:34.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1529232838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:34 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3019370681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.561 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.559+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efeb810c110 con 0x7efeb8102330 2023-10-18T09:35:34.562 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.559+0000 7efeaeffd700 1 -- 172.21.15.61:0/1465377122 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7efeb0099eb0 con 0x7efeb8102330 2023-10-18T09:35:34.562 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:35:34.564 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 msgr2=0x7efea40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 0x7efea40a3940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efeb81017f0 tx=0x7efeb4007400 comp rx=0 tx=0).stop 2023-10-18T09:35:34.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 msgr2=0x7efeb819dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efeb000ef30 tx=0x7efeb000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.565 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 shutdown_connections 2023-10-18T09:35:34.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7efea40a1480 0x7efea40a3940 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efeb8102330 0x7efeb819dc70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efeb81007a0 0x7efeb8196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 --2- 172.21.15.61:0/1465377122 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7efeb8101990 0x7efeb8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.566 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 >> 172.21.15.61:0/1465377122 conn(0x7efeb80fbcf0 msgr2=0x7efeb8104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:34.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 shutdown_connections 2023-10-18T09:35:34.567 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.562+0000 7efebfbff700 1 -- 172.21.15.61:0/1465377122 wait complete. 2023-10-18T09:35:34.583 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:35:34.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:34 smithi170 ceph-mon[135150]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-18T09:35:34.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:34 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1529232838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:34 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3019370681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:34 smithi110 ceph-mon[134818]: pgmap v868: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 938 B/s wr, 10 op/s 2023-10-18T09:35:34.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:34 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1529232838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:34 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3019370681' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:35:34.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 -- 172.21.15.61:0/1825278146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f41007a0 msgr2=0x7f11f4100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.793 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 --2- 172.21.15.61:0/1825278146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f41007a0 0x7f11f4100bc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f11e40099c0 tx=0x7f11e4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 -- 172.21.15.61:0/1825278146 shutdown_connections 2023-10-18T09:35:34.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 --2- 172.21.15.61:0/1825278146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4102330 0x7f11f4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 --2- 172.21.15.61:0/1825278146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f41007a0 0x7f11f4100bc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 --2- 172.21.15.61:0/1825278146 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f4101990 0x7f11f4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.794 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 -- 172.21.15.61:0/1825278146 >> 172.21.15.61:0/1825278146 conn(0x7f11f40fbcf0 msgr2=0x7f11f40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:34.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 -- 172.21.15.61:0/1825278146 shutdown_connections 2023-10-18T09:35:34.795 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.790+0000 7f11f97ff700 1 -- 172.21.15.61:0/1825278146 wait complete. 2023-10-18T09:35:34.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.791+0000 7f11f97ff700 1 Processor -- start 2023-10-18T09:35:34.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.791+0000 7f11f97ff700 1 -- start start 2023-10-18T09:35:34.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.797 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4101990 0x7f11f4072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 0x7f11f4073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4073820 con 0x7f11f41007a0 2023-10-18T09:35:34.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f4073990 con 0x7f11f4102330 2023-10-18T09:35:34.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f97ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41a0810 con 0x7f11f4101990 2023-10-18T09:35:34.798 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 0x7f11f4073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:60388/0 (socket says 172.21.15.61:60388) 2023-10-18T09:35:34.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 -- 172.21.15.61:0/2298346458 learned_addr learned my addr 172.21.15.61:0/2298346458 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:34.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f27fc700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4101990 0x7f11f4072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.799 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 -- 172.21.15.61:0/2298346458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4101990 msgr2=0x7f11f4072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4101990 0x7f11f4072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 -- 172.21.15.61:0/2298346458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 msgr2=0x7f11f4073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:34.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 0x7f11f4073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:34.800 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.792+0000 7f11f2ffd700 1 -- 172.21.15.61:0/2298346458 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40095f0 con 0x7f11f41007a0 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11f37fe700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 0x7f11f4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11f2ffd700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003730 tx=0x7f11e400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e401d070 con 0x7f11f41007a0 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e400bda0 con 0x7f11f41007a0 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4004b30 con 0x7f11f41007a0 2023-10-18T09:35:34.801 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f41a0a10 con 0x7f11f41007a0 2023-10-18T09:35:34.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.793+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f41a0e30 con 0x7f11f41007a0 2023-10-18T09:35:34.802 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.794+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4107c70 con 0x7f11f41007a0 2023-10-18T09:35:34.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.796+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f11e4004c90 con 0x7f11f41007a0 2023-10-18T09:35:34.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.797+0000 7f11dbfff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 0x7f11dc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:34.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.798+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f11e40cdd90 con 0x7f11f41007a0 2023-10-18T09:35:34.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.798+0000 7f11f27fc700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 0x7f11dc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:34.803 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.798+0000 7f11f27fc700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 0x7f11dc0a39c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009c60 tx=0x7f11e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:34.804 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:34.802+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f11e4097130 con 0x7f11f41007a0 2023-10-18T09:35:35.017 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.014+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f11f402cf60 con 0x7f11f41007a0 2023-10-18T09:35:35.018 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.015+0000 7f11dbfff700 1 -- 172.21.15.61:0/2298346458 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f11e409a750 con 0x7f11f41007a0 2023-10-18T09:35:35.018 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:35:35.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.017+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 msgr2=0x7f11dc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:35.020 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.017+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 0x7f11dc0a39c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009c60 tx=0x7f11e8009260 comp rx=0 tx=0).stop 2023-10-18T09:35:35.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 msgr2=0x7f11f4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:35.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003730 tx=0x7f11e400fa00 comp rx=0 tx=0).stop 2023-10-18T09:35:35.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 shutdown_connections 2023-10-18T09:35:35.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f11dc0a1500 0x7f11dc0a39c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.021 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11f4102330 0x7f11f4073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f11f41007a0 0x7f11f4074500 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 --2- 172.21.15.61:0/2298346458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11f4101990 0x7f11f4072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 >> 172.21.15.61:0/2298346458 conn(0x7f11f40fbcf0 msgr2=0x7f11f4104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:35.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 shutdown_connections 2023-10-18T09:35:35.022 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.018+0000 7f11f97ff700 1 -- 172.21.15.61:0/2298346458 wait complete. 2023-10-18T09:35:35.039 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-18T09:35:35.247 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 -- 172.21.15.61:0/2546414577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 msgr2=0x7f77f0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 --2- 172.21.15.61:0/2546414577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0072c20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009990 tx=0x7f77e0009ca0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 -- 172.21.15.61:0/2546414577 shutdown_connections 2023-10-18T09:35:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 --2- 172.21.15.61:0/2546414577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f0108610 0x7f77f010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.248 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 --2- 172.21.15.61:0/2546414577 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0072c20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 --2- 172.21.15.61:0/2546414577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f00731f0 0x7f77f0073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 -- 172.21.15.61:0/2546414577 >> 172.21.15.61:0/2546414577 conn(0x7f77f00fbc90 msgr2=0x7f77f00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 -- 172.21.15.61:0/2546414577 shutdown_connections 2023-10-18T09:35:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.244+0000 7f77f4dff700 1 -- 172.21.15.61:0/2546414577 wait complete. 2023-10-18T09:35:35.249 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 Processor -- start 2023-10-18T09:35:35.250 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 -- start start 2023-10-18T09:35:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f0108610 0x7f77f019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:35.252 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a03e0 con 0x7f77f00747d0 2023-10-18T09:35:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a0520 con 0x7f77f00731f0 2023-10-18T09:35:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.245+0000 7f77f4dff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f01a06b0 con 0x7f77f0108610 2023-10-18T09:35:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:47798/0 (socket says 172.21.15.61:47798) 2023-10-18T09:35:35.253 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 -- 172.21.15.61:0/730490239 learned_addr learned my addr 172.21.15.61:0/730490239 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:35:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77edd9b700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77eed9d700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f0108610 0x7f77f019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 -- 172.21.15.61:0/730490239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f0108610 msgr2=0x7f77f019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:35.254 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f0108610 0x7f77f019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 -- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 msgr2=0x7f77f0198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 -- 172.21.15.61:0/730490239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00095f0 con 0x7f77f00731f0 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.246+0000 7f77ee59c700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.247+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e001d070 con 0x7f77f00731f0 2023-10-18T09:35:35.255 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.247+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77e0021410 con 0x7f77f00731f0 2023-10-18T09:35:35.256 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.247+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f01a0930 con 0x7f77f00731f0 2023-10-18T09:35:35.256 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.247+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0017650 con 0x7f77f00731f0 2023-10-18T09:35:35.256 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.247+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f01a0d50 con 0x7f77f00731f0 2023-10-18T09:35:35.256 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.248+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f0107420 con 0x7f77f00731f0 2023-10-18T09:35:35.256 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.250+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f77e00217f0 con 0x7f77f00731f0 2023-10-18T09:35:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.251+0000 7f77df7fe700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 0x7f77d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:35:35.257 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.251+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f77e00cebe0 con 0x7f77f00731f0 2023-10-18T09:35:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.251+0000 7f77edd9b700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 0x7f77d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:35:35.258 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.252+0000 7f77edd9b700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 0x7f77d80a3940 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77e4007c60 tx=0x7f77e40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:35:35.259 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.255+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77e0097f80 con 0x7f77f00731f0 2023-10-18T09:35:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:35 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1465377122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:35:35.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:35 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2298346458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:35:35.531 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.529+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f77f010e170 con 0x7f77f00731f0 2023-10-18T09:35:35.541 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.539+0000 7f77df7fe700 1 -- 172.21.15.61:0/730490239 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733876 (secure 0 0 0) 0x7f77e009b5a0 con 0x7f77f00731f0 2023-10-18T09:35:35.552 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:35:35.552 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:35.552 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 18, 2023-10-18T09:35:35.552 INFO:teuthology.orchestra.run.smithi061.stdout: "active_gid": 14150, 2023-10-18T09:35:35.552 INFO:teuthology.orchestra.run.smithi061.stdout: "active_name": "x", 2023-10-18T09:35:35.553 INFO:teuthology.orchestra.run.smithi061.stdout: "active_addrs": { 2023-10-18T09:35:35.553 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-10-18T09:35:35.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.553 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-10-18T09:35:35.554 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6800", 2023-10-18T09:35:35.554 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3155127227 2023-10-18T09:35:35.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.555 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-10-18T09:35:35.556 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-10-18T09:35:35.556 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3155127227 2023-10-18T09:35:35.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.557 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:35.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.557 INFO:teuthology.orchestra.run.smithi061.stdout: "active_addr": "172.21.15.61:6801/3155127227", 2023-10-18T09:35:35.558 INFO:teuthology.orchestra.run.smithi061.stdout: "active_change": "2023-10-18T09:09:20.739876+0000", 2023-10-18T09:35:35.558 INFO:teuthology.orchestra.run.smithi061.stdout: "active_mgr_features": 4540138322906710015, 2023-10-18T09:35:35.558 INFO:teuthology.orchestra.run.smithi061.stdout: "available": true, 2023-10-18T09:35:35.559 INFO:teuthology.orchestra.run.smithi061.stdout: "standbys": [ 2023-10-18T09:35:35.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.559 INFO:teuthology.orchestra.run.smithi061.stdout: "gid": 24109, 2023-10-18T09:35:35.560 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "z", 2023-10-18T09:35:35.560 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_features": 4540138322906710015, 2023-10-18T09:35:35.560 INFO:teuthology.orchestra.run.smithi061.stdout: "available_modules": [ 2023-10-18T09:35:35.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.561 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alerts", 2023-10-18T09:35:35.562 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:35.562 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:35.562 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:35.563 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:35.563 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:35.563 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.564 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.564 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.564 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:35.565 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.565 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.565 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.565 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T09:35:35.566 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.566 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.566 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.567 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:35.567 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:35.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.568 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.568 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.568 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.569 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.570 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.570 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.570 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.570 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.571 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.571 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.571 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.571 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.571 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.572 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.572 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.572 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.572 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.573 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.574 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.574 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.574 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.574 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:35.575 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.576 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.576 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.576 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:35.576 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.577 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.578 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.578 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.578 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.578 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.578 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.579 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.579 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.579 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:35.579 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.580 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.581 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.581 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.581 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.581 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_destination": { 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_destination", 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.582 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.583 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.583 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.583 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.583 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.583 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email address to send alerts to", 2023-10-18T09:35:35.584 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.584 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.584 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.584 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_from_name": { 2023-10-18T09:35:35.585 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_from_name", 2023-10-18T09:35:35.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.585 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.586 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.586 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "Ceph", 2023-10-18T09:35:35.586 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.586 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.586 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email From: name", 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.587 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_host": { 2023-10-18T09:35:35.588 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_host", 2023-10-18T09:35:35.588 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.588 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.588 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.588 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.589 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.589 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.589 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.589 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", 2023-10-18T09:35:35.589 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.590 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.590 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.591 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_password": { 2023-10-18T09:35:35.591 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_password", 2023-10-18T09:35:35.591 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.592 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.592 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.592 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.593 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.593 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.593 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.593 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Password to authenticate with", 2023-10-18T09:35:35.594 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.594 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.594 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.595 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_port": { 2023-10-18T09:35:35.595 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_port", 2023-10-18T09:35:35.596 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.596 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.596 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.596 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "465", 2023-10-18T09:35:35.596 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP port", 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.597 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.598 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_sender": { 2023-10-18T09:35:35.598 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_sender", 2023-10-18T09:35:35.598 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.598 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.599 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP envelope sender", 2023-10-18T09:35:35.600 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.600 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.600 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_ssl": { 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_ssl", 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.601 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.602 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.602 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.602 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.602 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T09:35:35.602 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.603 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.603 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.603 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_user": { 2023-10-18T09:35:35.603 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_user", 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.604 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "User to authenticate as", 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.605 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.606 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "balancer", 2023-10-18T09:35:35.606 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:35.607 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:35.607 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:35.607 INFO:teuthology.orchestra.run.smithi061.stdout: "active": { 2023-10-18T09:35:35.607 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "active", 2023-10-18T09:35:35.607 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.608 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.609 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T09:35:35.609 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.609 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.609 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.610 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_time": { 2023-10-18T09:35:35.610 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_time", 2023-10-18T09:35:35.610 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.610 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.610 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T09:35:35.611 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:35.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.612 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.612 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_weekday": { 2023-10-18T09:35:35.612 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_weekday", 2023-10-18T09:35:35.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:35.613 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.613 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.613 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:35.613 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:35.614 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:35.614 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.614 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T09:35:35.614 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:35.614 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_max_iterations": { 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_max_iterations", 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:35.615 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.616 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.616 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "25", 2023-10-18T09:35:35.616 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:35.616 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "250", 2023-10-18T09:35:35.616 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.617 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T09:35:35.617 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.617 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.617 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_metrics": { 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_metrics", 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.618 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T09:35:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T09:35:35.619 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T09:35:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_step": { 2023-10-18T09:35:35.620 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_step", 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.5", 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0.001", 2023-10-18T09:35:35.621 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "0.999", 2023-10-18T09:35:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "aggressiveness of optimization", 2023-10-18T09:35:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T09:35:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.622 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.623 INFO:teuthology.orchestra.run.smithi061.stdout: "end_time": { 2023-10-18T09:35:35.623 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_time", 2023-10-18T09:35:35.623 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.623 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2359", 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.624 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T09:35:35.625 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:35.625 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.625 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.625 INFO:teuthology.orchestra.run.smithi061.stdout: "end_weekday": { 2023-10-18T09:35:35.626 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_weekday", 2023-10-18T09:35:35.626 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:35.626 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.626 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.626 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:35.627 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.628 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.628 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:35.628 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:35.628 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.629 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.629 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.629 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.629 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.629 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.630 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.630 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.630 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.631 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.631 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.631 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.632 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.632 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.632 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.633 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.633 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.633 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.634 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:35.634 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:35.635 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.635 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.636 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.636 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.636 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.637 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.637 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.637 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.637 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.638 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.638 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.638 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:35.639 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:35.639 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.639 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.639 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.639 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:35.640 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.640 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.640 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.640 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.640 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.641 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.641 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.641 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.641 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.641 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.642 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:35.643 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:35.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.643 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.643 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.645 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.645 INFO:teuthology.orchestra.run.smithi061.stdout: "min_score": { 2023-10-18T09:35:35.645 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_score", 2023-10-18T09:35:35.645 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.646 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T09:35:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.648 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "upmap", 2023-10-18T09:35:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-compat", 2023-10-18T09:35:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "none", 2023-10-18T09:35:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap" 2023-10-18T09:35:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Balancer mode", 2023-10-18T09:35:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_ids": { 2023-10-18T09:35:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_ids", 2023-10-18T09:35:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T09:35:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:35.655 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:35.655 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.655 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.656 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.656 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:35.656 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.657 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.657 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.657 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T09:35:35.657 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.658 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.658 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.659 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_deviation": { 2023-10-18T09:35:35.659 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_deviation", 2023-10-18T09:35:35.659 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.660 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:35.660 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:35.661 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.661 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.661 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T09:35:35.662 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T09:35:35.662 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.662 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.663 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_optimizations": { 2023-10-18T09:35:35.663 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_optimizations", 2023-10-18T09:35:35.663 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:35.664 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.664 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.664 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.665 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.665 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.665 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.666 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T09:35:35.666 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.666 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.666 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.667 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.667 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.668 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm", 2023-10-18T09:35:35.668 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:35.668 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:35.669 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:35.669 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_down_multiplier": { 2023-10-18T09:35:35.669 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_down_multiplier", 2023-10-18T09:35:35.669 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:35.670 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.670 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:35.670 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.670 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.671 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.671 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T09:35:35.671 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.671 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.671 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_refresh_rate": { 2023-10-18T09:35:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_refresh_rate", 2023-10-18T09:35:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "20", 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.673 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T09:35:35.674 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.674 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.674 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.675 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_starting_port": { 2023-10-18T09:35:35.675 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_starting_port", 2023-10-18T09:35:35.675 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.675 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.675 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.676 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4721", 2023-10-18T09:35:35.676 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.676 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.676 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.676 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T09:35:35.677 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.677 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.677 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.678 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_ptrace": { 2023-10-18T09:35:35.678 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_ptrace", 2023-10-18T09:35:35.678 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.678 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.678 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T09:35:35.679 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T09:35:35.680 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.680 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.680 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_interval": { 2023-10-18T09:35:35.680 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_interval", 2023-10-18T09:35:35.681 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.681 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.681 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.681 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:35.681 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.682 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.682 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.682 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T09:35:35.682 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.683 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.683 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.683 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_memory_target_ratio": { 2023-10-18T09:35:35.684 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T09:35:35.684 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:35.684 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.684 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.686 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.7", 2023-10-18T09:35:35.686 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.687 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.687 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.687 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T09:35:35.687 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.687 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm_log_destination": { 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm_log_destination", 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.688 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.689 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.689 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.689 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.689 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.689 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.690 INFO:teuthology.orchestra.run.smithi061.stdout: "file", 2023-10-18T09:35:35.690 INFO:teuthology.orchestra.run.smithi061.stdout: "file,syslog", 2023-10-18T09:35:35.690 INFO:teuthology.orchestra.run.smithi061.stdout: "syslog" 2023-10-18T09:35:35.690 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.690 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T09:35:35.691 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.691 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.691 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.691 INFO:teuthology.orchestra.run.smithi061.stdout: "cgroups_split": { 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cgroups_split", 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.692 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.693 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.693 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.693 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T09:35:35.693 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.693 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.694 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.694 INFO:teuthology.orchestra.run.smithi061.stdout: "config_checks_enabled": { 2023-10-18T09:35:35.694 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_checks_enabled", 2023-10-18T09:35:35.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.695 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.696 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T09:35:35.696 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.696 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.696 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: "config_dashboard": { 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_dashboard", 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.697 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.698 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.698 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.698 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.698 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.698 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_alertmanager": { 2023-10-18T09:35:35.699 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_alertmanager", 2023-10-18T09:35:35.700 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.700 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.700 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.700 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T09:35:35.700 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.701 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.702 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_base": { 2023-10-18T09:35:35.702 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_base", 2023-10-18T09:35:35.702 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.702 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.703 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Container image name, without the tag", 2023-10-18T09:35:35.704 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.704 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.704 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.704 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_elasticsearch": { 2023-10-18T09:35:35.705 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_elasticsearch", 2023-10-18T09:35:35.705 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.705 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.705 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.705 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T09:35:35.706 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.706 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.706 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.706 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "elasticsearch container image", 2023-10-18T09:35:35.706 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.707 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.707 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.707 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_grafana": { 2023-10-18T09:35:35.707 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_grafana", 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.708 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.709 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.709 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:35.709 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.709 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.709 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.710 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_haproxy": { 2023-10-18T09:35:35.710 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_haproxy", 2023-10-18T09:35:35.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.710 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.711 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.711 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T09:35:35.711 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.711 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.711 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.712 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "HAproxy container image", 2023-10-18T09:35:35.712 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.712 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.712 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.713 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_agent": { 2023-10-18T09:35:35.713 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_agent", 2023-10-18T09:35:35.713 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.713 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T09:35:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger agent container image", 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.715 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_collector": { 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_collector", 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T09:35:35.716 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.717 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.717 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.717 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", 2023-10-18T09:35:35.717 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.717 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.718 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.718 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_query": { 2023-10-18T09:35:35.718 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_query", 2023-10-18T09:35:35.718 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.719 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.720 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger query container image", 2023-10-18T09:35:35.720 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.720 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.720 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_keepalived": { 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_keepalived", 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.721 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T09:35:35.722 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.722 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.722 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.722 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Keepalived container image", 2023-10-18T09:35:35.722 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_loki": { 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_loki", 2023-10-18T09:35:35.723 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.724 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.724 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.724 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T09:35:35.724 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.724 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Loki container image", 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.726 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_node_exporter": { 2023-10-18T09:35:35.726 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_node_exporter", 2023-10-18T09:35:35.726 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.726 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.726 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:35.727 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.728 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.728 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.728 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_nvmeof": { 2023-10-18T09:35:35.728 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_nvmeof", 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.729 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Nvme-of container image", 2023-10-18T09:35:35.730 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.730 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.730 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_prometheus": { 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_prometheus", 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.731 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.732 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T09:35:35.732 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.732 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.732 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.732 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_promtail": { 2023-10-18T09:35:35.733 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_promtail", 2023-10-18T09:35:35.734 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.734 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.734 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.734 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T09:35:35.734 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Promtail container image", 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.735 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.736 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_snmp_gateway": { 2023-10-18T09:35:35.736 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_snmp_gateway", 2023-10-18T09:35:35.736 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.736 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.737 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.737 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T09:35:35.737 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.737 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.737 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.738 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", 2023-10-18T09:35:35.738 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.738 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.738 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.739 INFO:teuthology.orchestra.run.smithi061.stdout: "container_init": { 2023-10-18T09:35:35.739 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_init", 2023-10-18T09:35:35.739 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.739 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.740 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T09:35:35.741 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.741 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.741 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.741 INFO:teuthology.orchestra.run.smithi061.stdout: "daemon_cache_timeout": { 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "daemon_cache_timeout", 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:35.742 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.743 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.743 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.743 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T09:35:35.743 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.743 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.744 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.744 INFO:teuthology.orchestra.run.smithi061.stdout: "default_cephadm_command_timeout": { 2023-10-18T09:35:35.744 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T09:35:35.744 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "900", 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.745 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.746 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T09:35:35.746 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.746 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.746 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "default_registry": { 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_registry", 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.747 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io", 2023-10-18T09:35:35.748 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.748 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.748 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.748 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T09:35:35.748 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: "device_cache_timeout": { 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_cache_timeout", 2023-10-18T09:35:35.749 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.750 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.750 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.750 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1800", 2023-10-18T09:35:35.750 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.750 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.751 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.751 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache device inventory", 2023-10-18T09:35:35.751 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.751 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.751 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.752 INFO:teuthology.orchestra.run.smithi061.stdout: "device_enhanced_scan": { 2023-10-18T09:35:35.752 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_enhanced_scan", 2023-10-18T09:35:35.752 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.752 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.753 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T09:35:35.754 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.754 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.754 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.754 INFO:teuthology.orchestra.run.smithi061.stdout: "facts_cache_timeout": { 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "facts_cache_timeout", 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:35.755 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.756 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.756 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.756 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache host facts data", 2023-10-18T09:35:35.756 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.756 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.757 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.758 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.758 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:35.758 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.758 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.758 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to perform a host check", 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.759 INFO:teuthology.orchestra.run.smithi061.stdout: "inventory_list_all": { 2023-10-18T09:35:35.760 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "inventory_list_all", 2023-10-18T09:35:35.760 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.760 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.760 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.760 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.761 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.762 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.762 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:35.762 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:35.762 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.763 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.763 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.763 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.763 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.763 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.764 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.764 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.764 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.764 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.764 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.765 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.765 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.765 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.765 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.765 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.766 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.766 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.766 INFO:teuthology.orchestra.run.smithi061.stdout: "log_refresh_metadata": { 2023-10-18T09:35:35.766 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_refresh_metadata", 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.767 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.768 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.768 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T09:35:35.768 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.768 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.768 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.770 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.770 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.770 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.770 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.770 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T09:35:35.771 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.771 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.771 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.771 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:35.772 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.773 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.773 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.773 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.773 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.773 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.774 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.775 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.775 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.775 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:35.775 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:35.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.776 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.776 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.776 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.777 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.777 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.777 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.777 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T09:35:35.778 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.779 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.779 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.779 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.779 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.779 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.781 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T09:35:35.781 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T09:35:35.781 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.781 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.781 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.782 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "*", 2023-10-18T09:35:35.782 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.782 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.782 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.782 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: "max_count_per_host": { 2023-10-18T09:35:35.783 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_count_per_host", 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.784 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.785 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.785 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of daemons per service per host", 2023-10-18T09:35:35.785 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.785 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.785 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.786 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd_draining_count": { 2023-10-18T09:35:35.786 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_osd_draining_count", 2023-10-18T09:35:35.786 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.786 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.787 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T09:35:35.788 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.788 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.788 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.788 INFO:teuthology.orchestra.run.smithi061.stdout: "migration_current": { 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "migration_current", 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.789 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.790 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.790 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.790 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "internal - do not modify", 2023-10-18T09:35:35.790 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.790 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.791 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.792 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.792 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "root", 2023-10-18T09:35:35.792 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.792 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.792 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm-package", 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: "root" 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.794 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.794 INFO:teuthology.orchestra.run.smithi061.stdout: "prometheus_alerts_path": { 2023-10-18T09:35:35.794 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_alerts_path", 2023-10-18T09:35:35.794 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.795 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.795 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.795 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T09:35:35.795 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.795 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.796 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.796 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T09:35:35.796 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.796 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.796 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_insecure": { 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_insecure", 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.797 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.798 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.798 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.798 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.798 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.798 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T09:35:35.799 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.799 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.799 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.799 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_password": { 2023-10-18T09:35:35.800 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_password", 2023-10-18T09:35:35.800 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.800 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.800 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.800 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.801 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.802 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.802 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_url": { 2023-10-18T09:35:35.802 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_url", 2023-10-18T09:35:35.802 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.803 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.804 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_username": { 2023-10-18T09:35:35.805 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_username", 2023-10-18T09:35:35.805 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.805 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.805 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.805 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.806 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.807 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.807 INFO:teuthology.orchestra.run.smithi061.stdout: "secure_monitoring_stack": { 2023-10-18T09:35:35.807 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "secure_monitoring_stack", 2023-10-18T09:35:35.807 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.808 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.808 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.808 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.808 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.809 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.809 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T09:35:35.809 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.809 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: "service_discovery_port": { 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "service_discovery_port", 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.810 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.811 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", 2023-10-18T09:35:35.811 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.811 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.811 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.811 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "cephadm service discovery port", 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: "ssh_config_file": { 2023-10-18T09:35:35.812 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssh_config_file", 2023-10-18T09:35:35.813 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.813 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.813 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.813 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.814 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.814 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.814 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.814 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T09:35:35.814 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: "use_agent": { 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_agent", 2023-10-18T09:35:35.815 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.816 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.816 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.816 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.816 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.816 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.818 INFO:teuthology.orchestra.run.smithi061.stdout: "use_repo_digest": { 2023-10-18T09:35:35.818 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_repo_digest", 2023-10-18T09:35:35.818 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.818 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.818 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T09:35:35.819 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.820 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.820 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.820 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_failed_host_check": { 2023-10-18T09:35:35.820 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_failed_host_check", 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.821 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.822 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.822 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T09:35:35.822 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.822 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.823 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_daemons": { 2023-10-18T09:35:35.823 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_daemons", 2023-10-18T09:35:35.823 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.823 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.824 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_hosts": { 2023-10-18T09:35:35.825 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_hosts", 2023-10-18T09:35:35.826 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.826 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.826 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.826 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.826 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.827 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.828 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.828 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.828 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crash", 2023-10-18T09:35:35.829 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:35.829 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:35.829 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:35.829 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:35.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:35.830 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.830 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.830 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.830 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.830 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.831 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.832 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.833 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:35.833 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:35.833 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.834 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.835 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:35.836 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:35.836 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.836 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.836 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.836 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.837 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.838 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.838 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.838 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.838 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.838 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.839 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.839 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.839 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.839 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:35.840 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:35.840 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.840 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.840 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.840 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.842 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.842 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.842 INFO:teuthology.orchestra.run.smithi061.stdout: "retain_interval": { 2023-10-18T09:35:35.842 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retain_interval", 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "31536000", 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.845 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_recent_interval": { 2023-10-18T09:35:35.845 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_recent_interval", 2023-10-18T09:35:35.845 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:35.845 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.845 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.846 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1209600", 2023-10-18T09:35:35.846 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.846 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.846 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.846 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:35.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:35.848 INFO:teuthology.orchestra.run.smithi061.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T09:35:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T09:35:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.849 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.850 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T09:35:35.851 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.852 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.853 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.853 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.853 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.853 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.854 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.855 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.855 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.855 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.855 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.855 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_ENABLED": { 2023-10-18T09:35:35.856 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.857 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.858 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.858 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.858 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.859 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.860 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.860 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.860 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.860 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.860 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T09:35:35.861 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T09:35:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.863 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.864 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T09:35:35.864 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T09:35:35.864 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.864 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.865 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.865 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.865 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.865 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.865 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.866 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.866 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.866 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.867 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.868 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.868 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.868 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.868 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.868 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T09:35:35.869 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.870 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.871 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.871 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.871 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.871 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.872 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T09:35:35.872 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T09:35:35.872 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.872 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.872 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.873 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.874 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.874 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.874 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T09:35:35.874 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T09:35:35.875 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.875 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.875 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.875 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.875 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.876 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.876 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.876 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.876 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.876 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.877 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.877 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T09:35:35.877 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.878 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.879 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.879 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.879 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.879 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.879 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.880 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.881 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.882 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.882 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.882 INFO:teuthology.orchestra.run.smithi061.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T09:35:35.882 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.883 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.884 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.884 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.884 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.884 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.884 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.885 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.886 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.887 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.887 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.887 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T09:35:35.887 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.888 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.890 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_URL": { 2023-10-18T09:35:35.890 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_URL", 2023-10-18T09:35:35.890 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.890 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.891 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T09:35:35.892 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T09:35:35.893 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.893 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.893 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.893 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:35.893 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.894 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.894 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.894 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.894 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.894 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.895 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.896 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.897 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.897 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.897 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.898 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T09:35:35.898 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T09:35:35.898 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.898 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.898 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.899 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.899 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.899 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.899 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.900 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.900 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.900 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.900 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.901 INFO:teuthology.orchestra.run.smithi061.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T09:35:35.901 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T09:35:35.901 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.901 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.902 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.903 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.903 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.903 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.903 INFO:teuthology.orchestra.run.smithi061.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T09:35:35.904 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T09:35:35.904 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.905 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.905 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.905 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.905 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.905 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.906 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.906 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.906 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T09:35:35.906 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.907 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.908 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.908 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.908 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.908 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.908 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.909 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T09:35:35.909 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T09:35:35.909 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.910 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.910 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.910 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.910 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.910 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.911 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.911 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.911 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.911 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.912 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T09:35:35.912 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T09:35:35.912 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.912 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.913 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.913 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.913 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.913 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.913 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.914 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.914 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.914 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.914 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.915 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T09:35:35.915 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T09:35:35.915 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.916 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.916 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.916 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.916 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.916 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.917 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.917 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.917 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.917 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.917 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.918 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T09:35:35.918 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T09:35:35.918 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.918 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.919 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.920 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.920 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.920 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.920 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.921 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.922 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.922 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.922 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.922 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.922 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.923 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.924 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.924 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.924 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.925 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.925 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.925 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.925 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.926 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.926 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T09:35:35.926 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T09:35:35.926 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.927 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.927 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.927 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.927 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.927 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T09:35:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T09:35:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T09:35:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T09:35:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T09:35:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T09:35:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T09:35:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8", 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T09:35:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T09:35:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "45", 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T09:35:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T09:35:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T09:35:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T09:35:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T09:35:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T09:35:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T09:35:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T09:35:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T09:35:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T09:35:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T09:35:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { 2023-10-18T09:35:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", 2023-10-18T09:35:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "crt_file": { 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crt_file", 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "debug": { 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "debug", 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable/disable debug options", 2023-10-18T09:35:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "jwt_token_ttl": { 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "jwt_token_ttl", 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "28800", 2023-10-18T09:35:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "motd": { 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "motd", 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "The message of the day", 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect_resolve_ip_addr": { 2023-10-18T09:35:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T09:35:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8080", 2023-10-18T09:35:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl_server_port": { 2023-10-18T09:35:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl_server_port", 2023-10-18T09:35:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8443", 2023-10-18T09:35:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "redirect", 2023-10-18T09:35:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect" 2023-10-18T09:35:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "url_prefix": { 2023-10-18T09:35:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url_prefix", 2023-10-18T09:35:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "devicehealth", 2023-10-18T09:35:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_monitoring": { 2023-10-18T09:35:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_monitoring", 2023-10-18T09:35:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "monitor device health metrics", 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mark_out_threshold": { 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mark_out_threshold", 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2419200", 2023-10-18T09:35:36.040 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.040 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.040 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.040 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T09:35:36.040 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.041 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.041 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": { 2023-10-18T09:35:36.041 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_name", 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "device_health_metrics", 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.042 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.043 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.043 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T09:35:36.043 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.043 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.043 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: "retention_period": { 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retention_period", 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.044 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.045 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15552000", 2023-10-18T09:35:36.045 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.045 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.045 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.046 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain device health metrics", 2023-10-18T09:35:36.046 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.046 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.046 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_frequency": { 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_frequency", 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.047 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.048 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.049 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.049 INFO:teuthology.orchestra.run.smithi061.stdout: "self_heal": { 2023-10-18T09:35:36.049 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "self_heal", 2023-10-18T09:35:36.049 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.050 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.051 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T09:35:36.051 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.051 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.052 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:36.052 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:36.052 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.052 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.052 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.053 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:36.053 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.053 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.053 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.054 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T09:35:36.054 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.054 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_threshold": { 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_threshold", 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.056 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.056 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7257600", 2023-10-18T09:35:36.056 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.056 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.056 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.057 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T09:35:36.057 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.057 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.057 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.058 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.058 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "diskprediction_local", 2023-10-18T09:35:36.058 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.059 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.060 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.060 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.060 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.060 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.061 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.062 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.062 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.062 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.062 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.062 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.063 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.064 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.064 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.064 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.064 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.064 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.065 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.066 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.066 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.066 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.066 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.067 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.067 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.067 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.067 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.067 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.068 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.069 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.069 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.069 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.069 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.070 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.071 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.071 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.071 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.071 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.071 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: "predict_interval": { 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predict_interval", 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.072 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.073 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.073 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:36.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.073 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.073 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.074 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.074 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.074 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "predictor_model": { 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predictor_model", 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.075 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "prophetstor", 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.076 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.077 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.077 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:36.077 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:36.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.078 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.078 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.078 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:36.078 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.079 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.080 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "influx", 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": false, 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "influxdb python module not found", 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "batch_size": { 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "batch_size", 2023-10-18T09:35:36.081 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.082 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.082 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.082 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5000", 2023-10-18T09:35:36.082 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.082 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "database": { 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "database", 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.084 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "ceph", 2023-10-18T09:35:36.085 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.085 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.085 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.085 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T09:35:36.085 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": { 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "hostname", 2023-10-18T09:35:36.086 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.087 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.087 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.087 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.087 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.087 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server hostname", 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.089 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:36.089 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:36.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.089 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.089 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.090 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "30", 2023-10-18T09:35:36.090 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "5", 2023-10-18T09:35:36.090 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.090 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T09:35:36.091 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.091 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.091 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.091 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.092 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.092 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.092 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.092 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.092 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.093 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.093 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.093 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.093 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.093 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.094 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.094 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.094 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.094 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.094 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.095 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.095 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.095 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.095 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.096 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.097 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.097 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.097 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.097 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.097 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.098 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.099 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.099 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.099 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.099 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.100 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.101 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.101 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.101 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.101 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.101 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.102 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.103 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.103 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.103 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.103 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.103 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.105 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { 2023-10-18T09:35:36.105 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "password", 2023-10-18T09:35:36.105 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.105 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.105 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "password of InfluxDB server user", 2023-10-18T09:35:36.106 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.107 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.107 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.107 INFO:teuthology.orchestra.run.smithi061.stdout: "port": { 2023-10-18T09:35:36.107 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "port", 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8086", 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.108 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server port", 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.110 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:36.110 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:36.110 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.110 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.110 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.111 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "false", 2023-10-18T09:35:36.111 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.111 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.111 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.112 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:36.112 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.112 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.112 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "threads": { 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threads", 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.113 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:36.114 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:36.114 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "32", 2023-10-18T09:35:36.114 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.114 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T09:35:36.114 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.115 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.115 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.115 INFO:teuthology.orchestra.run.smithi061.stdout: "username": { 2023-10-18T09:35:36.115 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "username", 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.116 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.117 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.117 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "username of InfluxDB server user", 2023-10-18T09:35:36.117 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.117 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.117 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.118 INFO:teuthology.orchestra.run.smithi061.stdout: "verify_ssl": { 2023-10-18T09:35:36.118 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "verify_ssl", 2023-10-18T09:35:36.118 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.118 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "true", 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.119 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.121 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "insights", 2023-10-18T09:35:36.121 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.121 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.121 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.122 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.122 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.122 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.122 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.122 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.123 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.124 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.124 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.124 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.124 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.124 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.125 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.126 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.126 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.126 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.126 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.126 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.128 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.128 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.128 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "iostat", 2023-10-18T09:35:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "k8sevents", 2023-10-18T09:35:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_event_retention_days": { 2023-10-18T09:35:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ceph_event_retention_days", 2023-10-18T09:35:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7", 2023-10-18T09:35:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T09:35:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "config_check_secs": { 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_check_secs", 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "10", 2023-10-18T09:35:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T09:35:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "localpool", 2023-10-18T09:35:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "failure_domain": { 2023-10-18T09:35:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "failure_domain", 2023-10-18T09:35:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "host", 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "failure domain for any created local pool", 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T09:35:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": { 2023-10-18T09:35:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_size", 2023-10-18T09:35:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default min_size for any created local pool", 2023-10-18T09:35:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T09:35:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "num_rep": { 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "num_rep", 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3", 2023-10-18T09:35:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default replica count for any created local pool", 2023-10-18T09:35:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": { 2023-10-18T09:35:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_num", 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "128", 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T09:35:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "prefix": { 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prefix", 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name prefix for any created local pool", 2023-10-18T09:35:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "subtree": { 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "subtree", 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "rack", 2023-10-18T09:35:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T09:35:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T09:35:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mds_autoscaler", 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", 2023-10-18T09:35:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "nfs", 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.236 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.237 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.237 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.237 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.237 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.237 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.238 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.239 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.239 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.239 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.239 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.240 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.241 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.241 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.241 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.241 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.241 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.242 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.242 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.242 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.242 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.243 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.243 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.244 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.244 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.244 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.244 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.245 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.245 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.246 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.246 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.246 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.247 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.247 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.247 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.247 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.248 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.249 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:36.249 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.249 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.250 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.250 INFO:teuthology.orchestra.run.smithi061.stdout: "fail_fs": { 2023-10-18T09:35:36.251 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "fail_fs", 2023-10-18T09:35:36.251 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.251 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.252 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.252 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.252 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.252 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.253 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.253 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T09:35:36.253 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.253 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.253 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.254 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.254 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.254 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.254 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.255 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.255 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.255 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.256 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.256 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.256 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.256 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.256 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.257 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.258 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.258 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.258 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.258 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.259 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.260 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.260 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.260 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.260 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.260 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.261 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.262 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.262 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.262 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.262 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.262 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.263 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.264 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.264 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.264 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.264 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.265 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.266 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.266 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.266 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.266 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.266 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.267 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.267 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.267 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator": { 2023-10-18T09:35:36.267 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:36.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.268 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.268 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.268 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.268 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm", 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: "rook", 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: "test_orchestrator" 2023-10-18T09:35:36.269 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.270 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Orchestrator backend", 2023-10-18T09:35:36.270 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.270 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.270 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.270 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.271 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.271 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_perf_query", 2023-10-18T09:35:36.271 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.272 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.273 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.274 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.274 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.274 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.274 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.274 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.275 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.275 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.275 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.275 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.275 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.276 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.276 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.276 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.276 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.277 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.278 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.278 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.278 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.278 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.279 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.279 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.279 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.279 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.279 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.280 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.280 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.280 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.280 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.280 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.281 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.281 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.281 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.281 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.281 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.282 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.283 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.283 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.283 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.283 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.283 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.284 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.285 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.285 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.285 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.286 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_support", 2023-10-18T09:35:36.286 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.286 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.286 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.286 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.287 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.287 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.287 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.287 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.287 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.288 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.288 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.288 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.288 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.289 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.289 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.289 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.289 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.289 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.290 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.290 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.290 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.290 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.291 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.291 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.291 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.292 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.292 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.292 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.293 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.293 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.293 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.294 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.294 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.294 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.295 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.295 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.296 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.296 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.296 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.297 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.297 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.297 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.298 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.298 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.299 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.301 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.302 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.302 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.302 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.302 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.303 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.303 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.303 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.303 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.303 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.304 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.304 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.304 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.304 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.305 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.306 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.306 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.306 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.306 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.306 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.307 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_autoscaler", 2023-10-18T09:35:36.307 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.308 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.309 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.309 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.309 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.309 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.309 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.310 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.310 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.310 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.310 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.310 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.311 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.312 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.313 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.313 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.313 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.313 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.313 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.314 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.315 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.315 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.315 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.315 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.315 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.316 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.316 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.316 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.316 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.316 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.317 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.317 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.317 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.317 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.318 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.318 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.318 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.319 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.319 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.319 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.319 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.321 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.321 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.321 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.321 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.322 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:36.323 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.323 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.323 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.323 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.323 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: "threshold": { 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threshold", 2023-10-18T09:35:36.324 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.325 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.325 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.325 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:36.325 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1.0", 2023-10-18T09:35:36.325 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.326 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.326 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "scaling threshold", 2023-10-18T09:35:36.326 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T09:35:36.326 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.326 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.327 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.327 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.327 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "progress", 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_pg_recovery_event": { 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_pg_recovery_event", 2023-10-18T09:35:36.328 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.329 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.329 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.329 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.329 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.329 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.330 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.330 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T09:35:36.330 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.330 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.331 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.331 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:36.331 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:36.331 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.332 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.332 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.332 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.332 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.332 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.334 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.334 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.334 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.334 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.334 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.335 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.335 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.335 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.335 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.335 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.336 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.337 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.337 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.337 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.337 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.338 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.338 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.338 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.338 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.338 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.339 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.340 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.340 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.340 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.340 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.341 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.342 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.342 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.342 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.342 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.343 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.344 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.344 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.344 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.344 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.345 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.346 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.346 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.346 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.347 INFO:teuthology.orchestra.run.smithi061.stdout: "max_completed_events": { 2023-10-18T09:35:36.347 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_completed_events", 2023-10-18T09:35:36.347 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.347 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.348 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.348 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "50", 2023-10-18T09:35:36.348 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.348 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.348 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.349 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "number of past completed events to remember", 2023-10-18T09:35:36.349 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.349 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.349 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.350 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long the module is going to sleep", 2023-10-18T09:35:36.351 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.353 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus", 2023-10-18T09:35:36.353 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.353 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.353 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.353 INFO:teuthology.orchestra.run.smithi061.stdout: "cache": { 2023-10-18T09:35:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cache", 2023-10-18T09:35:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.354 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.356 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.356 INFO:teuthology.orchestra.run.smithi061.stdout: "exclude_perf_counters": { 2023-10-18T09:35:36.356 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "exclude_perf_counters", 2023-10-18T09:35:36.356 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.357 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.358 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T09:35:36.358 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T09:35:36.358 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.358 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.359 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.359 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.359 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.359 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.359 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.360 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.360 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.360 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.360 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.360 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.361 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.361 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.361 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.361 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.362 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.362 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.362 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.363 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.363 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.363 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.364 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.364 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.365 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.365 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.365 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.365 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.365 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.366 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.366 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.366 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.366 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.366 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.367 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.368 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.368 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.368 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.368 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.368 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.369 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.369 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.369 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.369 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.369 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.370 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.370 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.370 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.370 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.370 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.371 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.371 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.371 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.371 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.372 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.373 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.373 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.373 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.373 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.374 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools": { 2023-10-18T09:35:36.374 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools", 2023-10-18T09:35:36.374 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.374 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.374 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.375 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.376 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.376 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.376 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T09:35:36.376 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T09:35:36.377 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.377 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.377 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.377 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300", 2023-10-18T09:35:36.378 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.378 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.378 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.378 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.378 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.379 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.379 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.379 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_interval": { 2023-10-18T09:35:36.379 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_interval", 2023-10-18T09:35:36.380 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.380 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.380 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.380 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15.0", 2023-10-18T09:35:36.381 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.381 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.381 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.381 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.382 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.382 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.382 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.383 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:36.383 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:36.383 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.384 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.384 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.384 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:36.384 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.385 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.385 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.385 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T09:35:36.385 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.386 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.386 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.387 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:36.387 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:36.387 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.388 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.388 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.388 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "9283", 2023-10-18T09:35:36.388 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T09:35:36.390 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.390 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.390 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.391 INFO:teuthology.orchestra.run.smithi061.stdout: "stale_cache_strategy": { 2023-10-18T09:35:36.391 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stale_cache_strategy", 2023-10-18T09:35:36.392 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.392 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.392 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.392 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "log", 2023-10-18T09:35:36.393 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.393 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.393 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.394 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.394 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.394 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.395 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.395 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:36.395 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:36.396 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.396 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.396 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.396 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "default", 2023-10-18T09:35:36.397 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.397 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.397 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.397 INFO:teuthology.orchestra.run.smithi061.stdout: "default", 2023-10-18T09:35:36.398 INFO:teuthology.orchestra.run.smithi061.stdout: "error" 2023-10-18T09:35:36.398 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.398 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.398 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.398 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.399 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.400 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.400 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:36.400 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:36.400 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:36.400 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.401 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.402 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_support", 2023-10-18T09:35:36.402 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.402 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.403 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.403 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.403 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.403 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.403 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.404 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.405 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.406 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.406 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.406 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.406 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.407 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.407 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.407 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.407 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.407 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.408 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.408 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.408 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.408 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.410 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.410 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.410 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.410 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.410 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.411 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.411 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.411 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.411 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.411 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.412 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.413 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.413 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.413 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.413 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.414 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.415 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.415 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.415 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.415 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.415 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.416 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.416 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_snap_create": { 2023-10-18T09:35:36.416 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_snap_create", 2023-10-18T09:35:36.416 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.417 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.418 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.418 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.418 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.418 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.419 INFO:teuthology.orchestra.run.smithi061.stdout: "mirror_snapshot_schedule": { 2023-10-18T09:35:36.419 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T09:35:36.419 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.419 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.419 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.420 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.420 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.420 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.420 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.421 INFO:teuthology.orchestra.run.smithi061.stdout: "trash_purge_schedule": { 2023-10-18T09:35:36.422 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "trash_purge_schedule", 2023-10-18T09:35:36.422 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.422 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.422 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.422 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.423 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.424 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.424 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.424 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.425 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "restful", 2023-10-18T09:35:36.425 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.425 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.425 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.425 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_auth", 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.427 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.427 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.427 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.427 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.428 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.429 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.429 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.429 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.429 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.429 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.430 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.431 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.431 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.431 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.431 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.431 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.432 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.433 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.433 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.433 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.433 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.434 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.434 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.434 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.434 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.435 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.435 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.435 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.435 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.436 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.436 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.436 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.436 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.436 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.437 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.438 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.438 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.438 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.438 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.438 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.439 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.439 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.439 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.439 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.439 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.440 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.440 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.440 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.440 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.441 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.441 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.441 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.441 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.441 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.442 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.442 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.442 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.442 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.443 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.444 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.444 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.444 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.444 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.445 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.446 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.446 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.446 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.446 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.446 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.447 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.447 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:36.447 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:36.447 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.448 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.449 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.449 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.449 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.449 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.449 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rgw", 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.450 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.451 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.451 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.451 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.451 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.451 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.452 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.453 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.453 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.453 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.453 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.453 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.454 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.454 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.454 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.454 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.455 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.455 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.455 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.455 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.456 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.456 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.456 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.456 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.458 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.458 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.458 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.458 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.458 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.459 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.459 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.459 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.459 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.459 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.460 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.461 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.461 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.461 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.461 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.462 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.462 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.462 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.462 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.462 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.463 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.463 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.463 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.463 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.463 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.465 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rook", 2023-10-18T09:35:36.465 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.465 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.465 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.466 INFO:teuthology.orchestra.run.smithi061.stdout: "drive_group_interval": { 2023-10-18T09:35:36.466 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "drive_group_interval", 2023-10-18T09:35:36.466 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.466 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.466 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300.0", 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T09:35:36.467 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.468 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.468 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.468 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.469 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.469 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.469 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.469 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.469 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.470 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.470 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.470 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.470 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.470 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.471 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.471 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.471 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.471 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.471 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.472 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.473 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.473 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.473 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.473 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.473 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.475 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.475 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.475 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.475 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.476 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.477 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.477 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.477 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.477 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.477 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.478 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.478 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.478 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.478 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.478 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.479 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.479 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.479 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.479 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.480 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.481 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.481 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.481 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.481 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "storage_class": { 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "storage_class", 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.482 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "local", 2023-10-18T09:35:36.483 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.483 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.483 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.483 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T09:35:36.483 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "selftest", 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.485 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.486 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.486 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.486 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.486 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.486 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.487 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.488 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.488 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.488 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.488 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.488 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.489 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.489 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.489 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.489 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.490 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.490 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.490 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.490 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.491 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.492 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.492 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.492 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.493 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.494 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.494 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.494 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.494 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.494 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.495 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.496 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.496 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.496 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.496 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.497 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.497 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.497 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.497 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.497 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.498 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.498 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.498 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.498 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "roption1": { 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption1", 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.499 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.500 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.500 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.500 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.500 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.500 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: "roption2": { 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption2", 2023-10-18T09:35:36.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.502 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.502 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.502 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "xyz", 2023-10-18T09:35:36.502 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.503 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.504 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption1": { 2023-10-18T09:35:36.504 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption1", 2023-10-18T09:35:36.504 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.504 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.505 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.506 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.506 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.506 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.506 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption2": { 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption2", 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.507 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.508 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.508 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.508 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.508 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.508 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption3": { 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption3", 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.510 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.510 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.510 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.510 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.510 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.511 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption4": { 2023-10-18T09:35:36.512 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption4", 2023-10-18T09:35:36.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.512 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.512 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.513 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.514 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.514 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption5": { 2023-10-18T09:35:36.514 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption5", 2023-10-18T09:35:36.514 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.515 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.515 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.515 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.515 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.515 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.516 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.516 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.516 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.516 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.516 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.517 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { 2023-10-18T09:35:36.517 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", 2023-10-18T09:35:36.517 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.517 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.518 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.518 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.518 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.518 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.518 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.519 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption7": { 2023-10-18T09:35:36.520 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption7", 2023-10-18T09:35:36.520 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.520 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.520 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.520 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "42", 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.521 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.522 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.522 INFO:teuthology.orchestra.run.smithi061.stdout: "testkey": { 2023-10-18T09:35:36.522 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testkey", 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.523 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.524 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.524 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.524 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.524 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.524 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: "testlkey": { 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testlkey", 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.525 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.526 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.526 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.526 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.526 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.526 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: "testnewline": { 2023-10-18T09:35:36.527 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testnewline", 2023-10-18T09:35:36.528 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.528 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.528 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.528 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.528 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.529 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.530 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snap_schedule", 2023-10-18T09:35:36.531 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.531 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.531 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.531 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_m_granularity": { 2023-10-18T09:35:36.531 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_m_granularity", 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.532 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.533 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.533 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T09:35:36.533 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.533 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.533 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.534 INFO:teuthology.orchestra.run.smithi061.stdout: "dump_on_update": { 2023-10-18T09:35:36.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dump_on_update", 2023-10-18T09:35:36.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.534 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.535 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", 2023-10-18T09:35:36.536 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.536 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.536 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.536 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.537 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.538 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.538 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.538 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.538 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.538 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.539 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.540 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.540 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.540 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.540 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.541 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.542 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.542 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.542 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.542 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.542 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.543 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.544 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.544 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.544 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.544 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.544 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.545 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.546 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.546 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.546 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.546 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.547 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.548 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.548 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.548 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.548 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.548 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.550 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stats", 2023-10-18T09:35:36.550 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.550 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.550 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.550 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.552 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.553 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.554 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.554 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.554 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.554 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.555 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.555 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.555 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.555 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.555 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.556 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.556 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.556 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.556 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.557 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.558 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.558 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.558 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.558 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.558 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.559 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.560 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.560 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.560 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.560 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.560 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.561 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.562 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.562 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.562 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.562 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.562 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.563 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "status", 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.564 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.565 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.565 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.565 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.565 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.565 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.566 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.566 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.566 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.566 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.566 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.567 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.568 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.569 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.569 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.569 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.569 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.569 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.570 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.570 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.570 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.570 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.571 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.572 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.572 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.572 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.572 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.572 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.573 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.574 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.574 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.574 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.574 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.574 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.575 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.576 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.576 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.576 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.576 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.576 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.577 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.577 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.577 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.577 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.577 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telegraf", 2023-10-18T09:35:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.579 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.579 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.579 INFO:teuthology.orchestra.run.smithi061.stdout: "address": { 2023-10-18T09:35:36.579 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "address", 2023-10-18T09:35:36.579 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.580 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.580 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.580 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T09:35:36.580 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.580 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15", 2023-10-18T09:35:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.588 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.588 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.588 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.588 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.589 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.589 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.589 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.589 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.589 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.590 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.590 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.590 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.590 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.591 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.591 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.591 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.592 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.593 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.593 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.593 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.593 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.593 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.594 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.594 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.594 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.594 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.594 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.596 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.596 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.596 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.596 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.596 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.597 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.598 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.598 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.598 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telemetry", 2023-10-18T09:35:36.599 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.599 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.599 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.599 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_basic": { 2023-10-18T09:35:36.599 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_basic", 2023-10-18T09:35:36.600 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.600 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.600 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.600 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.600 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.601 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.601 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.601 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T09:35:36.601 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.601 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.602 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.602 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_crash": { 2023-10-18T09:35:36.602 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_crash", 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.603 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.604 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.604 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T09:35:36.604 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.604 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.604 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_device": { 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_device", 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.605 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.606 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.606 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.606 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.606 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.606 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_ident": { 2023-10-18T09:35:36.607 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_ident", 2023-10-18T09:35:36.608 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.608 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.608 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.608 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.608 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.609 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.609 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.609 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T09:35:36.609 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.609 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.610 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.610 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_perf": { 2023-10-18T09:35:36.610 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_perf", 2023-10-18T09:35:36.610 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.611 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.612 INFO:teuthology.orchestra.run.smithi061.stdout: "contact": { 2023-10-18T09:35:36.613 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "contact", 2023-10-18T09:35:36.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.613 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.613 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.613 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.614 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.614 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.614 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.614 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.614 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: "description": { 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "description", 2023-10-18T09:35:36.615 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.616 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.617 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.617 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.617 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.617 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.618 INFO:teuthology.orchestra.run.smithi061.stdout: "device_url": { 2023-10-18T09:35:36.618 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_url", 2023-10-18T09:35:36.618 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.618 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.618 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.619 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T09:35:36.619 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.619 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.619 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.619 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.620 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.620 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.620 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.621 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.622 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.622 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.622 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.622 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:36.623 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:36.624 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.624 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.624 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.624 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "24", 2023-10-18T09:35:36.624 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "8", 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.625 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.626 INFO:teuthology.orchestra.run.smithi061.stdout: "last_opt_revision": { 2023-10-18T09:35:36.626 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "last_opt_revision", 2023-10-18T09:35:36.626 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.626 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1", 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.627 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.628 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.628 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.628 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.628 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard": { 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard", 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.629 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.630 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.630 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.630 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.630 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.630 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.631 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.631 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard_description": { 2023-10-18T09:35:36.631 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard_description", 2023-10-18T09:35:36.631 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.632 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.632 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.632 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.632 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.633 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.633 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.633 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.633 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.633 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.634 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.634 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.634 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.634 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.635 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.636 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.637 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.637 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.637 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.637 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.637 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.638 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.638 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.638 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.638 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.639 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.640 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.641 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.641 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.641 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.641 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.641 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.642 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.643 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.643 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.643 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.643 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.643 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.644 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.644 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.644 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.645 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.645 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.645 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.645 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.645 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.646 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.646 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.646 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.646 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.646 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: "organization": { 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "organization", 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.647 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.648 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.648 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.648 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.648 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.648 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.649 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { 2023-10-18T09:35:36.650 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", 2023-10-18T09:35:36.650 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.650 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.650 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.650 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.651 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: "url": { 2023-10-18T09:35:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url", 2023-10-18T09:35:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.653 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.654 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.654 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.655 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.655 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "test_orchestrator", 2023-10-18T09:35:36.655 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.656 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.656 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.656 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.656 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.656 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.657 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.657 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.657 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.657 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.657 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.658 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.659 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.659 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.659 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.659 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.659 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.660 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.660 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.660 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.660 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.661 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.662 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.662 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.662 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.662 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.663 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.664 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.664 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.664 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.664 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.664 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.665 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.665 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.665 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.665 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.665 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.666 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.666 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.666 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.666 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.667 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.667 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.667 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.667 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.667 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.668 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.668 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.668 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.668 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.669 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.669 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.669 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.670 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.671 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "volumes", 2023-10-18T09:35:36.672 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.672 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.672 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.673 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.673 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.674 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.674 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.674 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.675 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.676 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.676 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.676 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.676 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.676 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.677 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.678 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.678 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.678 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.678 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.679 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.680 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.681 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.681 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.681 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.681 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.682 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.682 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.682 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.682 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.682 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.683 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.684 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.685 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.685 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.685 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.685 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.685 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.686 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_clones": { 2023-10-18T09:35:36.687 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_clones", 2023-10-18T09:35:36.687 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.687 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.687 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.687 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4", 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.688 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.689 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.689 INFO:teuthology.orchestra.run.smithi061.stdout: "periodic_async_work": { 2023-10-18T09:35:36.689 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "periodic_async_work", 2023-10-18T09:35:36.689 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.690 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.691 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Periodically check for async work", 2023-10-18T09:35:36.691 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.691 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.691 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "snapshot_clone_delay": { 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snapshot_clone_delay", 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.692 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:36.693 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.693 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.693 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.693 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T09:35:36.693 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.695 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix", 2023-10-18T09:35:36.695 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.695 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.695 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.695 INFO:teuthology.orchestra.run.smithi061.stdout: "discovery_interval": { 2023-10-18T09:35:36.696 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "discovery_interval", 2023-10-18T09:35:36.696 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:36.696 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.696 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.696 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "100", 2023-10-18T09:35:36.697 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.697 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.697 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.697 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.697 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.698 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.698 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.698 INFO:teuthology.orchestra.run.smithi061.stdout: "identifier": { 2023-10-18T09:35:36.698 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "identifier", 2023-10-18T09:35:36.699 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.699 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.699 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.699 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.699 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.700 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.701 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:36.701 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:36.701 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.701 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.703 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.703 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.703 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.703 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.704 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.704 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.704 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.704 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.704 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.705 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.705 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.705 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.705 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.705 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.706 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.706 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.706 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.706 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.706 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.707 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.707 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.707 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.708 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.708 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.709 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.709 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.709 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.710 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.710 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.711 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.711 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.711 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.712 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.712 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.712 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.713 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.713 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.714 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.714 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.714 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.714 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.715 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.715 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.715 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.715 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.716 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.716 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.716 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.716 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.716 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.717 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.717 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.717 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.717 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.717 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.718 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.718 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.719 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.719 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.719 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.719 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.720 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.720 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.720 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.720 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.721 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.721 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_host": { 2023-10-18T09:35:36.721 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_host", 2023-10-18T09:35:36.722 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.722 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.722 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.722 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.723 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.723 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.723 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.723 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.723 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_port": { 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_port", 2023-10-18T09:35:36.724 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.725 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.725 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.725 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10051", 2023-10-18T09:35:36.725 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.725 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.726 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.726 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.726 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.726 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.727 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.727 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_sender": { 2023-10-18T09:35:36.728 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_sender", 2023-10-18T09:35:36.728 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.729 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.729 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.729 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T09:35:36.730 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.730 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.731 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.731 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.732 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.732 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.732 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.732 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.732 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:36.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.733 INFO:teuthology.orchestra.run.smithi061.stdout: "gid": 24116, 2023-10-18T09:35:36.733 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "y", 2023-10-18T09:35:36.733 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_features": 4540138322906710015, 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: "available_modules": [ 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alerts", 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.734 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T09:35:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.741 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.742 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.742 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.742 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.742 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.742 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.743 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.743 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.743 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.743 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.743 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.744 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.744 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.744 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.744 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.745 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.745 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.745 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.745 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.745 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.746 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.746 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.746 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.746 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.746 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.747 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.748 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.748 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.748 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.748 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.749 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.750 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.750 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.750 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.750 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_destination": { 2023-10-18T09:35:36.751 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_destination", 2023-10-18T09:35:36.751 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.751 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.751 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.751 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email address to send alerts to", 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.752 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.753 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.753 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_from_name": { 2023-10-18T09:35:36.754 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_from_name", 2023-10-18T09:35:36.754 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.754 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.754 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.754 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "Ceph", 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email From: name", 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.755 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.756 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.756 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_host": { 2023-10-18T09:35:36.756 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_host", 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.757 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.758 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.758 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", 2023-10-18T09:35:36.758 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.758 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.758 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.759 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_password": { 2023-10-18T09:35:36.759 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_password", 2023-10-18T09:35:36.759 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.759 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.760 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Password to authenticate with", 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_port": { 2023-10-18T09:35:36.761 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_port", 2023-10-18T09:35:36.762 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.762 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.762 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.762 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "465", 2023-10-18T09:35:36.762 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.763 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.763 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.763 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP port", 2023-10-18T09:35:36.763 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.763 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_sender": { 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_sender", 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.764 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.765 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.765 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.765 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.766 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.766 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.767 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP envelope sender", 2023-10-18T09:35:36.767 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.767 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.767 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.768 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_ssl": { 2023-10-18T09:35:36.768 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_ssl", 2023-10-18T09:35:36.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.769 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.770 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.770 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.770 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.770 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.771 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_user": { 2023-10-18T09:35:36.772 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_user", 2023-10-18T09:35:36.772 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.772 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.772 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.773 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.773 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.773 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.773 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.773 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "User to authenticate as", 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.775 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "balancer", 2023-10-18T09:35:36.775 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.775 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.775 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "active": { 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "active", 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.776 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.777 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.777 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.777 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.777 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T09:35:36.777 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.778 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.778 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.778 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_time": { 2023-10-18T09:35:36.778 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_time", 2023-10-18T09:35:36.779 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.779 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.779 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.779 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", 2023-10-18T09:35:36.780 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.780 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.780 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.780 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T09:35:36.780 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:36.781 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.781 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.781 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_weekday": { 2023-10-18T09:35:36.781 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_weekday", 2023-10-18T09:35:36.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:36.782 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.782 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:36.782 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:36.783 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:36.783 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.783 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T09:35:36.783 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:36.783 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_max_iterations": { 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_max_iterations", 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:36.784 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.785 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.785 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "25", 2023-10-18T09:35:36.785 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:36.785 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "250", 2023-10-18T09:35:36.785 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.786 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T09:35:36.786 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.786 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.786 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_metrics": { 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_metrics", 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T09:35:36.788 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.788 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T09:35:36.788 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T09:35:36.789 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.789 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.789 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_step": { 2023-10-18T09:35:36.789 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_step", 2023-10-18T09:35:36.790 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.790 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.790 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.790 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.5", 2023-10-18T09:35:36.790 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0.001", 2023-10-18T09:35:36.791 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "0.999", 2023-10-18T09:35:36.791 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.791 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "aggressiveness of optimization", 2023-10-18T09:35:36.791 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T09:35:36.791 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: "end_time": { 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_time", 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.792 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.793 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.793 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2359", 2023-10-18T09:35:36.793 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.793 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.793 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.794 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T09:35:36.794 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:36.794 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.794 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "end_weekday": { 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_weekday", 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.795 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:36.796 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.797 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.797 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.797 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.797 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.798 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.798 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.798 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.798 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.798 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.799 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.800 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.800 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.800 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.800 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.800 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.801 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.801 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.801 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.801 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.802 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.802 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.802 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.802 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.802 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.804 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.804 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.804 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.804 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.805 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.806 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.806 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.806 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.806 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.806 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.807 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.807 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.807 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.807 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.807 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.809 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.810 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.810 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.810 INFO:teuthology.orchestra.run.smithi061.stdout: "min_score": { 2023-10-18T09:35:36.810 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_score", 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.811 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.812 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.812 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T09:35:36.812 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.812 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.812 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.813 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.814 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "upmap", 2023-10-18T09:35:36.814 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.814 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.814 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-compat", 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: "none", 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap" 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Balancer mode", 2023-10-18T09:35:36.815 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.816 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.816 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.816 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_ids": { 2023-10-18T09:35:36.816 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_ids", 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.817 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.818 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.818 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T09:35:36.818 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.818 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.818 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.819 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:36.819 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:36.819 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.819 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_deviation": { 2023-10-18T09:35:36.822 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_deviation", 2023-10-18T09:35:36.822 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.822 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.822 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.822 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:36.823 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:36.823 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.823 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.823 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T09:35:36.823 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T09:35:36.824 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.824 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.824 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_optimizations": { 2023-10-18T09:35:36.824 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_optimizations", 2023-10-18T09:35:36.825 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:36.825 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.825 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.825 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:36.825 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.826 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.826 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.826 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T09:35:36.826 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.827 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.827 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.828 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm", 2023-10-18T09:35:36.828 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.828 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_down_multiplier": { 2023-10-18T09:35:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_down_multiplier", 2023-10-18T09:35:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.830 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.830 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:36.830 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.830 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.830 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_refresh_rate": { 2023-10-18T09:35:36.832 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_refresh_rate", 2023-10-18T09:35:36.832 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.832 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.832 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.832 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "20", 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.834 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.834 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_starting_port": { 2023-10-18T09:35:36.834 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_starting_port", 2023-10-18T09:35:36.834 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.835 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.835 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.835 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4721", 2023-10-18T09:35:36.835 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.835 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.837 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_ptrace": { 2023-10-18T09:35:36.837 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_ptrace", 2023-10-18T09:35:36.837 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.837 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.837 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T09:35:36.838 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T09:35:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_interval": { 2023-10-18T09:35:36.839 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_interval", 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.840 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.841 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.841 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T09:35:36.841 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.841 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.842 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_memory_target_ratio": { 2023-10-18T09:35:36.842 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T09:35:36.842 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:36.842 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.7", 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.843 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T09:35:36.844 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.844 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.844 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.844 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm_log_destination": { 2023-10-18T09:35:36.845 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm_log_destination", 2023-10-18T09:35:36.845 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.845 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.845 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.845 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.846 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.846 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.846 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.846 INFO:teuthology.orchestra.run.smithi061.stdout: "file", 2023-10-18T09:35:36.847 INFO:teuthology.orchestra.run.smithi061.stdout: "file,syslog", 2023-10-18T09:35:36.847 INFO:teuthology.orchestra.run.smithi061.stdout: "syslog" 2023-10-18T09:35:36.847 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.847 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T09:35:36.847 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: "cgroups_split": { 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cgroups_split", 2023-10-18T09:35:36.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.849 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.849 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.849 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.849 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.849 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.850 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.850 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T09:35:36.850 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.850 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: "config_checks_enabled": { 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_checks_enabled", 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.851 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.852 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T09:35:36.853 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.853 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.853 INFO:teuthology.orchestra.run.smithi061.stdout: "config_dashboard": { 2023-10-18T09:35:36.854 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_dashboard", 2023-10-18T09:35:36.854 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.854 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.854 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.854 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.855 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.856 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.856 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_alertmanager": { 2023-10-18T09:35:36.856 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_alertmanager", 2023-10-18T09:35:36.856 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.857 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.857 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.857 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T09:35:36.857 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.858 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:36.858 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_base": { 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_base", 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.859 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.860 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.860 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T09:35:36.860 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.860 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.860 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Container image name, without the tag", 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.861 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_elasticsearch": { 2023-10-18T09:35:36.862 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_elasticsearch", 2023-10-18T09:35:36.862 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.862 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.862 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.862 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "elasticsearch container image", 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.863 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.864 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.864 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_grafana": { 2023-10-18T09:35:36.864 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_grafana", 2023-10-18T09:35:36.864 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.865 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.866 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:36.866 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.866 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.867 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_haproxy": { 2023-10-18T09:35:36.867 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_haproxy", 2023-10-18T09:35:36.867 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.867 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.868 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.868 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T09:35:36.868 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.868 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.868 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "HAproxy container image", 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.869 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_agent": { 2023-10-18T09:35:36.870 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_agent", 2023-10-18T09:35:36.870 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.870 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.870 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.870 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger agent container image", 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.871 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.872 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.872 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_collector": { 2023-10-18T09:35:36.872 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_collector", 2023-10-18T09:35:36.872 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.873 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.873 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.873 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T09:35:36.873 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.874 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.875 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_query": { 2023-10-18T09:35:36.875 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_query", 2023-10-18T09:35:36.875 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.875 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.876 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.876 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T09:35:36.876 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.876 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.876 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger query container image", 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.877 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_keepalived": { 2023-10-18T09:35:36.878 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_keepalived", 2023-10-18T09:35:36.878 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.878 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.878 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.878 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T09:35:36.879 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.879 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.879 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.879 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Keepalived container image", 2023-10-18T09:35:36.879 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.880 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.880 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.880 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_loki": { 2023-10-18T09:35:36.880 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_loki", 2023-10-18T09:35:36.881 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.881 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.881 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.881 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T09:35:36.881 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.882 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.882 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.882 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Loki container image", 2023-10-18T09:35:36.882 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.882 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.883 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.883 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_node_exporter": { 2023-10-18T09:35:36.883 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_node_exporter", 2023-10-18T09:35:36.883 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.884 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.885 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:36.885 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.885 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.885 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_nvmeof": { 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_nvmeof", 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.886 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T09:35:36.887 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.887 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.887 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.887 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Nvme-of container image", 2023-10-18T09:35:36.887 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_prometheus": { 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_prometheus", 2023-10-18T09:35:36.888 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.889 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.889 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.889 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T09:35:36.889 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.890 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.890 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.890 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:36.890 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.890 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_promtail": { 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_promtail", 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.891 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.892 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Promtail container image", 2023-10-18T09:35:36.893 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.893 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.893 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.893 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_snmp_gateway": { 2023-10-18T09:35:36.894 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_snmp_gateway", 2023-10-18T09:35:36.894 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.894 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.894 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.894 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.895 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: "container_init": { 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_init", 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.896 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.897 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.897 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.897 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.897 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.897 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.898 INFO:teuthology.orchestra.run.smithi061.stdout: "daemon_cache_timeout": { 2023-10-18T09:35:36.899 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "daemon_cache_timeout", 2023-10-18T09:35:36.899 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.899 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.899 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.899 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:36.900 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.900 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.900 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.900 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T09:35:36.900 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "default_cephadm_command_timeout": { 2023-10-18T09:35:36.901 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T09:35:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "900", 2023-10-18T09:35:36.902 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.903 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.904 INFO:teuthology.orchestra.run.smithi061.stdout: "default_registry": { 2023-10-18T09:35:36.904 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_registry", 2023-10-18T09:35:36.904 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.904 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.905 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.905 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io", 2023-10-18T09:35:36.905 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.905 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.905 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.906 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T09:35:36.906 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.906 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.906 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "device_cache_timeout": { 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_cache_timeout", 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.907 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1800", 2023-10-18T09:35:36.908 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.908 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.908 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.908 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache device inventory", 2023-10-18T09:35:36.908 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: "device_enhanced_scan": { 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_enhanced_scan", 2023-10-18T09:35:36.909 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.910 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.910 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.910 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.910 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.910 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.911 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.911 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T09:35:36.911 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.911 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.911 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: "facts_cache_timeout": { 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "facts_cache_timeout", 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.912 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.913 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:36.913 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.913 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.913 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.913 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache host facts data", 2023-10-18T09:35:36.914 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.914 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.914 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.915 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:36.916 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.916 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.916 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.916 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to perform a host check", 2023-10-18T09:35:36.916 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.917 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.917 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.917 INFO:teuthology.orchestra.run.smithi061.stdout: "inventory_list_all": { 2023-10-18T09:35:36.917 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "inventory_list_all", 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.918 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.919 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.919 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T09:35:36.919 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.919 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.920 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.920 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.920 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.920 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.921 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.922 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.922 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.922 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.922 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.922 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.924 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.924 INFO:teuthology.orchestra.run.smithi061.stdout: "log_refresh_metadata": { 2023-10-18T09:35:36.924 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_refresh_metadata", 2023-10-18T09:35:36.924 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.925 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.925 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.925 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.925 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.926 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.926 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T09:35:36.926 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.926 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.926 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.927 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.927 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.927 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.927 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.928 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T09:35:36.929 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.929 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.929 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.929 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.930 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.931 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.931 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.931 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.931 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.931 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.932 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.933 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.933 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.933 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:36.933 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:36.934 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.934 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.934 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.934 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.935 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.935 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.935 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.935 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.935 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T09:35:36.936 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.937 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.937 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.937 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.937 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.937 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.938 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.938 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T09:35:36.938 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.938 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.938 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.939 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T09:35:36.939 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T09:35:36.939 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.939 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "*", 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.940 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: "max_count_per_host": { 2023-10-18T09:35:36.941 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_count_per_host", 2023-10-18T09:35:36.942 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.942 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.942 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.942 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:36.942 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.943 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.943 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.943 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of daemons per service per host", 2023-10-18T09:35:36.943 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.943 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.944 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.944 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd_draining_count": { 2023-10-18T09:35:36.944 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_osd_draining_count", 2023-10-18T09:35:36.944 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.945 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.945 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.945 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:36.945 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.945 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.946 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.946 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T09:35:36.946 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.946 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.946 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.947 INFO:teuthology.orchestra.run.smithi061.stdout: "migration_current": { 2023-10-18T09:35:36.947 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "migration_current", 2023-10-18T09:35:36.947 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.947 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.948 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "internal - do not modify", 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:36.949 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:36.950 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.950 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.950 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.950 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "root", 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm-package", 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: "root" 2023-10-18T09:35:36.951 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.952 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T09:35:36.952 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.952 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.952 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "prometheus_alerts_path": { 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_alerts_path", 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.953 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T09:35:36.954 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.954 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.954 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.954 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T09:35:36.954 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.955 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.955 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.955 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_insecure": { 2023-10-18T09:35:36.955 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_insecure", 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.956 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.957 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.957 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T09:35:36.957 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.957 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.957 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.958 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_password": { 2023-10-18T09:35:36.958 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_password", 2023-10-18T09:35:36.958 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.959 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.960 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T09:35:36.960 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.960 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.960 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.961 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_url": { 2023-10-18T09:35:36.961 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_url", 2023-10-18T09:35:36.961 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.961 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.961 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.962 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.962 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.962 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.962 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.962 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_username": { 2023-10-18T09:35:36.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_username", 2023-10-18T09:35:36.964 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.964 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.964 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.964 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.964 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.965 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.966 INFO:teuthology.orchestra.run.smithi061.stdout: "secure_monitoring_stack": { 2023-10-18T09:35:36.966 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "secure_monitoring_stack", 2023-10-18T09:35:36.966 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.967 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.967 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.967 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.968 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.968 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T09:35:36.968 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.968 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: "service_discovery_port": { 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "service_discovery_port", 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.969 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.970 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", 2023-10-18T09:35:36.970 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.970 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.970 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.970 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "cephadm service discovery port", 2023-10-18T09:35:36.971 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.971 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.971 INFO:teuthology.orchestra.run.smithi061.stdout: "ssh_config_file": { 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssh_config_file", 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.973 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.973 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.973 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T09:35:36.973 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.973 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.974 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.974 INFO:teuthology.orchestra.run.smithi061.stdout: "use_agent": { 2023-10-18T09:35:36.974 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_agent", 2023-10-18T09:35:36.974 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.975 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.975 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.975 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T09:35:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: "use_repo_digest": { 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_repo_digest", 2023-10-18T09:35:36.977 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.978 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.978 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.978 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.978 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.979 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.980 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_failed_host_check": { 2023-10-18T09:35:36.980 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_failed_host_check", 2023-10-18T09:35:36.980 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.980 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.981 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.981 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.981 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.981 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.983 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_daemons": { 2023-10-18T09:35:36.983 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_daemons", 2023-10-18T09:35:36.983 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.983 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.984 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T09:35:36.985 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.985 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.985 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.985 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_hosts": { 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_hosts", 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:36.986 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.987 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.987 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.987 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T09:35:36.987 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.987 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:36.988 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crash", 2023-10-18T09:35:36.989 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:36.989 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:36.989 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:36.989 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:36.989 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:36.990 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.990 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.990 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:36.990 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.991 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.991 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.991 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.991 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.991 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.992 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.993 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:36.993 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.994 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.995 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:36.995 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.995 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.995 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.995 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:36.996 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:36.998 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:36.999 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:36.999 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:36.999 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:36.999 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:36.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.000 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.000 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.000 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.000 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.000 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.001 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.001 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.001 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.001 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.002 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.002 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.002 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.002 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.003 INFO:teuthology.orchestra.run.smithi061.stdout: "retain_interval": { 2023-10-18T09:35:37.003 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retain_interval", 2023-10-18T09:35:37.003 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.003 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.003 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.004 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "31536000", 2023-10-18T09:35:37.004 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.004 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.004 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.004 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T09:35:37.005 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.005 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.005 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.005 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_recent_interval": { 2023-10-18T09:35:37.006 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_recent_interval", 2023-10-18T09:35:37.006 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.006 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.006 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.006 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1209600", 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.007 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.008 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T09:35:37.009 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T09:35:37.010 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.010 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.010 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.010 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:37.010 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.011 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.011 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.011 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.011 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.011 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.012 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.013 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.013 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.013 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.013 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.013 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.014 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.014 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.014 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.014 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.015 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.017 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.017 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_ENABLED": { 2023-10-18T09:35:37.017 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T09:35:37.017 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.018 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.018 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.018 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.018 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.019 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.019 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.019 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.019 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.019 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.020 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.020 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T09:35:37.020 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T09:35:37.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.021 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.021 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.021 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.022 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.023 INFO:teuthology.orchestra.run.smithi061.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T09:35:37.023 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T09:35:37.023 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.023 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.024 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.025 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.025 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.025 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.025 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T09:35:37.026 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T09:35:37.026 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.026 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.026 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.027 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.027 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.027 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.027 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.027 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T09:35:37.028 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.029 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.029 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.029 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.029 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.029 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.030 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.030 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.030 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.030 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.031 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.031 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T09:35:37.031 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T09:35:37.031 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.032 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.032 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.032 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.032 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.032 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.033 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.033 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.033 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.034 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.034 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.035 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T09:35:37.035 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T09:35:37.035 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.036 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.036 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.037 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.037 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.037 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.037 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.038 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.038 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.038 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.039 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.040 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T09:35:37.040 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T09:35:37.040 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.041 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.041 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.041 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.041 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.042 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.042 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.042 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.042 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.042 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.043 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.043 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T09:35:37.043 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T09:35:37.043 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.044 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.045 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.045 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.045 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.045 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.046 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T09:35:37.046 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T09:35:37.046 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.046 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.047 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.047 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.047 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.047 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.047 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.048 INFO:teuthology.orchestra.run.smithi061.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.049 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.050 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.050 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.050 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.050 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.050 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.051 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.051 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T09:35:37.051 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T09:35:37.051 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.052 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.053 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.053 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.053 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.053 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.054 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T09:35:37.054 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T09:35:37.054 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.054 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.054 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.055 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.056 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.056 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.056 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_URL": { 2023-10-18T09:35:37.056 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_URL", 2023-10-18T09:35:37.057 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.057 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.057 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.057 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.058 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.058 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.058 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.058 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.058 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.059 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.059 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.059 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:37.060 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.061 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.061 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.061 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.061 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.061 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.062 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.062 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T09:35:37.062 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T09:35:37.062 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.063 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.064 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T09:35:37.065 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T09:35:37.065 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.065 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.065 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.065 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.066 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.067 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.067 INFO:teuthology.orchestra.run.smithi061.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T09:35:37.067 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.068 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.069 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.069 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.069 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.069 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.069 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.071 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.071 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.071 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.071 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.071 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T09:35:37.072 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T09:35:37.073 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.073 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.073 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.073 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.074 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.074 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.074 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.074 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.075 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.075 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.075 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T09:35:37.075 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.076 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.077 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.077 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.077 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.078 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.078 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T09:35:37.078 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T09:35:37.078 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.079 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.081 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T09:35:37.081 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T09:35:37.081 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.081 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.081 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.082 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.082 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.082 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.082 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.082 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.083 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.083 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.083 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T09:35:37.084 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T09:35:37.084 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.084 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.084 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.084 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.085 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.085 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.085 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.085 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.085 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.086 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.086 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.086 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T09:35:37.086 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.087 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.088 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.088 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.088 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.088 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.089 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T09:35:37.089 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T09:35:37.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.089 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.090 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.091 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.091 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.091 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.091 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.092 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.093 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.093 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.093 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.093 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.093 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.094 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.094 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T09:35:37.094 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T09:35:37.094 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.095 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.096 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.096 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.096 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.096 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.097 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.098 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.098 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.098 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.098 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.098 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.099 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.099 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.099 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T09:35:37.099 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T09:35:37.100 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.100 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.100 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.100 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T09:35:37.100 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.101 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.102 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T09:35:37.102 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T09:35:37.102 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.102 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.103 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.103 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:37.103 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.103 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.103 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.104 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.104 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.104 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.104 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.105 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8", 2023-10-18T09:35:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.107 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.107 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.107 INFO:teuthology.orchestra.run.smithi061.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T09:35:37.107 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "45", 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.108 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.109 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.109 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.109 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.109 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.109 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.110 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T09:35:37.110 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T09:35:37.110 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.110 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.111 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.111 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.111 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.111 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.111 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.112 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T09:35:37.113 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T09:35:37.113 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.113 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.113 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.113 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:37.114 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.114 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.114 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.114 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.114 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T09:35:37.115 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.116 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.116 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.116 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.116 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.116 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.118 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T09:35:37.118 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T09:35:37.118 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.118 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.119 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.119 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.119 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.120 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.120 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.120 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.122 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.122 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.123 INFO:teuthology.orchestra.run.smithi061.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T09:35:37.123 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T09:35:37.123 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.123 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.123 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.124 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.125 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.125 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.125 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T09:35:37.125 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T09:35:37.126 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.126 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.126 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.126 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:37.126 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.127 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.127 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.127 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.127 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.127 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.128 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.128 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T09:35:37.128 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T09:35:37.128 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.129 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.130 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.130 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.130 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.130 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.131 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T09:35:37.131 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T09:35:37.131 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.131 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.131 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.132 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", 2023-10-18T09:35:37.133 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.134 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.134 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.134 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.134 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.134 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.135 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.135 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.135 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.135 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.135 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.136 INFO:teuthology.orchestra.run.smithi061.stdout: "crt_file": { 2023-10-18T09:35:37.136 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crt_file", 2023-10-18T09:35:37.136 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.136 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.137 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.137 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.137 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.137 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.137 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.138 INFO:teuthology.orchestra.run.smithi061.stdout: "debug": { 2023-10-18T09:35:37.139 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "debug", 2023-10-18T09:35:37.139 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.139 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.139 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.139 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.140 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.140 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.140 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.140 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable/disable debug options", 2023-10-18T09:35:37.140 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.141 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.141 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.141 INFO:teuthology.orchestra.run.smithi061.stdout: "jwt_token_ttl": { 2023-10-18T09:35:37.141 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "jwt_token_ttl", 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "28800", 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.142 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.144 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:37.144 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:37.144 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.145 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.145 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.145 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.145 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.145 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.146 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.146 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.146 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.146 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.147 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.148 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.148 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.148 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.148 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.148 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.149 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.149 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.149 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.149 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.149 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.150 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.151 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.151 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.151 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.151 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.152 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.153 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.153 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.153 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.153 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.154 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.155 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.155 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.155 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.155 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.155 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.156 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.157 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.158 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.158 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.158 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.158 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.158 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: "motd": { 2023-10-18T09:35:37.159 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "motd", 2023-10-18T09:35:37.160 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.160 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.160 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.160 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "The message of the day", 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.161 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.162 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.162 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect_resolve_ip_addr": { 2023-10-18T09:35:37.162 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T09:35:37.162 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.163 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.163 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.163 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.163 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.163 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.165 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:37.166 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.166 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.166 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.166 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.166 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:37.168 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.168 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.168 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.168 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8080", 2023-10-18T09:35:37.168 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.169 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.169 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.169 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.170 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.171 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.171 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.171 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.171 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.171 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.172 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl_server_port": { 2023-10-18T09:35:37.173 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl_server_port", 2023-10-18T09:35:37.173 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.173 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.173 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.173 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8443", 2023-10-18T09:35:37.174 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.174 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.174 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.174 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.174 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.175 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.175 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.175 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:37.176 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:37.176 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.176 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.177 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.177 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "redirect", 2023-10-18T09:35:37.177 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.177 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.177 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.178 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.178 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect" 2023-10-18T09:35:37.178 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.178 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.179 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.179 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.179 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.179 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:37.180 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:37.180 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.180 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.180 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.180 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.181 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.182 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.182 INFO:teuthology.orchestra.run.smithi061.stdout: "url_prefix": { 2023-10-18T09:35:37.182 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url_prefix", 2023-10-18T09:35:37.183 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.183 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.183 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.183 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.183 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.184 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.185 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "devicehealth", 2023-10-18T09:35:37.186 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.186 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.186 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.186 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_monitoring": { 2023-10-18T09:35:37.186 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_monitoring", 2023-10-18T09:35:37.187 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.187 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.187 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.187 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.187 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "monitor device health metrics", 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.188 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.189 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.190 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.190 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.190 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.190 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.190 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.191 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.191 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.191 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.191 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.191 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.193 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.193 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.193 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.193 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.193 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.195 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.195 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.195 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.196 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.196 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.196 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.196 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.196 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "mark_out_threshold": { 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mark_out_threshold", 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2419200", 2023-10-18T09:35:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T09:35:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": { 2023-10-18T09:35:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_name", 2023-10-18T09:35:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "device_health_metrics", 2023-10-18T09:35:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T09:35:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: "retention_period": { 2023-10-18T09:35:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retention_period", 2023-10-18T09:35:37.209 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.209 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.209 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.209 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15552000", 2023-10-18T09:35:37.209 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.210 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.210 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.210 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain device health metrics", 2023-10-18T09:35:37.210 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.210 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.211 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.211 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_frequency": { 2023-10-18T09:35:37.211 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_frequency", 2023-10-18T09:35:37.212 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.212 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.212 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.212 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:37.213 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.213 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.213 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.213 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T09:35:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.215 INFO:teuthology.orchestra.run.smithi061.stdout: "self_heal": { 2023-10-18T09:35:37.215 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "self_heal", 2023-10-18T09:35:37.215 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.215 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.216 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.216 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.216 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.216 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.216 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.217 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T09:35:37.217 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.217 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.217 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.218 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:37.218 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:37.218 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.219 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.219 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.219 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:37.219 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.220 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.220 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T09:35:37.220 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.221 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.221 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.221 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_threshold": { 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_threshold", 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7257600", 2023-10-18T09:35:37.222 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.223 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.223 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.223 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T09:35:37.223 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.223 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.224 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "diskprediction_local", 2023-10-18T09:35:37.225 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.225 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.225 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.225 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.225 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.226 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.227 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.227 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.227 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.227 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.227 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.228 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.228 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.228 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.228 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.228 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.229 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.230 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.230 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.230 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.230 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.230 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.232 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.232 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.232 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.232 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.232 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.233 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.234 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.234 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.234 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.234 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.234 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.235 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.236 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.236 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.236 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.236 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.236 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.237 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.238 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.238 INFO:teuthology.orchestra.run.smithi061.stdout: "predict_interval": { 2023-10-18T09:35:37.238 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predict_interval", 2023-10-18T09:35:37.238 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.239 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.240 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.240 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.240 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.240 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "predictor_model": { 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predictor_model", 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.241 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "prophetstor", 2023-10-18T09:35:37.242 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.242 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.242 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.242 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.242 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:37.243 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.244 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.244 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.244 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:37.244 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.245 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.245 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.245 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.245 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.245 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.246 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "influx", 2023-10-18T09:35:37.247 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": false, 2023-10-18T09:35:37.247 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "influxdb python module not found", 2023-10-18T09:35:37.247 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.247 INFO:teuthology.orchestra.run.smithi061.stdout: "batch_size": { 2023-10-18T09:35:37.247 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "batch_size", 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5000", 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.248 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.249 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.249 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T09:35:37.249 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.249 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.249 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: "database": { 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "database", 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.251 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "ceph", 2023-10-18T09:35:37.251 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.251 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.251 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.251 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T09:35:37.252 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.252 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.252 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.252 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": { 2023-10-18T09:35:37.253 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "hostname", 2023-10-18T09:35:37.253 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.253 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.253 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.253 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server hostname", 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.254 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.255 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.255 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:37.255 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:37.255 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "30", 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "5", 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.256 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.257 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T09:35:37.257 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.257 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.257 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.259 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.259 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.259 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.259 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.259 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.260 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.261 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.261 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.261 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.261 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.262 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.262 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.262 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.262 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.262 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.263 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.263 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.263 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.263 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.263 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.264 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.264 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.264 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.264 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.265 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.266 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.266 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.266 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.266 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.266 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.267 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.267 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.267 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.267 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.267 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.269 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.269 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.269 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.269 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.269 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "password", 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.271 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.272 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.272 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.272 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.272 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.273 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.273 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "password of InfluxDB server user", 2023-10-18T09:35:37.273 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.274 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.274 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "port": { 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "port", 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.275 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8086", 2023-10-18T09:35:37.276 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.276 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.277 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.277 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server port", 2023-10-18T09:35:37.277 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.277 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.277 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.278 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:37.278 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:37.278 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.278 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.279 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.279 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "false", 2023-10-18T09:35:37.279 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.279 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.279 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.280 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:37.280 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.280 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.280 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "threads": { 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threads", 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.281 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:37.282 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:37.282 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "32", 2023-10-18T09:35:37.282 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.282 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T09:35:37.282 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.283 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.283 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.283 INFO:teuthology.orchestra.run.smithi061.stdout: "username": { 2023-10-18T09:35:37.283 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "username", 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.284 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.285 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.285 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "username of InfluxDB server user", 2023-10-18T09:35:37.285 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.285 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.285 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.286 INFO:teuthology.orchestra.run.smithi061.stdout: "verify_ssl": { 2023-10-18T09:35:37.286 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "verify_ssl", 2023-10-18T09:35:37.286 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.286 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "true", 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.287 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:37.288 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.288 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.288 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "insights", 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.289 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.290 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.290 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.290 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.290 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.290 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.291 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.292 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.292 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.292 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.292 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.292 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.293 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.294 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.294 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.294 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.294 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.294 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.295 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.297 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.297 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.297 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.297 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.297 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.298 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.299 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.299 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.299 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.299 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.299 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.300 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.301 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.301 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.301 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.301 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.301 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.302 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.303 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "iostat", 2023-10-18T09:35:37.303 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.303 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.304 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.305 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.305 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.305 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.305 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.305 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.306 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.306 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.306 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.306 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.306 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.307 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.307 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.307 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.307 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.307 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.308 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.308 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.308 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.308 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.309 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.309 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.309 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.309 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.309 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.310 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.310 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.310 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.310 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.311 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.312 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.312 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.312 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.312 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.312 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.313 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.314 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.314 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.314 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.314 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.315 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.316 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.316 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.316 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.316 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.316 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.317 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "k8sevents", 2023-10-18T09:35:37.318 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.318 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.318 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.318 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_event_retention_days": { 2023-10-18T09:35:37.318 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ceph_event_retention_days", 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7", 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.319 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.320 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T09:35:37.320 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.320 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.320 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.321 INFO:teuthology.orchestra.run.smithi061.stdout: "config_check_secs": { 2023-10-18T09:35:37.321 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_check_secs", 2023-10-18T09:35:37.321 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.322 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.322 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.322 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:37.323 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "10", 2023-10-18T09:35:37.323 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.323 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.324 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T09:35:37.324 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.325 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.325 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.326 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.326 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.326 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.327 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.327 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.327 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.328 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.328 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.329 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.329 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.329 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.330 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.330 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.331 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.331 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.331 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.332 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.332 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.332 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.333 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.333 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.334 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.334 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.335 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.335 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.335 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.336 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.336 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.336 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.337 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.337 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.337 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.338 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.339 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.339 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.339 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.340 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.340 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.340 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.341 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.341 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.342 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.342 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.342 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.343 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.343 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.343 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.343 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.344 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.344 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.344 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.344 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.344 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.345 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.346 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.346 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.346 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.346 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.346 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.348 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "localpool", 2023-10-18T09:35:37.348 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.348 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.348 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "failure_domain": { 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "failure_domain", 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.349 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "host", 2023-10-18T09:35:37.350 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.350 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.350 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.350 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "failure domain for any created local pool", 2023-10-18T09:35:37.350 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.351 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.352 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.353 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.353 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.353 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.353 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.353 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.354 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.354 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.354 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.354 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.354 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.355 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.356 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.356 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.356 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.356 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.356 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.357 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.358 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.358 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.358 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.358 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.358 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.359 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.360 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.361 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.361 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.361 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.362 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.362 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.362 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.362 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.362 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.363 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.363 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.363 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.363 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.363 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": { 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_size", 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.364 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.365 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.365 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.365 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.365 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.365 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.366 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default min_size for any created local pool", 2023-10-18T09:35:37.366 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T09:35:37.366 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.366 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "num_rep": { 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "num_rep", 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.367 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3", 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default replica count for any created local pool", 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.368 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.369 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.369 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": { 2023-10-18T09:35:37.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_num", 2023-10-18T09:35:37.369 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.370 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.370 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "128", 2023-10-18T09:35:37.370 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.370 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.371 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.371 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T09:35:37.371 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.371 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.371 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: "prefix": { 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prefix", 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.372 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.373 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.373 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.373 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.373 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.373 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name prefix for any created local pool", 2023-10-18T09:35:37.374 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.374 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.374 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.374 INFO:teuthology.orchestra.run.smithi061.stdout: "subtree": { 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "subtree", 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "rack", 2023-10-18T09:35:37.375 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T09:35:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T09:35:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mds_autoscaler", 2023-10-18T09:35:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", 2023-10-18T09:35:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "nfs", 2023-10-18T09:35:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "fail_fs": { 2023-10-18T09:35:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "fail_fs", 2023-10-18T09:35:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T09:35:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator": { 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm", 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "rook", 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "test_orchestrator" 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Orchestrator backend", 2023-10-18T09:35:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_perf_query", 2023-10-18T09:35:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_support", 2023-10-18T09:35:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_autoscaler", 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "threshold": { 2023-10-18T09:35:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threshold", 2023-10-18T09:35:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1.0", 2023-10-18T09:35:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "scaling threshold", 2023-10-18T09:35:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T09:35:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "progress", 2023-10-18T09:35:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_pg_recovery_event": { 2023-10-18T09:35:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_pg_recovery_event", 2023-10-18T09:35:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T09:35:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.518 INFO:teuthology.orchestra.run.smithi061.stdout: "max_completed_events": { 2023-10-18T09:35:37.518 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_completed_events", 2023-10-18T09:35:37.518 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.518 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.519 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.519 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "50", 2023-10-18T09:35:37.519 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.519 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.519 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "number of past completed events to remember", 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.520 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:37.521 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:37.521 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.521 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.521 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.521 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:37.522 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.522 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.522 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.522 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long the module is going to sleep", 2023-10-18T09:35:37.522 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.524 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus", 2023-10-18T09:35:37.524 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.524 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.524 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.525 INFO:teuthology.orchestra.run.smithi061.stdout: "cache": { 2023-10-18T09:35:37.525 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cache", 2023-10-18T09:35:37.525 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.526 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.526 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.526 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.527 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.527 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.527 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.528 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.528 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.528 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.529 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.529 INFO:teuthology.orchestra.run.smithi061.stdout: "exclude_perf_counters": { 2023-10-18T09:35:37.530 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "exclude_perf_counters", 2023-10-18T09:35:37.530 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.530 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.531 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.531 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.531 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.532 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.532 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.533 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T09:35:37.533 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T09:35:37.533 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.533 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.534 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.534 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.534 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.535 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.535 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.535 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.535 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.535 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.536 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.537 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.537 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.537 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.537 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.537 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.538 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.538 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.538 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.538 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.539 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.539 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.539 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.539 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.539 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.540 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.541 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.542 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.543 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.544 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.544 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.544 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.544 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.545 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.546 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.546 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.546 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.546 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.546 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.547 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.547 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.547 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools": { 2023-10-18T09:35:37.547 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools", 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.548 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.549 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.549 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.549 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.549 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.549 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.550 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T09:35:37.550 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T09:35:37.550 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.550 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.551 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300", 2023-10-18T09:35:37.551 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.551 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.551 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.552 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.552 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.552 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.552 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_interval": { 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_interval", 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15.0", 2023-10-18T09:35:37.554 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.554 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.554 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.554 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.554 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.555 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.555 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.555 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:37.555 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:37.556 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.556 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.556 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:37.556 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.557 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.558 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:37.558 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:37.558 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.558 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.559 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.559 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "9283", 2023-10-18T09:35:37.559 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.559 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.559 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.560 INFO:teuthology.orchestra.run.smithi061.stdout: "stale_cache_strategy": { 2023-10-18T09:35:37.561 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stale_cache_strategy", 2023-10-18T09:35:37.561 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.561 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.561 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.561 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "log", 2023-10-18T09:35:37.562 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.562 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.562 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.562 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.562 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.563 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.563 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.563 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:37.563 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:37.564 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.564 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.564 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.564 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "default", 2023-10-18T09:35:37.564 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.565 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.565 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.565 INFO:teuthology.orchestra.run.smithi061.stdout: "default", 2023-10-18T09:35:37.565 INFO:teuthology.orchestra.run.smithi061.stdout: "error" 2023-10-18T09:35:37.565 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.566 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.566 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.566 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.566 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.567 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:37.567 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:37.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.567 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.567 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.568 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:37.568 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:37.568 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:37.568 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.568 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.570 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_support", 2023-10-18T09:35:37.570 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.570 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.571 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.572 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.572 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.572 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.572 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.573 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.574 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.574 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.574 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.574 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.574 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.575 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.575 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.575 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.575 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.576 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.577 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.577 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.577 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.577 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.578 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.578 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.578 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.578 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.578 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.579 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.579 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.579 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.579 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.579 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.580 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.581 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.581 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.582 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.582 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.582 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.582 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.582 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.583 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.584 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.584 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.584 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_snap_create": { 2023-10-18T09:35:37.584 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_snap_create", 2023-10-18T09:35:37.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.585 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.585 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:37.585 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.586 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.586 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.586 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.586 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.586 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: "mirror_snapshot_schedule": { 2023-10-18T09:35:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T09:35:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.588 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.588 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.588 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.588 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.588 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.589 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.589 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.589 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.589 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.589 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.590 INFO:teuthology.orchestra.run.smithi061.stdout: "trash_purge_schedule": { 2023-10-18T09:35:37.590 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "trash_purge_schedule", 2023-10-18T09:35:37.590 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.590 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.591 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.591 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.591 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.591 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.591 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.592 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.593 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "restful", 2023-10-18T09:35:37.593 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.593 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.594 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.594 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { 2023-10-18T09:35:37.594 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_auth", 2023-10-18T09:35:37.594 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.594 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.595 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.596 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.596 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.596 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.596 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.597 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.598 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.598 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.598 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.598 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.599 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.599 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.599 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.599 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.600 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.601 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.601 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.601 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.601 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.601 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.602 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.602 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.602 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.602 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.602 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.603 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.603 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.603 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.603 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.604 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.606 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.606 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.606 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.606 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.606 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.607 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.608 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.608 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.608 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.608 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.608 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.609 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.609 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.609 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.609 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.610 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.610 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.610 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.610 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.610 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.611 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.611 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.611 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.611 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.611 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:37.612 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:37.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.613 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.613 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.613 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.613 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.614 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.614 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.614 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.614 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.614 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.615 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.616 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.616 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.616 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.616 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.616 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.618 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rgw", 2023-10-18T09:35:37.618 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.618 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.619 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.619 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.619 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.619 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.619 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.620 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.621 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.621 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.621 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.621 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.621 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.622 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.622 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.622 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.622 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.622 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.623 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.624 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.624 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.624 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.624 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.624 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.625 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.625 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.625 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.625 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.626 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.626 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.626 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.626 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.626 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.627 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.627 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.627 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.627 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.627 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.628 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.628 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.628 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.628 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.628 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.629 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.629 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.629 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.629 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.630 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.630 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.630 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.630 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.630 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.631 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.631 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.631 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.631 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.632 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.632 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.632 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.632 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.632 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.633 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.633 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rook", 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "drive_group_interval": { 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "drive_group_interval", 2023-10-18T09:35:37.634 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.635 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.635 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300.0", 2023-10-18T09:35:37.635 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.635 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.636 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.636 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T09:35:37.636 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.636 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.636 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.637 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.638 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.638 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.638 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.638 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.638 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.639 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.640 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.640 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.640 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.640 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.641 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.641 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.641 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.641 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.641 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.642 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.642 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.642 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.642 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.643 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.644 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.644 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.644 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.644 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.644 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.645 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.646 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.646 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.646 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.646 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.646 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.647 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.647 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.647 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.647 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.648 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.649 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.649 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.649 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.649 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.650 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.650 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.650 INFO:teuthology.orchestra.run.smithi061.stdout: "storage_class": { 2023-10-18T09:35:37.651 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "storage_class", 2023-10-18T09:35:37.651 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.651 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.652 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.652 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "local", 2023-10-18T09:35:37.652 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.653 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.653 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.653 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T09:35:37.654 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.654 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.655 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.655 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.656 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "selftest", 2023-10-18T09:35:37.656 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.657 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.657 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.657 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.658 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.658 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.659 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.659 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.660 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.660 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.660 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.660 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.660 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.661 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.661 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.661 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.661 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.661 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.662 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.662 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.662 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.663 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.663 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.663 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.663 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.663 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.664 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.664 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.664 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.664 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.664 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.665 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.665 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.665 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.666 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.666 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.666 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.666 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.666 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.667 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.668 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.669 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.669 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.669 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.669 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.670 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.670 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.670 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.670 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.671 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.671 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.671 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.671 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.671 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: "roption1": { 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption1", 2023-10-18T09:35:37.672 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.673 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.673 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.673 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.674 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.674 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.674 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: "roption2": { 2023-10-18T09:35:37.675 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption2", 2023-10-18T09:35:37.676 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.676 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.676 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.676 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "xyz", 2023-10-18T09:35:37.676 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.677 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.677 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.677 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.677 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.677 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.678 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.678 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption1": { 2023-10-18T09:35:37.678 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption1", 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.679 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.680 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.680 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.680 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.680 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.680 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.681 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption2": { 2023-10-18T09:35:37.681 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption2", 2023-10-18T09:35:37.681 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.681 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.682 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.682 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.682 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.682 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.682 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.683 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.683 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.683 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.683 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption3": { 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption3", 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.684 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.685 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.685 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.685 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.685 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.685 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.686 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.686 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.686 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption4": { 2023-10-18T09:35:37.686 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption4", 2023-10-18T09:35:37.687 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.687 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.687 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.687 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.687 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.688 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.688 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.688 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.688 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.689 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.689 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.689 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption5": { 2023-10-18T09:35:37.689 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption5", 2023-10-18T09:35:37.690 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.690 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.690 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.690 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.691 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.691 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.691 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.691 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.691 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.692 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.693 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.693 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.693 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.693 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.693 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.694 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.694 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.694 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.694 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.695 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption7": { 2023-10-18T09:35:37.695 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption7", 2023-10-18T09:35:37.695 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.695 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.695 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "42", 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.696 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "testkey": { 2023-10-18T09:35:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testkey", 2023-10-18T09:35:37.698 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.698 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.698 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.698 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.698 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.699 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.699 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.699 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.699 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.700 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.700 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.700 INFO:teuthology.orchestra.run.smithi061.stdout: "testlkey": { 2023-10-18T09:35:37.700 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testlkey", 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.701 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.702 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.702 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.702 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.702 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.702 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.703 INFO:teuthology.orchestra.run.smithi061.stdout: "testnewline": { 2023-10-18T09:35:37.703 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testnewline", 2023-10-18T09:35:37.703 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.703 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.704 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.706 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snap_schedule", 2023-10-18T09:35:37.706 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.706 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.706 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.707 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_m_granularity": { 2023-10-18T09:35:37.707 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_m_granularity", 2023-10-18T09:35:37.707 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.707 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.707 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.708 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.708 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.708 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.708 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T09:35:37.709 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.709 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.709 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.709 INFO:teuthology.orchestra.run.smithi061.stdout: "dump_on_update": { 2023-10-18T09:35:37.710 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dump_on_update", 2023-10-18T09:35:37.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.710 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.710 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.711 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.711 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.711 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.711 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", 2023-10-18T09:35:37.712 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.712 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.712 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.712 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.713 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.714 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.714 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.714 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.714 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.714 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.715 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.715 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.715 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.715 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.715 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.716 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.716 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.716 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.717 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.718 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.718 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.718 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.718 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.718 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.719 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.719 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.719 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.719 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.720 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.720 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.720 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.720 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.720 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.721 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.721 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.721 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.721 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.721 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.722 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.722 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.722 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.722 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.722 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.723 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.724 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.724 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.724 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.724 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.724 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stats", 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.728 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.728 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.728 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.728 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.728 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.729 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.730 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.730 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.730 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.730 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.730 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.731 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.731 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.731 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.731 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.732 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.732 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.732 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.732 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.732 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.733 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.733 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.733 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.733 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.734 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.734 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.734 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.734 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.735 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.735 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.735 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.735 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.735 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.736 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.736 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.736 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.736 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.736 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.737 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.737 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.737 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.737 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.738 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.738 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.738 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.738 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.741 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.742 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.742 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.742 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.742 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.743 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.743 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.743 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.743 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.744 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.744 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.744 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.744 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.745 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "status", 2023-10-18T09:35:37.746 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.746 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.746 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.746 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.747 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.747 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.747 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.748 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.748 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.748 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.748 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.748 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.749 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.749 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.749 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.749 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.750 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.750 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.750 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.750 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.751 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.751 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.751 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.751 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.752 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.752 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.752 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.752 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.753 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.754 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.754 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.754 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.754 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.755 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.755 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.755 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.755 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.755 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.756 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.756 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.756 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.756 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.756 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.757 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.757 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.757 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.757 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.757 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.758 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.758 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.758 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.759 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.759 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.759 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.759 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.759 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.760 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.760 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.760 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.760 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.761 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.761 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.762 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telegraf", 2023-10-18T09:35:37.762 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.762 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.763 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.763 INFO:teuthology.orchestra.run.smithi061.stdout: "address": { 2023-10-18T09:35:37.763 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "address", 2023-10-18T09:35:37.763 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.763 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.764 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.765 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.765 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.765 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.766 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:37.766 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:37.766 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.766 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.766 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.767 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15", 2023-10-18T09:35:37.767 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.767 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.767 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.767 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.768 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.769 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.769 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.769 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.770 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.770 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.770 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.770 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.770 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.771 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.772 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.773 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.773 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.773 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.773 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.774 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.774 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.774 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.774 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.774 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.775 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.775 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.775 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.776 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.777 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.777 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.777 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.778 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.778 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.778 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.779 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.779 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.779 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.780 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.780 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.780 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.781 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.781 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.781 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.782 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.783 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.783 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.783 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.784 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.784 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.784 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.785 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.785 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.785 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.785 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.786 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.786 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telemetry", 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.787 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.788 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_basic": { 2023-10-18T09:35:37.788 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_basic", 2023-10-18T09:35:37.788 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.788 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.789 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.789 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.789 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.789 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.789 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.790 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T09:35:37.790 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.790 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.790 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.791 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_crash": { 2023-10-18T09:35:37.791 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_crash", 2023-10-18T09:35:37.791 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.791 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.791 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.792 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.792 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.792 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.792 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.792 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_device": { 2023-10-18T09:35:37.793 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_device", 2023-10-18T09:35:37.794 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.794 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.794 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.794 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.795 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.796 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.796 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_ident": { 2023-10-18T09:35:37.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_ident", 2023-10-18T09:35:37.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.797 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.797 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.797 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.797 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.797 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.798 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.798 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T09:35:37.798 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.798 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.798 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_perf": { 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_perf", 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.799 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.800 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.800 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.800 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.800 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T09:35:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.802 INFO:teuthology.orchestra.run.smithi061.stdout: "contact": { 2023-10-18T09:35:37.802 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "contact", 2023-10-18T09:35:37.802 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.802 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.802 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.803 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: "description": { 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "description", 2023-10-18T09:35:37.804 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.805 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.805 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.805 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.805 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.805 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.806 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.806 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.806 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.806 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.806 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.807 INFO:teuthology.orchestra.run.smithi061.stdout: "device_url": { 2023-10-18T09:35:37.807 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_url", 2023-10-18T09:35:37.807 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.807 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.808 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.809 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.809 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.809 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.810 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:37.810 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:37.810 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.810 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.810 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.811 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.812 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.812 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.812 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:37.813 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:37.813 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.813 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.813 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.813 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "24", 2023-10-18T09:35:37.814 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "8", 2023-10-18T09:35:37.814 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.814 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.814 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.814 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.815 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.815 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.815 INFO:teuthology.orchestra.run.smithi061.stdout: "last_opt_revision": { 2023-10-18T09:35:37.815 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "last_opt_revision", 2023-10-18T09:35:37.816 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.816 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.816 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.816 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1", 2023-10-18T09:35:37.816 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.817 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.818 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard": { 2023-10-18T09:35:37.818 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard", 2023-10-18T09:35:37.818 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.818 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.819 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.820 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.820 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.820 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.820 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard_description": { 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard_description", 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.821 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.822 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.822 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.822 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.822 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.823 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.823 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.823 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.823 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.824 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.824 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.824 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.824 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.824 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.825 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.826 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.826 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.826 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.826 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.826 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.827 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.827 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.827 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.827 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.828 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.829 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.829 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.829 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.829 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.830 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.831 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.831 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.831 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.831 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.831 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.832 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.832 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.832 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.832 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.832 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.833 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.833 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.833 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.833 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.834 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.834 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.834 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.834 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.835 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.835 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.835 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.835 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.835 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.836 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.836 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.836 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.836 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "organization": { 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "organization", 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.837 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.838 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.838 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.838 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.838 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.838 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.839 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.839 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.839 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { 2023-10-18T09:35:37.839 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", 2023-10-18T09:35:37.840 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.840 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.840 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.840 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.840 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.841 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.842 INFO:teuthology.orchestra.run.smithi061.stdout: "url": { 2023-10-18T09:35:37.842 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url", 2023-10-18T09:35:37.842 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.842 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.843 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.844 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.844 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.844 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.844 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.844 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.845 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "test_orchestrator", 2023-10-18T09:35:37.845 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.846 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.846 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.846 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.846 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.846 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.847 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.848 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.848 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.848 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.848 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.848 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.849 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.850 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.850 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.850 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.850 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.851 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.852 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.852 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.852 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.852 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.853 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.854 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.854 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.854 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.854 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.854 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.855 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.855 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.855 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.855 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.855 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.856 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.857 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.857 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.857 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.857 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.857 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.858 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.858 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.858 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.858 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.859 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "volumes", 2023-10-18T09:35:37.860 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.860 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.860 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.860 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.860 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.861 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.861 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.861 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.861 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.861 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.862 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.862 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.862 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.862 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.862 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.863 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.864 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.865 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.865 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.865 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.865 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.865 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.866 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.866 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.866 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.867 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.867 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.867 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.867 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.868 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.868 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.868 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.868 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.869 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.869 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.869 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.869 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.869 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.871 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.871 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.871 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.871 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.872 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.872 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.872 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.872 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.872 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.874 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_clones": { 2023-10-18T09:35:37.874 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_clones", 2023-10-18T09:35:37.874 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.874 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4", 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.875 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T09:35:37.876 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.876 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.876 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.876 INFO:teuthology.orchestra.run.smithi061.stdout: "periodic_async_work": { 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "periodic_async_work", 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.877 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.878 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.878 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.878 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Periodically check for async work", 2023-10-18T09:35:37.878 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.878 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.879 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.879 INFO:teuthology.orchestra.run.smithi061.stdout: "snapshot_clone_delay": { 2023-10-18T09:35:37.879 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snapshot_clone_delay", 2023-10-18T09:35:37.880 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.880 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.880 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.880 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:37.880 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.881 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.881 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.881 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T09:35:37.881 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.881 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.882 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.882 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.882 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.883 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix", 2023-10-18T09:35:37.883 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.883 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.883 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.883 INFO:teuthology.orchestra.run.smithi061.stdout: "discovery_interval": { 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "discovery_interval", 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "100", 2023-10-18T09:35:37.884 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.885 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.885 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.885 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.885 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.885 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "identifier": { 2023-10-18T09:35:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "identifier", 2023-10-18T09:35:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.887 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.887 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.887 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.887 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.887 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.889 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:37.889 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:37.889 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.889 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.889 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.890 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:37.890 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.890 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.890 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.890 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.891 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.891 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.891 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.891 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.892 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.892 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.892 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.892 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.892 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.893 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.894 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.894 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.894 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.894 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.894 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.895 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.896 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.896 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.896 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.896 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.896 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.897 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.897 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.897 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.897 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.897 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.898 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.898 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.898 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.898 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.899 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.899 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.899 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.899 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.899 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.900 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.901 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.901 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.901 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.901 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.901 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.902 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.903 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.903 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.903 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.903 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.903 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.904 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.904 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.904 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.904 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_host": { 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_host", 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.905 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.906 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.906 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.906 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.906 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.907 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.907 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.907 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.907 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_port": { 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_port", 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10051", 2023-10-18T09:35:37.908 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.909 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.909 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.909 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.909 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.909 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.910 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.910 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_sender": { 2023-10-18T09:35:37.910 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_sender", 2023-10-18T09:35:37.910 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.911 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.911 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:37.911 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T09:35:37.911 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.911 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.912 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.912 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.912 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.912 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.912 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.913 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:37.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.914 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.914 INFO:teuthology.orchestra.run.smithi061.stdout: "modules": [ 2023-10-18T09:35:37.914 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm", 2023-10-18T09:35:37.914 INFO:teuthology.orchestra.run.smithi061.stdout: "dashboard", 2023-10-18T09:35:37.914 INFO:teuthology.orchestra.run.smithi061.stdout: "iostat", 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: "nfs", 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: "restful" 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: "available_modules": [ 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.915 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "alerts", 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:37.916 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:37.917 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.917 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.917 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:37.917 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.917 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How frequently to reexamine health status", 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.919 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.919 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.919 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.919 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.919 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.920 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.920 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.920 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.920 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.920 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.921 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.922 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.922 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.922 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.922 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.923 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.923 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.923 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.923 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.923 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.924 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.925 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.926 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.926 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.926 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.926 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.926 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.927 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.928 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.929 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.929 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.929 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.929 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.930 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.930 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.930 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.930 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.931 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_destination": { 2023-10-18T09:35:37.932 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_destination", 2023-10-18T09:35:37.932 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.932 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.932 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.932 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.933 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.933 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.933 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.933 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email address to send alerts to", 2023-10-18T09:35:37.933 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.934 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.934 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.934 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_from_name": { 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_from_name", 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "Ceph", 2023-10-18T09:35:37.935 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.936 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.936 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.936 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Email From: name", 2023-10-18T09:35:37.936 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.936 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_host": { 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_host", 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.937 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.938 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.938 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.938 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.938 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.938 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.939 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP server", 2023-10-18T09:35:37.939 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.939 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.939 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.940 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_password": { 2023-10-18T09:35:37.940 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_password", 2023-10-18T09:35:37.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.940 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.940 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Password to authenticate with", 2023-10-18T09:35:37.941 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.942 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.942 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.942 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_port": { 2023-10-18T09:35:37.943 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_port", 2023-10-18T09:35:37.943 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:37.943 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.943 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.943 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "465", 2023-10-18T09:35:37.944 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.944 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.944 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.944 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP port", 2023-10-18T09:35:37.944 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_sender": { 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_sender", 2023-10-18T09:35:37.945 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.946 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.946 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.946 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.946 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.946 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.947 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.947 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SMTP envelope sender", 2023-10-18T09:35:37.947 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.947 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.947 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.948 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_ssl": { 2023-10-18T09:35:37.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_ssl", 2023-10-18T09:35:37.948 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.948 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.949 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.949 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.949 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.949 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.949 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.950 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-18T09:35:37.950 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.950 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.950 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.951 INFO:teuthology.orchestra.run.smithi061.stdout: "smtp_user": { 2023-10-18T09:35:37.951 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "smtp_user", 2023-10-18T09:35:37.951 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.951 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "User to authenticate as", 2023-10-18T09:35:37.952 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.953 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.953 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:37.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:37.954 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "balancer", 2023-10-18T09:35:37.954 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:37.954 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:37.954 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:37.955 INFO:teuthology.orchestra.run.smithi061.stdout: "active": { 2023-10-18T09:35:37.955 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "active", 2023-10-18T09:35:37.955 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.955 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.955 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.956 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:37.956 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.956 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.956 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.957 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically balance PGs across cluster", 2023-10-18T09:35:37.957 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.957 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.957 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.958 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_time": { 2023-10-18T09:35:37.958 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_time", 2023-10-18T09:35:37.958 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.958 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.958 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.959 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0000", 2023-10-18T09:35:37.959 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.959 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.959 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.959 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "beginning time of day to automatically balance", 2023-10-18T09:35:37.960 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:37.960 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.960 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.960 INFO:teuthology.orchestra.run.smithi061.stdout: "begin_weekday": { 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "begin_weekday", 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:37.961 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:37.962 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:37.962 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.962 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-18T09:35:37.962 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:37.962 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.963 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.963 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_max_iterations": { 2023-10-18T09:35:37.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_max_iterations", 2023-10-18T09:35:37.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "25", 2023-10-18T09:35:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "250", 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.966 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_metrics": { 2023-10-18T09:35:37.966 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_metrics", 2023-10-18T09:35:37.966 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.966 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.967 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "pgs,objects,bytes", 2023-10-18T09:35:37.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.967 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.967 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.968 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-18T09:35:37.968 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-18T09:35:37.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.968 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.969 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_compat_step": { 2023-10-18T09:35:37.969 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crush_compat_step", 2023-10-18T09:35:37.969 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.969 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.970 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.970 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.5", 2023-10-18T09:35:37.970 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0.001", 2023-10-18T09:35:37.970 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "0.999", 2023-10-18T09:35:37.970 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.971 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "aggressiveness of optimization", 2023-10-18T09:35:37.971 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-18T09:35:37.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.971 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "end_time": { 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_time", 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.972 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2359", 2023-10-18T09:35:37.973 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.973 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.973 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.973 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ending time of day to automatically balance", 2023-10-18T09:35:37.973 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-18T09:35:37.974 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.974 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.974 INFO:teuthology.orchestra.run.smithi061.stdout: "end_weekday": { 2023-10-18T09:35:37.974 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "end_weekday", 2023-10-18T09:35:37.975 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:37.975 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.975 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.975 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "0", 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "6", 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-18T09:35:37.976 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.977 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.977 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:37.977 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:37.977 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.978 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.978 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.978 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.978 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.978 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.979 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.980 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.980 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.980 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.980 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:37.981 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.982 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.982 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.982 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.982 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.983 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.983 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.983 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.983 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.983 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.984 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.984 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:37.984 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:37.984 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.985 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.985 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.985 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:37.985 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.985 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.986 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.986 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:37.986 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:37.986 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:37.986 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.987 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.988 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.988 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:37.988 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:37.988 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:37.989 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.989 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.989 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:37.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.990 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.990 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:37.990 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.990 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.990 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.991 INFO:teuthology.orchestra.run.smithi061.stdout: "min_score": { 2023-10-18T09:35:37.991 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_score", 2023-10-18T09:35:37.991 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:37.992 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.992 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.992 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:37.992 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.992 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.994 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:37.994 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:37.994 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.994 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.994 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.995 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "upmap", 2023-10-18T09:35:37.995 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.995 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.995 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:37.995 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-compat", 2023-10-18T09:35:37.996 INFO:teuthology.orchestra.run.smithi061.stdout: "none", 2023-10-18T09:35:37.996 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap" 2023-10-18T09:35:37.996 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:37.996 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Balancer mode", 2023-10-18T09:35:37.996 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_ids": { 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_ids", 2023-10-18T09:35:37.997 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:37.998 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:37.998 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:37.998 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:37.998 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:37.998 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:37.999 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:37.999 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-18T09:35:37.999 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:37.999 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:37.999 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.000 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:38.000 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:38.000 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.000 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.002 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-18T09:35:38.002 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.002 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.002 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.003 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_deviation": { 2023-10-18T09:35:38.003 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_deviation", 2023-10-18T09:35:38.003 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.003 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.004 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-18T09:35:38.005 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-18T09:35:38.005 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.005 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.005 INFO:teuthology.orchestra.run.smithi061.stdout: "upmap_max_optimizations": { 2023-10-18T09:35:38.006 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "upmap_max_optimizations", 2023-10-18T09:35:38.006 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:38.006 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.006 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.007 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.007 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.007 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.007 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.008 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-18T09:35:38.008 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.008 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.008 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.009 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.009 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm", 2023-10-18T09:35:38.009 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.010 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.010 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.010 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_down_multiplier": { 2023-10-18T09:35:38.010 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_down_multiplier", 2023-10-18T09:35:38.010 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.011 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.012 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-18T09:35:38.012 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.012 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.012 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.013 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_refresh_rate": { 2023-10-18T09:35:38.013 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_refresh_rate", 2023-10-18T09:35:38.014 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.014 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.014 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.014 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "20", 2023-10-18T09:35:38.014 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.015 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.015 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.015 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-18T09:35:38.015 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.016 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.016 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.016 INFO:teuthology.orchestra.run.smithi061.stdout: "agent_starting_port": { 2023-10-18T09:35:38.016 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "agent_starting_port", 2023-10-18T09:35:38.017 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.017 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.017 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.017 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4721", 2023-10-18T09:35:38.017 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.018 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.018 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.018 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-18T09:35:38.018 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.018 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.019 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.019 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_ptrace": { 2023-10-18T09:35:38.019 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_ptrace", 2023-10-18T09:35:38.019 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.020 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.020 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.020 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.020 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.020 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.021 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.021 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-18T09:35:38.021 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-18T09:35:38.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.021 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.022 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_interval": { 2023-10-18T09:35:38.022 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_interval", 2023-10-18T09:35:38.022 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.023 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.023 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.023 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:38.023 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.024 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.024 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.024 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-18T09:35:38.024 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.024 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.025 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.025 INFO:teuthology.orchestra.run.smithi061.stdout: "autotune_memory_target_ratio": { 2023-10-18T09:35:38.025 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "autotune_memory_target_ratio", 2023-10-18T09:35:38.025 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.026 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.026 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0.7", 2023-10-18T09:35:38.026 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.026 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.027 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.027 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-18T09:35:38.027 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.027 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.027 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm_log_destination": { 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cephadm_log_destination", 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.028 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.029 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.029 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.029 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.029 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.029 INFO:teuthology.orchestra.run.smithi061.stdout: "file", 2023-10-18T09:35:38.030 INFO:teuthology.orchestra.run.smithi061.stdout: "file,syslog", 2023-10-18T09:35:38.030 INFO:teuthology.orchestra.run.smithi061.stdout: "syslog" 2023-10-18T09:35:38.030 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.030 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-18T09:35:38.030 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: "cgroups_split": { 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cgroups_split", 2023-10-18T09:35:38.031 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.032 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.032 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.032 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.032 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.032 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.033 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.033 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-18T09:35:38.033 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.033 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.033 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.034 INFO:teuthology.orchestra.run.smithi061.stdout: "config_checks_enabled": { 2023-10-18T09:35:38.034 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_checks_enabled", 2023-10-18T09:35:38.034 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-18T09:35:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "config_dashboard": { 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_dashboard", 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-18T09:35:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_alertmanager": { 2023-10-18T09:35:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_alertmanager", 2023-10-18T09:35:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.040 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_base": { 2023-10-18T09:35:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_base", 2023-10-18T09:35:38.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.043 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.043 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.043 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-18T09:35:38.043 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.043 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Container image name, without the tag", 2023-10-18T09:35:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.045 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_elasticsearch": { 2023-10-18T09:35:38.045 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_elasticsearch", 2023-10-18T09:35:38.045 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.045 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.046 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.046 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-18T09:35:38.046 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.046 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.046 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.047 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "elasticsearch container image", 2023-10-18T09:35:38.047 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.047 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.047 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.048 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_grafana": { 2023-10-18T09:35:38.048 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_grafana", 2023-10-18T09:35:38.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.048 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.049 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-18T09:35:38.049 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.049 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.049 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.049 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:38.050 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.050 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.050 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.050 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_haproxy": { 2023-10-18T09:35:38.051 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_haproxy", 2023-10-18T09:35:38.051 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.051 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.051 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.051 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-18T09:35:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "HAproxy container image", 2023-10-18T09:35:38.052 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.053 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.053 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.053 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_agent": { 2023-10-18T09:35:38.053 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_agent", 2023-10-18T09:35:38.054 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.054 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.054 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.054 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-18T09:35:38.054 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger agent container image", 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.055 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.056 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_collector": { 2023-10-18T09:35:38.056 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_collector", 2023-10-18T09:35:38.056 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.056 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.057 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.057 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-18T09:35:38.057 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.057 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.057 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.058 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger collector container image", 2023-10-18T09:35:38.058 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.058 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.058 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.059 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_jaeger_query": { 2023-10-18T09:35:38.059 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_jaeger_query", 2023-10-18T09:35:38.060 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.060 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.061 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-18T09:35:38.061 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.061 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.062 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.062 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Jaeger query container image", 2023-10-18T09:35:38.062 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.063 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.063 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.064 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_keepalived": { 2023-10-18T09:35:38.064 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_keepalived", 2023-10-18T09:35:38.064 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.064 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.064 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.065 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-18T09:35:38.065 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.065 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.065 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.065 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Keepalived container image", 2023-10-18T09:35:38.066 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.066 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.066 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.066 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_loki": { 2023-10-18T09:35:38.067 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_loki", 2023-10-18T09:35:38.067 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.067 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.067 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.068 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-18T09:35:38.068 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.068 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.068 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.068 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Loki container image", 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_node_exporter": { 2023-10-18T09:35:38.069 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_node_exporter", 2023-10-18T09:35:38.070 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.070 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.070 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.071 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_nvmeof": { 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_nvmeof", 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.072 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.073 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.073 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-18T09:35:38.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.073 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.073 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.074 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Nvme-of container image", 2023-10-18T09:35:38.074 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.074 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.075 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_prometheus": { 2023-10-18T09:35:38.075 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_prometheus", 2023-10-18T09:35:38.075 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.075 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.076 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-18T09:35:38.076 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.076 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.076 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.076 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Prometheus container image", 2023-10-18T09:35:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_promtail": { 2023-10-18T09:35:38.078 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_promtail", 2023-10-18T09:35:38.078 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.078 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.078 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.078 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Promtail container image", 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.079 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.080 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.080 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image_snmp_gateway": { 2023-10-18T09:35:38.080 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_image_snmp_gateway", 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.081 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.082 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.082 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "SNMP Gateway container image", 2023-10-18T09:35:38.082 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.082 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.082 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.083 INFO:teuthology.orchestra.run.smithi061.stdout: "container_init": { 2023-10-18T09:35:38.083 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "container_init", 2023-10-18T09:35:38.083 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.083 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.084 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Run podman/docker with `--init`", 2023-10-18T09:35:38.085 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.085 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.085 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.086 INFO:teuthology.orchestra.run.smithi061.stdout: "daemon_cache_timeout": { 2023-10-18T09:35:38.086 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "daemon_cache_timeout", 2023-10-18T09:35:38.086 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.087 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.087 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.087 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:38.087 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.087 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.088 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.088 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-18T09:35:38.088 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.088 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.089 INFO:teuthology.orchestra.run.smithi061.stdout: "default_cephadm_command_timeout": { 2023-10-18T09:35:38.089 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_cephadm_command_timeout", 2023-10-18T09:35:38.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.090 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.090 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.090 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "900", 2023-10-18T09:35:38.091 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.091 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.091 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.092 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-18T09:35:38.092 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.092 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.093 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.093 INFO:teuthology.orchestra.run.smithi061.stdout: "default_registry": { 2023-10-18T09:35:38.093 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "default_registry", 2023-10-18T09:35:38.094 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.094 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.094 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.095 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "docker.io", 2023-10-18T09:35:38.095 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.095 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.096 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.096 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-18T09:35:38.096 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.097 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.097 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.097 INFO:teuthology.orchestra.run.smithi061.stdout: "device_cache_timeout": { 2023-10-18T09:35:38.098 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_cache_timeout", 2023-10-18T09:35:38.098 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.098 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.099 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.099 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1800", 2023-10-18T09:35:38.099 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.099 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.100 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.100 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache device inventory", 2023-10-18T09:35:38.100 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.100 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: "device_enhanced_scan": { 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_enhanced_scan", 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.102 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.102 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.102 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.102 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.102 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.103 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-18T09:35:38.103 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.103 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.103 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "facts_cache_timeout": { 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "facts_cache_timeout", 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.104 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:38.105 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.105 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.105 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.105 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "seconds to cache host facts data", 2023-10-18T09:35:38.105 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.106 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.106 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.106 INFO:teuthology.orchestra.run.smithi061.stdout: "host_check_interval": { 2023-10-18T09:35:38.107 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "host_check_interval", 2023-10-18T09:35:38.107 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.107 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.107 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.107 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to perform a host check", 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.108 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.109 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.109 INFO:teuthology.orchestra.run.smithi061.stdout: "inventory_list_all": { 2023-10-18T09:35:38.109 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "inventory_list_all", 2023-10-18T09:35:38.109 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.110 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.110 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.110 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.110 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.110 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.111 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.111 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-18T09:35:38.111 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.111 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.112 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.112 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.112 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.113 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.113 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.113 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.113 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.113 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.114 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.114 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.114 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.114 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.115 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.116 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.116 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.116 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.116 INFO:teuthology.orchestra.run.smithi061.stdout: "log_refresh_metadata": { 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_refresh_metadata", 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.117 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.118 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.118 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.118 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-18T09:35:38.118 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.118 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.119 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.119 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.119 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.119 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.120 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.120 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.120 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.120 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.120 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.121 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.121 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-18T09:35:38.121 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.121 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.122 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.122 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.122 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.122 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.123 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.123 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.123 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.123 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.123 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.124 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.125 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.126 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.126 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.126 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.126 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.127 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.127 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.127 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.127 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.127 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.128 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.128 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.128 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.128 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.129 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.130 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.130 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.130 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.130 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-18T09:35:38.130 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.131 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.131 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.131 INFO:teuthology.orchestra.run.smithi061.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-18T09:35:38.132 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-18T09:35:38.132 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.132 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.132 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.132 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "*", 2023-10-18T09:35:38.133 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.133 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.133 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.133 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-18T09:35:38.133 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: "max_count_per_host": { 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_count_per_host", 2023-10-18T09:35:38.134 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.135 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.135 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.135 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.136 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.136 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.136 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.137 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of daemons per service per host", 2023-10-18T09:35:38.137 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.137 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.138 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.138 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd_draining_count": { 2023-10-18T09:35:38.139 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_osd_draining_count", 2023-10-18T09:35:38.139 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.139 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.140 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.140 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.140 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.141 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.141 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.141 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-18T09:35:38.142 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.142 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.142 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.143 INFO:teuthology.orchestra.run.smithi061.stdout: "migration_current": { 2023-10-18T09:35:38.144 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "migration_current", 2023-10-18T09:35:38.144 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.144 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.145 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.145 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.145 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.145 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.146 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.146 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "internal - do not modify", 2023-10-18T09:35:38.146 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.146 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.146 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.147 INFO:teuthology.orchestra.run.smithi061.stdout: "mode": { 2023-10-18T09:35:38.147 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mode", 2023-10-18T09:35:38.147 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.147 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "root", 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.148 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm-package", 2023-10-18T09:35:38.149 INFO:teuthology.orchestra.run.smithi061.stdout: "root" 2023-10-18T09:35:38.149 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.149 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "mode for remote execution of cephadm", 2023-10-18T09:35:38.149 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.149 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.150 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.150 INFO:teuthology.orchestra.run.smithi061.stdout: "prometheus_alerts_path": { 2023-10-18T09:35:38.150 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus_alerts_path", 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.151 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.152 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.152 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-18T09:35:38.152 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.152 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.152 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.153 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_insecure": { 2023-10-18T09:35:38.153 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_insecure", 2023-10-18T09:35:38.153 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.153 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.154 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.154 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.154 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.154 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.154 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.155 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-18T09:35:38.155 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.155 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.155 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.156 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_password": { 2023-10-18T09:35:38.156 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_password", 2023-10-18T09:35:38.156 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.156 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.156 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-18T09:35:38.157 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.158 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.158 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.158 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_url": { 2023-10-18T09:35:38.158 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_url", 2023-10-18T09:35:38.159 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.159 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.159 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.159 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.160 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.160 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.160 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.160 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-18T09:35:38.160 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: "registry_username": { 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "registry_username", 2023-10-18T09:35:38.161 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.162 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.162 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.162 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.162 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.162 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.163 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.163 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-18T09:35:38.163 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.163 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.163 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.164 INFO:teuthology.orchestra.run.smithi061.stdout: "secure_monitoring_stack": { 2023-10-18T09:35:38.164 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "secure_monitoring_stack", 2023-10-18T09:35:38.164 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.164 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.165 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.165 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.165 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.165 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.165 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.166 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-18T09:35:38.166 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.166 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.166 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.167 INFO:teuthology.orchestra.run.smithi061.stdout: "service_discovery_port": { 2023-10-18T09:35:38.167 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "service_discovery_port", 2023-10-18T09:35:38.167 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.167 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.167 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.168 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8765", 2023-10-18T09:35:38.168 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.168 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.168 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "cephadm service discovery port", 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.169 INFO:teuthology.orchestra.run.smithi061.stdout: "ssh_config_file": { 2023-10-18T09:35:38.170 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssh_config_file", 2023-10-18T09:35:38.170 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.170 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.170 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.170 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.171 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.171 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.171 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.171 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-18T09:35:38.171 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: "use_agent": { 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_agent", 2023-10-18T09:35:38.172 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.173 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.173 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.173 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.173 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.173 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.174 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.174 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-18T09:35:38.174 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.174 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.174 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: "use_repo_digest": { 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "use_repo_digest", 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.175 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.176 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.176 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.176 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.176 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_failed_host_check": { 2023-10-18T09:35:38.178 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_failed_host_check", 2023-10-18T09:35:38.178 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.178 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.178 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.178 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if the host check fails", 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.179 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.180 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.180 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_daemons": { 2023-10-18T09:35:38.180 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_daemons", 2023-10-18T09:35:38.180 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.181 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.181 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.181 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.181 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.181 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-18T09:35:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_on_stray_hosts": { 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_on_stray_hosts", 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.184 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.184 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.184 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.184 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.184 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-18T09:35:38.185 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.185 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.185 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.186 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crash", 2023-10-18T09:35:38.186 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.186 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.187 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.188 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.188 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.188 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.188 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.188 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.191 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.191 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.192 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.192 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.192 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.193 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.193 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.193 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.194 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.194 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.195 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.195 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.195 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.196 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.196 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.196 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.197 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.197 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.197 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.198 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.198 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.198 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.199 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.200 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.200 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.200 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.200 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.200 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.201 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.202 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.202 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.202 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.202 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.202 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.203 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.203 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.203 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.203 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.203 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.204 INFO:teuthology.orchestra.run.smithi061.stdout: "retain_interval": { 2023-10-18T09:35:38.204 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retain_interval", 2023-10-18T09:35:38.204 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.204 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.205 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.205 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "31536000", 2023-10-18T09:35:38.205 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.206 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.206 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.206 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-18T09:35:38.207 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.207 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.207 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.208 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_recent_interval": { 2023-10-18T09:35:38.208 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_recent_interval", 2023-10-18T09:35:38.209 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.209 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.209 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.210 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1209600", 2023-10-18T09:35:38.210 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.210 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.211 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.211 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-18T09:35:38.211 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.212 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.212 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.212 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.213 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.214 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dashboard", 2023-10-18T09:35:38.214 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.215 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.215 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.215 INFO:teuthology.orchestra.run.smithi061.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-18T09:35:38.216 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-18T09:35:38.216 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.216 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.216 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.216 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.218 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.218 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.218 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_HOST": { 2023-10-18T09:35:38.218 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-18T09:35:38.219 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.219 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.219 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.219 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.219 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.220 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.221 INFO:teuthology.orchestra.run.smithi061.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-18T09:35:38.221 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-18T09:35:38.221 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.221 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.222 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.222 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.222 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.222 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.222 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.223 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.223 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.223 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.223 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_ENABLED": { 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_ENABLED", 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.224 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.225 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.226 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.226 INFO:teuthology.orchestra.run.smithi061.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-18T09:35:38.226 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-18T09:35:38.226 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.227 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.227 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.227 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.227 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.227 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.228 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.228 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.228 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.228 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.228 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.229 INFO:teuthology.orchestra.run.smithi061.stdout: "ENABLE_BROWSABLE_API": { 2023-10-18T09:35:38.229 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-18T09:35:38.229 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.229 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.230 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.230 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.230 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.230 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.230 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.231 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-18T09:35:38.232 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-18T09:35:38.232 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.232 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.232 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.232 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.233 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.233 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.233 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.233 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.234 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.234 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.234 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.235 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-18T09:35:38.235 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-18T09:35:38.235 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.235 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.235 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.236 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.236 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.236 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.236 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.237 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.237 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.237 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.237 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.238 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-18T09:35:38.238 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-18T09:35:38.238 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.238 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.238 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.241 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.242 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.242 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.242 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.242 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.242 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.243 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.243 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-18T09:35:38.243 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-18T09:35:38.243 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.244 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.245 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.245 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.245 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.245 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.246 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-18T09:35:38.246 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-18T09:35:38.246 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.246 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.246 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.247 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.248 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.248 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.248 INFO:teuthology.orchestra.run.smithi061.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-18T09:35:38.248 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-18T09:35:38.249 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.249 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.249 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.249 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.249 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.250 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.250 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.250 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.251 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.251 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.251 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.252 INFO:teuthology.orchestra.run.smithi061.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-18T09:35:38.253 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-18T09:35:38.253 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.253 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.254 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.254 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.254 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.255 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.255 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.255 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.256 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.256 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.256 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.257 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_PASSWORD": { 2023-10-18T09:35:38.257 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-18T09:35:38.258 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.258 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.258 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.259 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:38.259 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.259 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.259 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.260 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-18T09:35:38.261 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-18T09:35:38.261 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.261 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.261 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.261 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.262 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.262 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.262 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.262 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.262 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.263 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.263 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.263 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_URL": { 2023-10-18T09:35:38.263 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_URL", 2023-10-18T09:35:38.264 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.264 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.264 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.264 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.264 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.265 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.266 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_API_USERNAME": { 2023-10-18T09:35:38.266 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-18T09:35:38.266 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.266 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.267 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.267 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:38.267 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.267 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.267 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.268 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.268 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.268 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.269 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.269 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-18T09:35:38.269 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-18T09:35:38.269 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.270 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.270 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.270 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.270 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.270 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.271 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.271 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.271 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.271 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.271 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.272 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.273 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.273 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.273 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.273 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.273 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.274 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.274 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.274 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.274 INFO:teuthology.orchestra.run.smithi061.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-18T09:35:38.275 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-18T09:35:38.275 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.275 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.275 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.275 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.276 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.277 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.277 INFO:teuthology.orchestra.run.smithi061.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-18T09:35:38.277 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-18T09:35:38.278 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.278 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.278 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.278 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.278 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.279 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.280 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_HOST": { 2023-10-18T09:35:38.280 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-18T09:35:38.280 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.280 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.281 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.282 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.282 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.282 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.283 INFO:teuthology.orchestra.run.smithi061.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-18T09:35:38.283 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-18T09:35:38.283 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.283 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.283 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.284 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.284 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.284 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.284 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.285 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.285 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.285 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.285 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.286 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-18T09:35:38.286 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-18T09:35:38.286 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.287 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.287 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.287 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.287 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.288 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.288 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.288 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.288 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.289 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.289 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.289 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-18T09:35:38.290 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-18T09:35:38.290 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.290 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.290 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.291 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.291 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.291 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.291 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.292 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.292 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.292 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.292 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.293 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-18T09:35:38.293 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-18T09:35:38.293 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.294 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.294 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.294 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.294 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.295 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.295 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.295 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.295 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.295 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.296 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.296 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-18T09:35:38.296 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-18T09:35:38.296 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.297 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.297 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.297 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.297 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.298 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.298 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.298 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.298 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-18T09:35:38.299 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.300 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.300 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.300 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.300 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.300 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.301 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.301 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.301 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.301 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.301 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.302 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-18T09:35:38.302 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-18T09:35:38.302 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.302 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.303 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.304 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.304 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.304 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.304 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-18T09:35:38.305 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-18T09:35:38.305 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.305 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.305 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.306 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.306 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.306 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.306 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.306 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_ENABLED": { 2023-10-18T09:35:38.307 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-18T09:35:38.308 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.308 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.308 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.308 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.308 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.309 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.309 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.309 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.309 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.309 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.310 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.310 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-18T09:35:38.310 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-18T09:35:38.310 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.311 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.312 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.312 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.312 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.312 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.313 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.314 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.314 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.314 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.314 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.314 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.315 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.315 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.315 INFO:teuthology.orchestra.run.smithi061.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8", 2023-10-18T09:35:38.316 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.317 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.317 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.317 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.317 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-18T09:35:38.318 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.319 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.319 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.319 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "45", 2023-10-18T09:35:38.319 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.319 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.321 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ACCESS_KEY": { 2023-10-18T09:35:38.321 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-18T09:35:38.321 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.321 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.321 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.322 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.322 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.322 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.322 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.322 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-18T09:35:38.323 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-18T09:35:38.324 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.324 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.324 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.324 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "admin", 2023-10-18T09:35:38.324 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.325 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.326 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SECRET_KEY": { 2023-10-18T09:35:38.326 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-18T09:35:38.326 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.326 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.327 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.327 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.327 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.327 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.327 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.328 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.328 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.328 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.328 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.329 INFO:teuthology.orchestra.run.smithi061.stdout: "RGW_API_SSL_VERIFY": { 2023-10-18T09:35:38.329 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-18T09:35:38.329 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.329 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.329 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.330 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.330 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.330 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.330 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.330 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: "UNSAFE_TLS_v1_2": { 2023-10-18T09:35:38.331 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-18T09:35:38.332 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.332 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.332 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.332 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.332 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.334 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-18T09:35:38.334 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-18T09:35:38.334 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.334 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.335 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.335 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:38.335 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.335 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.335 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.336 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.336 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.336 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.336 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.337 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-18T09:35:38.337 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-18T09:35:38.337 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.337 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.337 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-18T09:35:38.339 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-18T09:35:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.341 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.342 INFO:teuthology.orchestra.run.smithi061.stdout: "cross_origin_url": { 2023-10-18T09:35:38.342 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cross_origin_url", 2023-10-18T09:35:38.342 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.342 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.343 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.344 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.344 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.344 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.344 INFO:teuthology.orchestra.run.smithi061.stdout: "crt_file": { 2023-10-18T09:35:38.345 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "crt_file", 2023-10-18T09:35:38.345 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.345 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.345 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.345 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.346 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.347 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.347 INFO:teuthology.orchestra.run.smithi061.stdout: "debug": { 2023-10-18T09:35:38.347 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "debug", 2023-10-18T09:35:38.347 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.348 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.349 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Enable/disable debug options", 2023-10-18T09:35:38.349 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.349 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.349 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.350 INFO:teuthology.orchestra.run.smithi061.stdout: "jwt_token_ttl": { 2023-10-18T09:35:38.350 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "jwt_token_ttl", 2023-10-18T09:35:38.350 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.350 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "28800", 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.351 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.352 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.352 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.352 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.352 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.353 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.354 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.354 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.354 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.354 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.354 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.355 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.356 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.356 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.356 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.356 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.356 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.357 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.357 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.357 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.357 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.357 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.358 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.358 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.358 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.358 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.358 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.359 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.359 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.360 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.360 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.360 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.360 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.361 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.361 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.361 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.361 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.361 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.362 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.362 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.362 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.362 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.363 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.363 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.363 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.363 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.363 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.364 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.365 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.365 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.365 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.365 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.365 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.366 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.366 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.366 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.366 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.367 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.367 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.367 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.367 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.367 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.368 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.368 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.368 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.368 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.368 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.369 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.369 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.369 INFO:teuthology.orchestra.run.smithi061.stdout: "motd": { 2023-10-18T09:35:38.369 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "motd", 2023-10-18T09:35:38.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.370 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.370 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.370 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.370 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.371 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.371 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.371 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "The message of the day", 2023-10-18T09:35:38.371 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.371 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.372 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.372 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect_resolve_ip_addr": { 2023-10-18T09:35:38.372 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "redirect_resolve_ip_addr", 2023-10-18T09:35:38.372 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.373 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.374 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.374 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.374 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.374 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.375 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:38.375 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:38.375 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.375 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.375 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.376 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.377 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.377 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.377 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:38.377 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:38.378 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.378 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.378 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.378 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8080", 2023-10-18T09:35:38.378 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.379 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.379 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.379 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.379 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.379 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.380 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.380 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:38.380 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:38.380 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.381 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.382 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.382 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.382 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.382 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.383 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl_server_port": { 2023-10-18T09:35:38.383 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl_server_port", 2023-10-18T09:35:38.383 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.383 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.384 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.384 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8443", 2023-10-18T09:35:38.384 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.384 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.384 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.385 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:38.386 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:38.386 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.386 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.386 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.386 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "redirect", 2023-10-18T09:35:38.387 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.387 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.387 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.387 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.387 INFO:teuthology.orchestra.run.smithi061.stdout: "redirect" 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.389 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:38.389 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:38.389 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.389 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.389 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.390 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:38.390 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:38.390 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:38.390 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.390 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: "url_prefix": { 2023-10-18T09:35:38.391 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url_prefix", 2023-10-18T09:35:38.392 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.392 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.392 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.392 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.392 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.393 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.393 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.393 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.393 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.393 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.394 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.394 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.394 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "devicehealth", 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.395 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_monitoring": { 2023-10-18T09:35:38.396 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_monitoring", 2023-10-18T09:35:38.396 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.396 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.396 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.396 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "monitor device health metrics", 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.397 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.398 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.398 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.398 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.398 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.399 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.400 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.400 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.400 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.400 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.400 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.401 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.402 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.402 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.402 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.402 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.403 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.403 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.403 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.403 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.403 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.404 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.404 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.404 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.404 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.405 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.405 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.405 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.405 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.406 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.406 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.406 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.406 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.406 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.407 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.409 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.409 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.409 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.409 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.409 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.410 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.410 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.410 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.410 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.410 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: "mark_out_threshold": { 2023-10-18T09:35:38.411 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mark_out_threshold", 2023-10-18T09:35:38.412 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.412 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.412 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.412 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "2419200", 2023-10-18T09:35:38.412 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.413 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.414 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": { 2023-10-18T09:35:38.414 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pool_name", 2023-10-18T09:35:38.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.414 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.415 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.415 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "device_health_metrics", 2023-10-18T09:35:38.415 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.415 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.416 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.416 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-18T09:35:38.416 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: "retention_period": { 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "retention_period", 2023-10-18T09:35:38.417 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.418 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.418 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.418 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15552000", 2023-10-18T09:35:38.418 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.418 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.419 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.419 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long to retain device health metrics", 2023-10-18T09:35:38.419 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.419 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.419 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.420 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_frequency": { 2023-10-18T09:35:38.420 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_frequency", 2023-10-18T09:35:38.420 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.420 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.421 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-18T09:35:38.422 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.422 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.422 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.422 INFO:teuthology.orchestra.run.smithi061.stdout: "self_heal": { 2023-10-18T09:35:38.423 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "self_heal", 2023-10-18T09:35:38.423 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.423 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.423 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.423 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.424 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.425 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.425 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:38.425 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:38.425 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.426 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.426 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.426 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:38.426 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.426 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.427 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.427 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how frequently to wake up and check device health", 2023-10-18T09:35:38.427 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.427 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: "warn_threshold": { 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "warn_threshold", 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.428 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.429 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7257600", 2023-10-18T09:35:38.429 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.429 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.429 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.429 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.431 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "diskprediction_local", 2023-10-18T09:35:38.431 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.431 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.431 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.432 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.432 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.432 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.432 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.432 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.433 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.433 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.433 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.433 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.433 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.434 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.434 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.434 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.434 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.434 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.436 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.436 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.436 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.436 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.437 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.437 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.437 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.437 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.437 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.438 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.438 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.438 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.438 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.439 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.440 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.440 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.440 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.440 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.440 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.441 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.442 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.442 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.442 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.443 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.443 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.443 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.443 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.444 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.444 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.445 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.445 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.445 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.446 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.446 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.446 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.447 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.447 INFO:teuthology.orchestra.run.smithi061.stdout: "predict_interval": { 2023-10-18T09:35:38.448 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predict_interval", 2023-10-18T09:35:38.448 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.448 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.449 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.449 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "86400", 2023-10-18T09:35:38.449 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.449 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.450 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.450 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.450 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.451 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.451 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.452 INFO:teuthology.orchestra.run.smithi061.stdout: "predictor_model": { 2023-10-18T09:35:38.452 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "predictor_model", 2023-10-18T09:35:38.452 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.452 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.452 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "prophetstor", 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.453 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.454 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.454 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:38.454 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:38.455 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.455 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.455 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.455 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "600", 2023-10-18T09:35:38.455 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.456 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.457 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.457 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.457 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "influx", 2023-10-18T09:35:38.458 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": false, 2023-10-18T09:35:38.458 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "influxdb python module not found", 2023-10-18T09:35:38.458 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.458 INFO:teuthology.orchestra.run.smithi061.stdout: "batch_size": { 2023-10-18T09:35:38.458 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "batch_size", 2023-10-18T09:35:38.459 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.459 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.459 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.459 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5000", 2023-10-18T09:35:38.459 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.460 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.460 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.460 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-18T09:35:38.460 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.460 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.461 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.461 INFO:teuthology.orchestra.run.smithi061.stdout: "database": { 2023-10-18T09:35:38.461 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "database", 2023-10-18T09:35:38.461 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.462 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.462 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.462 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "ceph", 2023-10-18T09:35:38.462 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.462 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.463 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.463 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-18T09:35:38.463 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.463 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.463 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: "hostname": { 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "hostname", 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.464 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.465 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.465 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.465 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.465 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.465 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server hostname", 2023-10-18T09:35:38.466 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.466 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.466 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.466 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:38.467 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:38.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.467 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.467 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.467 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "30", 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "5", 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.468 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.469 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.469 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.469 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.470 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.470 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.470 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.470 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.470 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.471 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.472 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.472 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.472 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.472 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.472 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.473 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.473 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.473 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.473 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.474 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.474 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.474 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.474 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.474 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.475 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.476 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.476 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.476 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.476 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.477 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.478 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.478 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.478 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.478 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.478 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.480 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.480 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.480 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.480 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.480 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.481 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.481 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.481 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.481 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.481 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.482 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.482 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.482 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.482 INFO:teuthology.orchestra.run.smithi061.stdout: "password": { 2023-10-18T09:35:38.483 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "password", 2023-10-18T09:35:38.483 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.483 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.484 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.484 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.484 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.484 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "password of InfluxDB server user", 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: "port": { 2023-10-18T09:35:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "port", 2023-10-18T09:35:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.487 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "8086", 2023-10-18T09:35:38.487 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.487 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.487 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.487 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "InfluxDB server port", 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: "ssl": { 2023-10-18T09:35:38.488 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ssl", 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "false", 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.489 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.490 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.490 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:38.490 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.490 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.490 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.491 INFO:teuthology.orchestra.run.smithi061.stdout: "threads": { 2023-10-18T09:35:38.491 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threads", 2023-10-18T09:35:38.491 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.491 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.492 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.492 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:38.492 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:38.492 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "32", 2023-10-18T09:35:38.492 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.493 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-18T09:35:38.493 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.493 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.493 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.494 INFO:teuthology.orchestra.run.smithi061.stdout: "username": { 2023-10-18T09:35:38.494 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "username", 2023-10-18T09:35:38.494 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.494 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.494 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.495 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.495 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.495 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.495 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.495 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "username of InfluxDB server user", 2023-10-18T09:35:38.496 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.496 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.496 INFO:teuthology.orchestra.run.smithi061.stdout: "verify_ssl": { 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "verify_ssl", 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "true", 2023-10-18T09:35:38.497 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.498 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.498 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.498 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-18T09:35:38.498 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.500 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "insights", 2023-10-18T09:35:38.500 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.500 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.501 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.501 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.501 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.502 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.502 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.502 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.502 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.502 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.503 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.503 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.503 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.503 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.503 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.504 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.504 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.504 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.504 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.505 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.505 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.505 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.506 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.506 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.506 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.506 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.506 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.507 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.507 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.507 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.507 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.507 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.508 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.509 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.509 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.509 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.509 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.510 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.510 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.510 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.510 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.510 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.511 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.511 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.511 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.511 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.511 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.512 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.512 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.512 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.512 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.514 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.514 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.514 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.514 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.514 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.515 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.515 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.516 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "iostat", 2023-10-18T09:35:38.516 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.516 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.517 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.517 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.517 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.517 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.517 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.518 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.518 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.518 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.518 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.518 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.519 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.520 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.520 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.520 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.520 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.520 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.521 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.522 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.522 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.522 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.522 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.523 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.523 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.523 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.523 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.529 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.530 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.530 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.530 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.531 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "k8sevents", 2023-10-18T09:35:38.531 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.531 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_event_retention_days": { 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "ceph_event_retention_days", 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.532 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.533 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "7", 2023-10-18T09:35:38.533 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.533 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.533 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.533 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-18T09:35:38.534 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.534 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.534 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.534 INFO:teuthology.orchestra.run.smithi061.stdout: "config_check_secs": { 2023-10-18T09:35:38.535 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "config_check_secs", 2023-10-18T09:35:38.535 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.535 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.535 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.535 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.536 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "10", 2023-10-18T09:35:38.536 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.536 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.536 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-18T09:35:38.536 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.537 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.537 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.537 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.537 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.538 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.538 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.538 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.538 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.538 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.540 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.540 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.540 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.540 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.540 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.542 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.542 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.542 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.542 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.543 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.543 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.543 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.543 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.543 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.544 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.545 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.545 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.545 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.545 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.545 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.546 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.546 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.546 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.546 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.546 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.547 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.547 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.547 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.547 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.547 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.548 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.548 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.548 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.548 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.549 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.550 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.550 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.550 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.550 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.551 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.551 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "localpool", 2023-10-18T09:35:38.551 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "failure_domain": { 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "failure_domain", 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.552 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.553 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.553 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "host", 2023-10-18T09:35:38.553 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.553 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.553 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.554 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "failure domain for any created local pool", 2023-10-18T09:35:38.554 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-18T09:35:38.554 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.554 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.555 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.555 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.555 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.555 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.555 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.556 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.556 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.556 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.556 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.556 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.557 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.557 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.557 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.557 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.557 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.558 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.558 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.558 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.559 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.559 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.559 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.559 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.560 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.560 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.560 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.560 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.560 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.561 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.561 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.561 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.561 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.561 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.562 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.562 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.562 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.562 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.563 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.563 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.563 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.563 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.563 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.564 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.564 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.564 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.564 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.564 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.566 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.566 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.567 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.568 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.568 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.568 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.569 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.569 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.569 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.569 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.570 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.570 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.571 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.571 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": { 2023-10-18T09:35:38.572 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "min_size", 2023-10-18T09:35:38.572 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.572 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.573 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.573 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.573 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.574 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.574 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.574 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default min_size for any created local pool", 2023-10-18T09:35:38.575 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-18T09:35:38.575 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.575 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.576 INFO:teuthology.orchestra.run.smithi061.stdout: "num_rep": { 2023-10-18T09:35:38.576 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "num_rep", 2023-10-18T09:35:38.576 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.576 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.577 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.577 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3", 2023-10-18T09:35:38.577 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.577 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.578 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.578 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default replica count for any created local pool", 2023-10-18T09:35:38.578 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.578 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.578 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.579 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": { 2023-10-18T09:35:38.579 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_num", 2023-10-18T09:35:38.579 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.579 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.580 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.580 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "128", 2023-10-18T09:35:38.580 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.580 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.580 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "default pg_num for any created local pool", 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.581 INFO:teuthology.orchestra.run.smithi061.stdout: "prefix": { 2023-10-18T09:35:38.582 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prefix", 2023-10-18T09:35:38.582 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.582 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.582 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.582 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.583 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.583 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.583 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.583 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "name prefix for any created local pool", 2023-10-18T09:35:38.583 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.584 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.584 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.584 INFO:teuthology.orchestra.run.smithi061.stdout: "subtree": { 2023-10-18T09:35:38.584 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "subtree", 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "rack", 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.585 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.586 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.586 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-18T09:35:38.586 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-18T09:35:38.586 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.586 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.587 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mds_autoscaler", 2023-10-18T09:35:38.588 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.588 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.588 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.588 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.588 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.589 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.589 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.589 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.589 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.589 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.590 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.590 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.590 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.590 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.590 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.591 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.592 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.592 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.592 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.592 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.593 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.594 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.594 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.594 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.594 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.594 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.595 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.596 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.596 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.596 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.596 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.596 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.597 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.598 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.598 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.598 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.598 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.599 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.599 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.599 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.600 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.600 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.600 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.601 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.601 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.601 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.602 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.602 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.602 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.603 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.603 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.603 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.604 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirroring", 2023-10-18T09:35:38.605 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.605 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.605 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.606 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.606 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.606 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.607 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.607 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.607 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.608 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.608 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.608 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.608 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.608 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.609 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.610 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.610 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.610 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.610 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.611 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.611 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.611 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.611 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.611 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.613 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.613 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.613 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.614 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.614 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.614 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.614 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.614 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.615 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.616 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.616 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.616 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.616 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.616 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.617 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.618 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.618 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.618 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.618 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.618 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.620 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "nfs", 2023-10-18T09:35:38.620 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.620 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.621 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.622 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.622 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.622 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.622 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.622 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.623 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.624 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.624 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.624 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.624 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.625 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.626 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.626 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.626 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.626 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.626 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.627 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.628 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.628 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.628 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.628 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.628 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.629 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.630 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.630 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.630 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.630 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.630 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.631 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.632 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.632 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.632 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.632 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.632 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.633 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.634 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:38.634 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.634 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "fail_fs": { 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "fail_fs", 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.636 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.636 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.636 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.636 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.636 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.637 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.638 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.638 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.638 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.638 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.638 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.639 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.640 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.640 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.640 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.640 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.640 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.641 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.642 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.642 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.642 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.642 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.642 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.643 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.643 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.643 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.643 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.643 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.644 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.645 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.645 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.645 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.645 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.645 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.646 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.647 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.647 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.647 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.647 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.648 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.649 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.649 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.649 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.649 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.649 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.650 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.650 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.650 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator": { 2023-10-18T09:35:38.650 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "orchestrator", 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.651 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.652 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.652 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm", 2023-10-18T09:35:38.652 INFO:teuthology.orchestra.run.smithi061.stdout: "rook", 2023-10-18T09:35:38.652 INFO:teuthology.orchestra.run.smithi061.stdout: "test_orchestrator" 2023-10-18T09:35:38.652 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Orchestrator backend", 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.654 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_perf_query", 2023-10-18T09:35:38.654 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.654 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.655 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.656 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.656 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.656 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.656 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.656 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.657 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.657 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.657 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.657 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.657 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.658 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.658 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.658 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.658 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.658 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.659 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.659 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.659 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.659 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.660 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.661 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.662 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.662 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.662 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.662 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.662 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.663 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.664 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.664 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.664 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.664 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.664 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.665 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.666 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.666 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.666 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.666 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.666 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.667 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.668 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.668 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "osd_support", 2023-10-18T09:35:38.668 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.669 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.670 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.670 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.670 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.670 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.671 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.672 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.672 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.672 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.672 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.672 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.674 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.674 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.674 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.674 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.674 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.675 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.675 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.675 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.675 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.676 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.677 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.677 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.677 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.677 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.677 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.678 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.678 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.678 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.678 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.678 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.679 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.680 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.680 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.680 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.680 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.680 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.681 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.682 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "pg_autoscaler", 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.683 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.684 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.684 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.684 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.684 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.684 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.685 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.686 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.686 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.686 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.686 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.686 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.687 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.689 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.689 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.689 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.689 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.690 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.691 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.691 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.691 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.691 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.691 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.692 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.693 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.694 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.694 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.694 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.695 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.696 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.697 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:38.697 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.697 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.697 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.697 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.698 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.698 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.698 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.698 INFO:teuthology.orchestra.run.smithi061.stdout: "threshold": { 2023-10-18T09:35:38.699 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "threshold", 2023-10-18T09:35:38.699 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.699 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.699 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.699 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "3.0", 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1.0", 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "scaling threshold", 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-18T09:35:38.700 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.701 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.701 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.701 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.702 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "progress", 2023-10-18T09:35:38.702 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.702 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.702 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.702 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_pg_recovery_event": { 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_pg_recovery_event", 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.703 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.704 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.705 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:38.705 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:38.705 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.705 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.706 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.706 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.706 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.706 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.706 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.707 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.708 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.709 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.709 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.709 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.709 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.709 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.710 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.710 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.710 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.710 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.710 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.711 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.712 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.712 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.712 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.712 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.712 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.713 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.715 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.715 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.715 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.715 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.715 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.716 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.717 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.718 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.718 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.718 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.718 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.718 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.719 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.719 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.719 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.719 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.719 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.720 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.720 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.720 INFO:teuthology.orchestra.run.smithi061.stdout: "max_completed_events": { 2023-10-18T09:35:38.720 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_completed_events", 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "50", 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.721 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.722 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.722 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "number of past completed events to remember", 2023-10-18T09:35:38.722 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.722 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.722 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: "sleep_interval": { 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "sleep_interval", 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.723 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "5", 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "how long the module is going to sleep", 2023-10-18T09:35:38.724 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.726 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "prometheus", 2023-10-18T09:35:38.726 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.726 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.726 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.726 INFO:teuthology.orchestra.run.smithi061.stdout: "cache": { 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "cache", 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.727 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.728 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.728 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.728 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.728 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.728 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: "exclude_perf_counters": { 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "exclude_perf_counters", 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.729 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.730 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.730 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.730 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.730 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.730 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.731 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-18T09:35:38.731 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-18T09:35:38.731 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.731 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.732 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.733 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.734 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.734 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.734 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.734 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.734 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.735 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.735 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.735 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.735 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.736 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.737 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.737 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.737 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.737 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.737 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.738 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.739 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.739 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.739 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.739 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.739 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.740 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.741 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.741 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.741 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.741 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.741 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.742 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.742 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.742 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.742 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.743 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.743 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.743 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.743 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.743 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.744 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools": { 2023-10-18T09:35:38.745 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools", 2023-10-18T09:35:38.745 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.745 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.745 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.745 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.746 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.746 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.746 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.746 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.746 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-18T09:35:38.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300", 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.748 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.749 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.749 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.749 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.749 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.750 INFO:teuthology.orchestra.run.smithi061.stdout: "scrape_interval": { 2023-10-18T09:35:38.750 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "scrape_interval", 2023-10-18T09:35:38.750 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.750 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.750 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15.0", 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.751 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.752 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.752 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.752 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:38.752 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:38.753 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.753 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.753 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.753 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "::", 2023-10-18T09:35:38.753 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.754 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.755 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:38.755 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:38.755 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.755 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "9283", 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.756 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-18T09:35:38.757 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.757 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.757 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.757 INFO:teuthology.orchestra.run.smithi061.stdout: "stale_cache_strategy": { 2023-10-18T09:35:38.758 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stale_cache_strategy", 2023-10-18T09:35:38.758 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.758 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.758 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.758 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "log", 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.759 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.760 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.760 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_behaviour": { 2023-10-18T09:35:38.760 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_behaviour", 2023-10-18T09:35:38.760 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "default", 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.761 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: "default", 2023-10-18T09:35:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: "error" 2023-10-18T09:35:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.763 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.763 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.763 INFO:teuthology.orchestra.run.smithi061.stdout: "standby_error_status_code": { 2023-10-18T09:35:38.763 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "standby_error_status_code", 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "500", 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "400", 2023-10-18T09:35:38.764 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "599", 2023-10-18T09:35:38.765 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.765 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.765 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.765 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.765 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_support", 2023-10-18T09:35:38.766 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.767 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.767 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.769 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.769 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.770 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.770 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.770 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.770 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.770 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.771 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.772 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.772 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.772 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.772 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.772 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.773 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.774 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.774 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.774 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.774 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.774 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.775 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.776 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.777 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.778 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.778 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.778 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.778 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.778 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.779 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.780 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.780 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.780 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.780 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.780 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.782 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_snap_create": { 2023-10-18T09:35:38.782 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_snap_create", 2023-10-18T09:35:38.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.782 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10", 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.783 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.784 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.784 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.784 INFO:teuthology.orchestra.run.smithi061.stdout: "mirror_snapshot_schedule": { 2023-10-18T09:35:38.784 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mirror_snapshot_schedule", 2023-10-18T09:35:38.785 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.785 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.785 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.785 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.785 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.786 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.787 INFO:teuthology.orchestra.run.smithi061.stdout: "trash_purge_schedule": { 2023-10-18T09:35:38.787 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "trash_purge_schedule", 2023-10-18T09:35:38.787 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.787 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.788 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.789 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.789 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.789 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "restful", 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.790 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.791 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_auth": { 2023-10-18T09:35:38.791 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enable_auth", 2023-10-18T09:35:38.791 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.791 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.791 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.792 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: "key_file": { 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "key_file", 2023-10-18T09:35:38.793 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.794 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.794 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.794 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.794 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.794 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.796 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.796 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.796 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.796 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.797 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.798 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.798 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.798 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.798 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.798 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.799 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.800 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.800 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.800 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.800 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.800 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.801 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.802 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.802 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.802 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.802 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.803 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.804 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.805 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.805 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.805 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.805 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.805 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.806 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.807 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.807 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.807 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.807 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.807 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: "server_addr": { 2023-10-18T09:35:38.808 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_addr", 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.809 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.810 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.810 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.810 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.810 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.810 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: "server_port": { 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "server_port", 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.811 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.812 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.812 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.812 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.812 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.812 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.813 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.813 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.813 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rgw", 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.814 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.815 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.815 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.815 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.815 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.815 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.816 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.817 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.817 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.817 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.817 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.817 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.818 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.819 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.819 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.819 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.819 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.819 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.820 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.821 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.821 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.821 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.822 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.822 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.822 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.822 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.822 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.823 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.824 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.825 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.825 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.825 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.825 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.826 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.827 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.827 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.827 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rook", 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.828 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.829 INFO:teuthology.orchestra.run.smithi061.stdout: "drive_group_interval": { 2023-10-18T09:35:38.829 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "drive_group_interval", 2023-10-18T09:35:38.829 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.829 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.829 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "300.0", 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-18T09:35:38.830 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.831 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.832 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.832 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.832 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.832 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.832 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.833 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.834 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.834 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.834 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.834 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.834 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.835 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.836 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.836 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.836 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.836 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.836 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.837 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.838 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.839 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.839 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.839 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.839 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.839 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.840 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.841 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.842 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.842 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.842 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.842 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.842 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.844 INFO:teuthology.orchestra.run.smithi061.stdout: "storage_class": { 2023-10-18T09:35:38.844 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "storage_class", 2023-10-18T09:35:38.844 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.844 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.844 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "local", 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-18T09:35:38.845 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.846 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.846 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "selftest", 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.847 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.848 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.848 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.848 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.849 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.850 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.850 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.850 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.850 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.850 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.851 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.852 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.853 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.853 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.853 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.853 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.854 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.854 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.854 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.855 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.855 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.855 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.855 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.855 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.856 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.856 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.856 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.856 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.856 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.858 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.858 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.859 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.859 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.859 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.859 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.859 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.861 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.861 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.861 INFO:teuthology.orchestra.run.smithi061.stdout: "roption1": { 2023-10-18T09:35:38.861 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption1", 2023-10-18T09:35:38.862 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.862 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.862 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.862 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.863 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.863 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.863 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.863 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.863 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.864 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.864 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.864 INFO:teuthology.orchestra.run.smithi061.stdout: "roption2": { 2023-10-18T09:35:38.864 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "roption2", 2023-10-18T09:35:38.865 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.865 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.865 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.865 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "xyz", 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.867 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.867 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption1": { 2023-10-18T09:35:38.867 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption1", 2023-10-18T09:35:38.867 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.868 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.869 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption2": { 2023-10-18T09:35:38.870 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption2", 2023-10-18T09:35:38.870 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.870 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.870 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.870 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.871 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.872 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.872 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption3": { 2023-10-18T09:35:38.872 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption3", 2023-10-18T09:35:38.872 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "float", 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.873 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.874 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.874 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.874 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.874 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption4": { 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption4", 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.875 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.876 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.877 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.877 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption5": { 2023-10-18T09:35:38.877 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption5", 2023-10-18T09:35:38.877 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.878 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.879 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption6": { 2023-10-18T09:35:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption6", 2023-10-18T09:35:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.881 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.882 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.882 INFO:teuthology.orchestra.run.smithi061.stdout: "rwoption7": { 2023-10-18T09:35:38.882 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rwoption7", 2023-10-18T09:35:38.882 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "1", 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "42", 2023-10-18T09:35:38.883 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.884 INFO:teuthology.orchestra.run.smithi061.stdout: "testkey": { 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testkey", 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.885 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.886 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.886 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.886 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.886 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.886 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: "testlkey": { 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testlkey", 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.887 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.888 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.888 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.888 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.888 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.888 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.889 INFO:teuthology.orchestra.run.smithi061.stdout: "testnewline": { 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "testnewline", 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.890 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.891 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.891 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.891 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.891 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.891 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.892 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snap_schedule", 2023-10-18T09:35:38.893 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.893 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.893 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.893 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_m_granularity": { 2023-10-18T09:35:38.893 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "allow_m_granularity", 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.894 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.895 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.895 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "allow minute scheduled snapshots", 2023-10-18T09:35:38.895 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.895 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.895 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: "dump_on_update": { 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "dump_on_update", 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.896 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "dump database to debug log on update", 2023-10-18T09:35:38.897 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.898 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.898 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.898 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.898 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.899 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.900 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.900 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.900 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.900 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.900 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.901 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.902 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.902 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.902 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.903 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.904 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.905 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.905 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.905 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.905 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.905 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.906 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.907 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.907 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.907 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.907 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.907 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.908 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.909 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.909 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.909 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.909 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.909 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.910 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "stats", 2023-10-18T09:35:38.911 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.912 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.912 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.912 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.912 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.912 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.913 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.914 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.914 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.914 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.914 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.914 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.915 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.916 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.917 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.917 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.917 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.917 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.917 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.918 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.919 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.919 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.919 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.919 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.919 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.920 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.921 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.922 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.922 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.922 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.922 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.923 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.925 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "status", 2023-10-18T09:35:38.925 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.925 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.926 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.927 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.928 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.928 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.928 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.928 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.928 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.929 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.930 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.930 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.930 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.930 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.931 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.932 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.933 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.933 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.933 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.933 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.933 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.934 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.936 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.936 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.936 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.936 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.936 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.937 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.938 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.938 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telegraf", 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.939 INFO:teuthology.orchestra.run.smithi061.stdout: "address": { 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "address", 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-18T09:35:38.940 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.941 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.941 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.941 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.941 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:38.942 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.943 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.943 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "15", 2023-10-18T09:35:38.943 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.943 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.943 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.944 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.946 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.947 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.948 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.948 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.948 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.948 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.949 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.950 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.950 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.950 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.950 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.950 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.952 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.953 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.953 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.953 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.953 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.953 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.954 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.955 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.956 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.956 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.956 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.956 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.956 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:38.957 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "telemetry", 2023-10-18T09:35:38.958 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:38.958 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:38.958 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:38.958 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_basic": { 2023-10-18T09:35:38.958 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_basic", 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.959 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.961 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_crash": { 2023-10-18T09:35:38.961 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_crash", 2023-10-18T09:35:38.961 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.961 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.961 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-18T09:35:38.962 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_device": { 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_device", 2023-10-18T09:35:38.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.964 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.964 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.964 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "True", 2023-10-18T09:35:38.964 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.964 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.966 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_ident": { 2023-10-18T09:35:38.966 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_ident", 2023-10-18T09:35:38.966 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.966 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.966 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-18T09:35:38.967 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: "channel_perf": { 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "channel_perf", 2023-10-18T09:35:38.968 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.969 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.969 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.969 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.969 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.969 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.971 INFO:teuthology.orchestra.run.smithi061.stdout: "contact": { 2023-10-18T09:35:38.971 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "contact", 2023-10-18T09:35:38.971 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.971 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.971 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.973 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.974 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.974 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.974 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.974 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.974 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: "description": { 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "description", 2023-10-18T09:35:38.975 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.976 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.976 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.976 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.976 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.976 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.978 INFO:teuthology.orchestra.run.smithi061.stdout: "device_url": { 2023-10-18T09:35:38.978 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "device_url", 2023-10-18T09:35:38.978 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.978 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.978 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.979 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-18T09:35:38.979 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.979 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.979 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.979 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: "enabled": { 2023-10-18T09:35:38.980 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "enabled", 2023-10-18T09:35:38.981 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.981 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.981 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.981 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.981 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.982 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.983 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:38.983 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:38.983 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.983 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "24", 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "8", 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.984 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.985 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.985 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.985 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.985 INFO:teuthology.orchestra.run.smithi061.stdout: "last_opt_revision": { 2023-10-18T09:35:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "last_opt_revision", 2023-10-18T09:35:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.986 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "1", 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.987 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard": { 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard", 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.988 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.989 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.989 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.989 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.989 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.989 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.990 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.990 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.990 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.990 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "leaderboard_description": { 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "leaderboard_description", 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:38.991 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.992 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.992 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.992 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.992 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.992 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:38.993 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.994 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:38.995 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:38.995 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:38.995 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:38.995 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:38.995 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.996 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.997 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:38.997 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:38.997 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:38.997 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:38.998 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:38.999 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:38.999 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:38.999 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:38.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:38.999 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:39.000 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.001 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.001 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.001 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.001 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.002 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.002 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.002 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.002 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.002 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:39.003 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.004 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.005 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.005 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.005 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.005 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.005 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.006 INFO:teuthology.orchestra.run.smithi061.stdout: "organization": { 2023-10-18T09:35:39.006 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "organization", 2023-10-18T09:35:39.006 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.006 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.007 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.007 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.007 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.007 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.007 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.008 INFO:teuthology.orchestra.run.smithi061.stdout: "proxy": { 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "proxy", 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.009 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.010 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.010 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.010 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.010 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.010 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.011 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.011 INFO:teuthology.orchestra.run.smithi061.stdout: "url": { 2023-10-18T09:35:39.011 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "url", 2023-10-18T09:35:39.011 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.012 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.013 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.013 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.013 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.013 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.013 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.014 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.014 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "test_orchestrator", 2023-10-18T09:35:39.014 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.015 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.016 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.016 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.016 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.016 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.016 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.017 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.018 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.018 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.018 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.018 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.018 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.019 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.020 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.021 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.021 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:39.021 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:39.022 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.022 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.022 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.022 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:39.022 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.023 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.024 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.024 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.024 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.024 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.024 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:39.025 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.026 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.026 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.026 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.026 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.027 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.027 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.027 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.027 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.027 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.028 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "volumes", 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:39.029 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.030 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.030 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.030 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.030 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.030 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.031 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.032 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.032 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.032 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.032 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.032 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.033 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.033 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:39.033 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:39.033 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.034 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.034 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.034 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.034 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.034 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.036 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:39.036 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:39.036 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.036 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.036 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.037 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.038 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.038 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.038 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.038 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.038 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.039 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:39.040 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:39.040 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.040 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.040 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.040 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.042 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.042 INFO:teuthology.orchestra.run.smithi061.stdout: "max_concurrent_clones": { 2023-10-18T09:35:39.042 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "max_concurrent_clones", 2023-10-18T09:35:39.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "4", 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.043 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.044 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-18T09:35:39.044 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.044 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.044 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.045 INFO:teuthology.orchestra.run.smithi061.stdout: "periodic_async_work": { 2023-10-18T09:35:39.045 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "periodic_async_work", 2023-10-18T09:35:39.045 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.045 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.045 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Periodically check for async work", 2023-10-18T09:35:39.046 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: "snapshot_clone_delay": { 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "snapshot_clone_delay", 2023-10-18T09:35:39.047 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:39.048 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.048 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.048 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "0", 2023-10-18T09:35:39.048 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.048 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.049 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.049 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-18T09:35:39.049 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.049 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.049 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix", 2023-10-18T09:35:39.050 INFO:teuthology.orchestra.run.smithi061.stdout: "can_run": true, 2023-10-18T09:35:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "error_string": "", 2023-10-18T09:35:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "module_options": { 2023-10-18T09:35:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "discovery_interval": { 2023-10-18T09:35:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "discovery_interval", 2023-10-18T09:35:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "uint", 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "100", 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.052 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.053 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.053 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.053 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.053 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "identifier": { 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "identifier", 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.054 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.055 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.055 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.055 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.055 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.055 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: "interval": { 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "interval", 2023-10-18T09:35:39.056 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "secs", 2023-10-18T09:35:39.057 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.057 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.057 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "60", 2023-10-18T09:35:39.057 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.057 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.059 INFO:teuthology.orchestra.run.smithi061.stdout: "log_level": { 2023-10-18T09:35:39.059 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_level", 2023-10-18T09:35:39.059 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.059 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.059 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.060 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.061 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.061 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.061 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.061 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.061 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.062 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster": { 2023-10-18T09:35:39.063 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster", 2023-10-18T09:35:39.063 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.063 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.063 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.063 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.064 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.065 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.065 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_cluster_level": { 2023-10-18T09:35:39.065 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_cluster_level", 2023-10-18T09:35:39.065 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.066 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.066 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.066 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "info", 2023-10-18T09:35:39.066 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.066 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [ 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "", 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "critical", 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "debug", 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "error", 2023-10-18T09:35:39.067 INFO:teuthology.orchestra.run.smithi061.stdout: "info", 2023-10-18T09:35:39.068 INFO:teuthology.orchestra.run.smithi061.stdout: "warning" 2023-10-18T09:35:39.068 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.068 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.068 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.069 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.069 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": { 2023-10-18T09:35:39.069 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "log_to_file", 2023-10-18T09:35:39.069 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "bool", 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "False", 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.070 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.071 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.071 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.071 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.071 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_host": { 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_host", 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.072 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "", 2023-10-18T09:35:39.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.073 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.073 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.073 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.073 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.074 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.074 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.074 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_port": { 2023-10-18T09:35:39.074 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_port", 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "int", 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "10051", 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.075 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.077 INFO:teuthology.orchestra.run.smithi061.stdout: "zabbix_sender": { 2023-10-18T09:35:39.077 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "zabbix_sender", 2023-10-18T09:35:39.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "str", 2023-10-18T09:35:39.077 INFO:teuthology.orchestra.run.smithi061.stdout: "level": "advanced", 2023-10-18T09:35:39.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 0, 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "min": "", 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "max": "", 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "enum_allowed": [], 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "desc": "", 2023-10-18T09:35:39.078 INFO:teuthology.orchestra.run.smithi061.stdout: "long_desc": "", 2023-10-18T09:35:39.079 INFO:teuthology.orchestra.run.smithi061.stdout: "tags": [], 2023-10-18T09:35:39.079 INFO:teuthology.orchestra.run.smithi061.stdout: "see_also": [] 2023-10-18T09:35:39.079 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.079 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.079 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: "services": { 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: "dashboard": "https://172.21.15.61:8443/" 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: "always_on_modules": { 2023-10-18T09:35:39.080 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus": [ 2023-10-18T09:35:39.081 INFO:teuthology.orchestra.run.smithi061.stdout: "balancer", 2023-10-18T09:35:39.081 INFO:teuthology.orchestra.run.smithi061.stdout: "crash", 2023-10-18T09:35:39.081 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", 2023-10-18T09:35:39.081 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", 2023-10-18T09:35:39.081 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: "progress", 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_support", 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: "status", 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: "telemetry", 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: "volumes" 2023-10-18T09:35:39.082 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific": [ 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "balancer", 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "crash", 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", 2023-10-18T09:35:39.083 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", 2023-10-18T09:35:39.084 INFO:teuthology.orchestra.run.smithi061.stdout: "progress", 2023-10-18T09:35:39.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_support", 2023-10-18T09:35:39.084 INFO:teuthology.orchestra.run.smithi061.stdout: "status", 2023-10-18T09:35:39.084 INFO:teuthology.orchestra.run.smithi061.stdout: "telemetry", 2023-10-18T09:35:39.084 INFO:teuthology.orchestra.run.smithi061.stdout: "volumes" 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy": [ 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: "balancer", 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: "crash", 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", 2023-10-18T09:35:39.085 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", 2023-10-18T09:35:39.086 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", 2023-10-18T09:35:39.086 INFO:teuthology.orchestra.run.smithi061.stdout: "progress", 2023-10-18T09:35:39.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_support", 2023-10-18T09:35:39.086 INFO:teuthology.orchestra.run.smithi061.stdout: "status", 2023-10-18T09:35:39.086 INFO:teuthology.orchestra.run.smithi061.stdout: "telemetry", 2023-10-18T09:35:39.087 INFO:teuthology.orchestra.run.smithi061.stdout: "volumes" 2023-10-18T09:35:39.087 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-10-18T09:35:39.087 INFO:teuthology.orchestra.run.smithi061.stdout: "reef": [ 2023-10-18T09:35:39.087 INFO:teuthology.orchestra.run.smithi061.stdout: "balancer", 2023-10-18T09:35:39.087 INFO:teuthology.orchestra.run.smithi061.stdout: "crash", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "devicehealth", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "orchestrator", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscaler", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "progress", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_support", 2023-10-18T09:35:39.088 INFO:teuthology.orchestra.run.smithi061.stdout: "status", 2023-10-18T09:35:39.089 INFO:teuthology.orchestra.run.smithi061.stdout: "telemetry", 2023-10-18T09:35:39.089 INFO:teuthology.orchestra.run.smithi061.stdout: "volumes" 2023-10-18T09:35:39.089 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.089 INFO:teuthology.orchestra.run.smithi061.stdout: "last_failure_osd_epoch": 3, 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: "active_clients": [ 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "devicehealth", 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.090 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-10-18T09:35:39.091 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:0", 2023-10-18T09:35:39.091 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 283719276 2023-10-18T09:35:39.091 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.091 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "libcephsqlite", 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-10-18T09:35:39.092 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:0", 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 217188384 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.093 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "rbd_support", 2023-10-18T09:35:39.094 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-10-18T09:35:39.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.094 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-10-18T09:35:39.094 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:0", 2023-10-18T09:35:39.094 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 1868139328 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "volumes", 2023-10-18T09:35:39.095 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-10-18T09:35:39.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-10-18T09:35:39.096 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-10-18T09:35:39.096 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:0", 2023-10-18T09:35:39.096 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3233637384 2023-10-18T09:35:39.096 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.097 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.097 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:39.097 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-10-18T09:35:39.097 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:39.099 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:35 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1465377122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:35:39.099 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:35 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2298346458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:35:39.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:36 smithi110 ceph-mon[134818]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:39.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:36 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/730490239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T09:35:39.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:37 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:35:39.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:35:39.100 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.101 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.101 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.101 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.101 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:35:39.101 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:35:39.102 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:38 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:35:39.102 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:35 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1465377122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:35:39.102 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:35 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2298346458' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:35:39.102 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:36 smithi170 ceph-mon[135150]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:39.102 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:36 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/730490239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:37 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.103 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:35:39.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:35:39.104 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:38 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:35:39.104 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 msgr2=0x7f77d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:39.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 0x7f77d80a3940 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77e4007c60 tx=0x7f77e40073d0 comp rx=0 tx=0).stop 2023-10-18T09:35:39.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 msgr2=0x7f77f01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:35:39.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000c00 tx=0x7f77e0005740 comp rx=0 tx=0).stop 2023-10-18T09:35:39.105 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 shutdown_connections 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77d80a1480 0x7f77d80a3940 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77f0108610 0x7f77f019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77f00731f0 0x7f77f01988b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 --2- 172.21.15.61:0/730490239 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f77f00747d0 0x7f77f0198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.546+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 >> 172.21.15.61:0/730490239 conn(0x7f77f00fbc90 msgr2=0x7f77f0104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:35:39.106 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.547+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 shutdown_connections 2023-10-18T09:35:39.107 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:35:35.547+0000 7f77f4dff700 1 -- 172.21.15.61:0/730490239 wait complete. 2023-10-18T09:35:39.113 DEBUG:tasks.check_counter:Getting stats from a 2023-10-18T09:35:39.113 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-18T09:35:39.114 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:36 smithi061 ceph-mon[98573]: pgmap v869: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:35:39.114 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:36 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/730490239' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-18T09:35:39.114 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:37 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-18T09:35:39.114 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: pgmap v870: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:35:39.115 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.115 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.115 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.115 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-18T09:35:39.115 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-18T09:35:39.116 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-18T09:35:39.116 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:38 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' 2023-10-18T09:35:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:39 smithi061 ceph-mon[98573]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:35:39.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:39 smithi061 ceph-mon[98573]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:35:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:39 smithi170 ceph-mon[135150]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:35:39.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:39 smithi170 ceph-mon[135150]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:35:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:39 smithi110 ceph-mon[134818]: Adjusting osd_memory_target on smithi061 to 221.9M 2023-10-18T09:35:39.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:39 smithi110 ceph-mon[134818]: Unable to set osd_memory_target on smithi061 to 232775065: error parsing value: Value '232775065' is below minimum 939524096 2023-10-18T09:35:39.866 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:35:40.405 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:40 smithi061 ceph-mon[98573]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-10-18T09:35:40.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:40 smithi170 ceph-mon[135150]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-10-18T09:35:40.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:40 smithi110 ceph-mon[134818]: pgmap v871: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2023-10-18T09:35:41.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:41 smithi110 ceph-mon[134818]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-10-18T09:35:41.754 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:41 smithi061 ceph-mon[98573]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-10-18T09:35:41.777 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:41.777 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:35:41.777 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 1066, 2023-10-18T09:35:41.778 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 1069, 2023-10-18T09:35:41.778 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 358121, 2023-10-18T09:35:41.778 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 331527, 2023-10-18T09:35:41.778 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 8, 2023-10-18T09:35:41.778 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 2, 2023-10-18T09:35:41.779 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.219199423, 2023-10-18T09:35:41.779 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.089450400, 2023-10-18T09:35:41.779 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.041821211, 2023-10-18T09:35:41.779 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.067101332, 2023-10-18T09:35:41.780 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:41.780 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 1066, 2023-10-18T09:35:41.780 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.219514697, 2023-10-18T09:35:41.780 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000205923 2023-10-18T09:35:41.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.781 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T09:35:41.782 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T09:35:41.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.782 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:35:41.782 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 2206, 2023-10-18T09:35:41.783 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 2440, 2023-10-18T09:35:41.783 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 1626008, 2023-10-18T09:35:41.783 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 1169048, 2023-10-18T09:35:41.783 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 6, 2023-10-18T09:35:41.783 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 4, 2023-10-18T09:35:41.784 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.333551255, 2023-10-18T09:35:41.784 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.103856710, 2023-10-18T09:35:41.784 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.076033454, 2023-10-18T09:35:41.784 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.127472339, 2023-10-18T09:35:41.784 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:41.785 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 2438, 2023-10-18T09:35:41.785 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.340906158, 2023-10-18T09:35:41.785 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000139830 2023-10-18T09:35:41.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.785 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:41.786 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.786 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.786 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.787 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-10-18T09:35:41.787 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 484480 2023-10-18T09:35:41.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.788 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:35:41.788 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 346, 2023-10-18T09:35:41.788 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 324, 2023-10-18T09:35:41.788 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 772708, 2023-10-18T09:35:41.789 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 189424, 2023-10-18T09:35:41.789 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 7, 2023-10-18T09:35:41.789 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 3, 2023-10-18T09:35:41.789 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.087933981, 2023-10-18T09:35:41.789 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.032987676, 2023-10-18T09:35:41.790 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.034088339, 2023-10-18T09:35:41.790 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.017827396, 2023-10-18T09:35:41.790 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:41.791 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 323, 2023-10-18T09:35:41.791 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.065987980, 2023-10-18T09:35:41.791 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000204297 2023-10-18T09:35:41.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.792 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:41.792 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.792 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 771360, 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 188480 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.793 INFO:teuthology.orchestra.run.smithi061.stdout: "cct": { 2023-10-18T09:35:41.794 INFO:teuthology.orchestra.run.smithi061.stdout: "total_workers": 1, 2023-10-18T09:35:41.794 INFO:teuthology.orchestra.run.smithi061.stdout: "unhealthy_workers": 0 2023-10-18T09:35:41.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.794 INFO:teuthology.orchestra.run.smithi061.stdout: "finisher-MDSRank": { 2023-10-18T09:35:41.794 INFO:teuthology.orchestra.run.smithi061.stdout: "queue_len": 0, 2023-10-18T09:35:41.795 INFO:teuthology.orchestra.run.smithi061.stdout: "complete_latency": { 2023-10-18T09:35:41.795 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 5334, 2023-10-18T09:35:41.795 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.239899143, 2023-10-18T09:35:41.795 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000044975 2023-10-18T09:35:41.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.796 INFO:teuthology.orchestra.run.smithi061.stdout: "finisher-PurgeQueue": { 2023-10-18T09:35:41.796 INFO:teuthology.orchestra.run.smithi061.stdout: "queue_len": 0, 2023-10-18T09:35:41.796 INFO:teuthology.orchestra.run.smithi061.stdout: "complete_latency": { 2023-10-18T09:35:41.796 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.797 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.797 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.798 INFO:teuthology.orchestra.run.smithi061.stdout: "mds": { 2023-10-18T09:35:41.798 INFO:teuthology.orchestra.run.smithi061.stdout: "request": 0, 2023-10-18T09:35:41.799 INFO:teuthology.orchestra.run.smithi061.stdout: "reply": 0, 2023-10-18T09:35:41.799 INFO:teuthology.orchestra.run.smithi061.stdout: "reply_latency": { 2023-10-18T09:35:41.799 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.800 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.800 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.800 INFO:teuthology.orchestra.run.smithi061.stdout: "slow_reply": 0, 2023-10-18T09:35:41.801 INFO:teuthology.orchestra.run.smithi061.stdout: "forward": 0, 2023-10-18T09:35:41.801 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_fetch_complete": 0, 2023-10-18T09:35:41.801 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_fetch_keys": 0, 2023-10-18T09:35:41.802 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_commit": 0, 2023-10-18T09:35:41.802 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_split": 0, 2023-10-18T09:35:41.803 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_merge": 0, 2023-10-18T09:35:41.803 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes": 318, 2023-10-18T09:35:41.803 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_top": 0, 2023-10-18T09:35:41.804 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_bottom": 0, 2023-10-18T09:35:41.804 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_pin_tail": 318, 2023-10-18T09:35:41.804 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_pinned": 318, 2023-10-18T09:35:41.805 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_expired": 0, 2023-10-18T09:35:41.805 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_with_caps": 0, 2023-10-18T09:35:41.805 INFO:teuthology.orchestra.run.smithi061.stdout: "caps": 0, 2023-10-18T09:35:41.806 INFO:teuthology.orchestra.run.smithi061.stdout: "subtrees": 10, 2023-10-18T09:35:41.806 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse": 0, 2023-10-18T09:35:41.806 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_hit": 0, 2023-10-18T09:35:41.807 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_forward": 0, 2023-10-18T09:35:41.807 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_discover": 0, 2023-10-18T09:35:41.808 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:35:41.808 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_remote_ino": 0, 2023-10-18T09:35:41.808 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_lock": 0, 2023-10-18T09:35:41.808 INFO:teuthology.orchestra.run.smithi061.stdout: "load_cent": 0, 2023-10-18T09:35:41.809 INFO:teuthology.orchestra.run.smithi061.stdout: "q": 0, 2023-10-18T09:35:41.809 INFO:teuthology.orchestra.run.smithi061.stdout: "exported": 0, 2023-10-18T09:35:41.809 INFO:teuthology.orchestra.run.smithi061.stdout: "exported_inodes": 0, 2023-10-18T09:35:41.809 INFO:teuthology.orchestra.run.smithi061.stdout: "imported": 0, 2023-10-18T09:35:41.809 INFO:teuthology.orchestra.run.smithi061.stdout: "imported_inodes": 0, 2023-10-18T09:35:41.810 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_dir_fetch": 0, 2023-10-18T09:35:41.810 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_backtrace_fetch": 0, 2023-10-18T09:35:41.810 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_peer_discover": 0, 2023-10-18T09:35:41.810 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rfiles": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rbytes": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rsnaps": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_set_tag": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:35:41.811 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:35:41.812 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:35:41.812 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:35:41.812 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:35:41.812 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_file_inodes": 0, 2023-10-18T09:35:41.813 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:35:41.813 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T09:35:41.813 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_grant": 0, 2023-10-18T09:35:41.813 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:35:41.814 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:35:41.814 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:35:41.814 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_caps": 0, 2023-10-18T09:35:41.814 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_caps_dirty": 0, 2023-10-18T09:35:41.814 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_cap_release": 0, 2023-10-18T09:35:41.815 INFO:teuthology.orchestra.run.smithi061.stdout: "process_request_cap_release": 0 2023-10-18T09:35:41.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.815 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache": { 2023-10-18T09:35:41.815 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_update": 0, 2023-10-18T09:35:41.815 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_update_receipt": 0, 2023-10-18T09:35:41.816 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_try_discover": 0, 2023-10-18T09:35:41.816 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_send_discover": 0, 2023-10-18T09:35:41.816 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_handle_discover": 0, 2023-10-18T09:35:41.816 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays": 0, 2023-10-18T09:35:41.816 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays_delayed": 0, 2023-10-18T09:35:41.817 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:35:41.817 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_created": 0, 2023-10-18T09:35:41.817 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_enqueued": 0, 2023-10-18T09:35:41.817 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_reintegrated": 0, 2023-10-18T09:35:41.818 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_migrated": 0, 2023-10-18T09:35:41.818 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_processing": 0, 2023-10-18T09:35:41.818 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:35:41.819 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:35:41.819 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_started": 0, 2023-10-18T09:35:41.819 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_completed": 0, 2023-10-18T09:35:41.820 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:35:41.820 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_exportdir": 0, 2023-10-18T09:35:41.821 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_flush": 0, 2023-10-18T09:35:41.821 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:35:41.821 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_fragstats": 0, 2023-10-18T09:35:41.822 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_inodestats": 0 2023-10-18T09:35:41.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.823 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log": { 2023-10-18T09:35:41.823 INFO:teuthology.orchestra.run.smithi061.stdout: "evlrg": 0, 2023-10-18T09:35:41.823 INFO:teuthology.orchestra.run.smithi061.stdout: "evadd": 0, 2023-10-18T09:35:41.824 INFO:teuthology.orchestra.run.smithi061.stdout: "evex": 0, 2023-10-18T09:35:41.824 INFO:teuthology.orchestra.run.smithi061.stdout: "evtrm": 0, 2023-10-18T09:35:41.824 INFO:teuthology.orchestra.run.smithi061.stdout: "ev": 714, 2023-10-18T09:35:41.825 INFO:teuthology.orchestra.run.smithi061.stdout: "evexg": 0, 2023-10-18T09:35:41.825 INFO:teuthology.orchestra.run.smithi061.stdout: "evexd": 0, 2023-10-18T09:35:41.825 INFO:teuthology.orchestra.run.smithi061.stdout: "segadd": 0, 2023-10-18T09:35:41.826 INFO:teuthology.orchestra.run.smithi061.stdout: "segex": 0, 2023-10-18T09:35:41.826 INFO:teuthology.orchestra.run.smithi061.stdout: "segtrm": 0, 2023-10-18T09:35:41.827 INFO:teuthology.orchestra.run.smithi061.stdout: "seg": 2, 2023-10-18T09:35:41.827 INFO:teuthology.orchestra.run.smithi061.stdout: "segmjr": 2, 2023-10-18T09:35:41.827 INFO:teuthology.orchestra.run.smithi061.stdout: "segexg": 0, 2023-10-18T09:35:41.827 INFO:teuthology.orchestra.run.smithi061.stdout: "segexd": 0, 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "expos": 4194304, 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "wrpos": 5258610, 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "rdpos": 5258036, 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "jlat": { 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.828 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.829 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.829 INFO:teuthology.orchestra.run.smithi061.stdout: "replayed": 714 2023-10-18T09:35:41.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.829 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_mem": { 2023-10-18T09:35:41.830 INFO:teuthology.orchestra.run.smithi061.stdout: "ino": 316, 2023-10-18T09:35:41.830 INFO:teuthology.orchestra.run.smithi061.stdout: "ino+": 317, 2023-10-18T09:35:41.830 INFO:teuthology.orchestra.run.smithi061.stdout: "ino-": 1, 2023-10-18T09:35:41.831 INFO:teuthology.orchestra.run.smithi061.stdout: "dir": 18, 2023-10-18T09:35:41.833 INFO:teuthology.orchestra.run.smithi061.stdout: "dir+": 18, 2023-10-18T09:35:41.834 INFO:teuthology.orchestra.run.smithi061.stdout: "dir-": 0, 2023-10-18T09:35:41.834 INFO:teuthology.orchestra.run.smithi061.stdout: "dn": 318, 2023-10-18T09:35:41.834 INFO:teuthology.orchestra.run.smithi061.stdout: "dn+": 318, 2023-10-18T09:35:41.834 INFO:teuthology.orchestra.run.smithi061.stdout: "dn-": 0, 2023-10-18T09:35:41.835 INFO:teuthology.orchestra.run.smithi061.stdout: "cap": 0, 2023-10-18T09:35:41.835 INFO:teuthology.orchestra.run.smithi061.stdout: "cap+": 0, 2023-10-18T09:35:41.835 INFO:teuthology.orchestra.run.smithi061.stdout: "cap-": 0, 2023-10-18T09:35:41.835 INFO:teuthology.orchestra.run.smithi061.stdout: "rss": 51196, 2023-10-18T09:35:41.835 INFO:teuthology.orchestra.run.smithi061.stdout: "heap": 207152 2023-10-18T09:35:41.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.836 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_server": { 2023-10-18T09:35:41.836 INFO:teuthology.orchestra.run.smithi061.stdout: "dispatch_client_request": 0, 2023-10-18T09:35:41.836 INFO:teuthology.orchestra.run.smithi061.stdout: "dispatch_server_request": 0, 2023-10-18T09:35:41.836 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_request": 0, 2023-10-18T09:35:41.837 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_session": 0, 2023-10-18T09:35:41.837 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_peer_request": 0, 2023-10-18T09:35:41.837 INFO:teuthology.orchestra.run.smithi061.stdout: "req_create_latency": { 2023-10-18T09:35:41.837 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.837 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.838 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.838 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getattr_latency": { 2023-10-18T09:35:41.838 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.838 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.839 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.839 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getfilelock_latency": { 2023-10-18T09:35:41.839 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.839 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.840 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.840 INFO:teuthology.orchestra.run.smithi061.stdout: "req_link_latency": { 2023-10-18T09:35:41.840 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.840 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.841 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.841 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookup_latency": { 2023-10-18T09:35:41.841 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.841 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.842 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.842 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookuphash_latency": { 2023-10-18T09:35:41.842 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.842 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.843 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.843 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupino_latency": { 2023-10-18T09:35:41.843 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.843 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.844 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.844 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupname_latency": { 2023-10-18T09:35:41.844 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.844 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.845 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.845 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupparent_latency": { 2023-10-18T09:35:41.845 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.845 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.846 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.846 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupsnap_latency": { 2023-10-18T09:35:41.847 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.847 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.847 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lssnap_latency": { 2023-10-18T09:35:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mkdir_latency": { 2023-10-18T09:35:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mknod_latency": { 2023-10-18T09:35:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.853 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.853 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mksnap_latency": { 2023-10-18T09:35:41.853 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.854 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.854 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.855 INFO:teuthology.orchestra.run.smithi061.stdout: "req_open_latency": { 2023-10-18T09:35:41.855 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.855 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.855 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.856 INFO:teuthology.orchestra.run.smithi061.stdout: "req_readdir_latency": { 2023-10-18T09:35:41.856 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.856 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.856 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.857 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rename_latency": { 2023-10-18T09:35:41.857 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.857 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.857 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.858 INFO:teuthology.orchestra.run.smithi061.stdout: "req_renamesnap_latency": { 2023-10-18T09:35:41.858 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.858 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.858 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.859 INFO:teuthology.orchestra.run.smithi061.stdout: "req_snapdiff_latency": { 2023-10-18T09:35:41.859 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.859 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.860 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.860 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmdir_latency": { 2023-10-18T09:35:41.860 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.860 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.861 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.861 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmsnap_latency": { 2023-10-18T09:35:41.861 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.862 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.862 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.862 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmxattr_latency": { 2023-10-18T09:35:41.862 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.863 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.863 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.863 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setattr_latency": { 2023-10-18T09:35:41.863 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.864 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.864 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.865 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setdirlayout_latency": { 2023-10-18T09:35:41.865 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.865 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.865 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.866 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setfilelock_latency": { 2023-10-18T09:35:41.866 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.866 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.867 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.867 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setlayout_latency": { 2023-10-18T09:35:41.867 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.867 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setxattr_latency": { 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.868 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.869 INFO:teuthology.orchestra.run.smithi061.stdout: "req_symlink_latency": { 2023-10-18T09:35:41.869 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.869 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.869 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.870 INFO:teuthology.orchestra.run.smithi061.stdout: "req_unlink_latency": { 2023-10-18T09:35:41.870 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.870 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.871 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.871 INFO:teuthology.orchestra.run.smithi061.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:35:41.871 INFO:teuthology.orchestra.run.smithi061.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:35:41.871 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getvxattr_latency": { 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.872 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_sessions": { 2023-10-18T09:35:41.873 INFO:teuthology.orchestra.run.smithi061.stdout: "session_count": 1, 2023-10-18T09:35:41.873 INFO:teuthology.orchestra.run.smithi061.stdout: "session_add": 1, 2023-10-18T09:35:41.873 INFO:teuthology.orchestra.run.smithi061.stdout: "session_remove": 0, 2023-10-18T09:35:41.873 INFO:teuthology.orchestra.run.smithi061.stdout: "sessions_open": 1, 2023-10-18T09:35:41.873 INFO:teuthology.orchestra.run.smithi061.stdout: "sessions_stale": 0, 2023-10-18T09:35:41.874 INFO:teuthology.orchestra.run.smithi061.stdout: "total_load": 0, 2023-10-18T09:35:41.874 INFO:teuthology.orchestra.run.smithi061.stdout: "average_load": 0, 2023-10-18T09:35:41.874 INFO:teuthology.orchestra.run.smithi061.stdout: "avg_session_uptime": 2672, 2023-10-18T09:35:41.874 INFO:teuthology.orchestra.run.smithi061.stdout: "mdthresh_evicted": 0 2023-10-18T09:35:41.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.875 INFO:teuthology.orchestra.run.smithi061.stdout: "mempool": { 2023-10-18T09:35:41.875 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:35:41.875 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_items": 0, 2023-10-18T09:35:41.875 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:35:41.875 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:35:41.876 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:35:41.876 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:35:41.876 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:35:41.876 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:35:41.876 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:35:41.877 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:35:41.877 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:35:41.877 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:35:41.877 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:35:41.877 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:35:41.878 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:35:41.878 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_items": 0, 2023-10-18T09:35:41.878 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:35:41.879 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_items": 0, 2023-10-18T09:35:41.879 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:35:41.879 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:35:41.880 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:35:41.880 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:35:41.880 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:35:41.881 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:35:41.881 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:35:41.881 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_items": 0, 2023-10-18T09:35:41.882 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:35:41.882 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:35:41.882 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:35:41.883 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_items": 0, 2023-10-18T09:35:41.883 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_bytes": 0, 2023-10-18T09:35:41.883 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_items": 0, 2023-10-18T09:35:41.883 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:35:41.884 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:35:41.884 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:35:41.884 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:35:41.885 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_bytes": 209661, 2023-10-18T09:35:41.885 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_items": 365, 2023-10-18T09:35:41.885 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:35:41.886 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_items": 0, 2023-10-18T09:35:41.886 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bytes": 0, 2023-10-18T09:35:41.886 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_items": 0, 2023-10-18T09:35:41.886 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:35:41.887 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_items": 0, 2023-10-18T09:35:41.887 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:35:41.887 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, 2023-10-18T09:35:41.887 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 8888, 2023-10-18T09:35:41.888 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_items": 170, 2023-10-18T09:35:41.888 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:35:41.888 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:35:41.888 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_bytes": 0, 2023-10-18T09:35:41.888 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_items": 0, 2023-10-18T09:35:41.889 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_bytes": 961145, 2023-10-18T09:35:41.889 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_items": 8799, 2023-10-18T09:35:41.889 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_bytes": 0, 2023-10-18T09:35:41.889 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_items": 0, 2023-10-18T09:35:41.889 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_bytes": 0, 2023-10-18T09:35:41.890 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_items": 0 2023-10-18T09:35:41.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.890 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter": { 2023-10-18T09:35:41.890 INFO:teuthology.orchestra.run.smithi061.stdout: "op_active": 0, 2023-10-18T09:35:41.890 INFO:teuthology.orchestra.run.smithi061.stdout: "op_laggy": 0, 2023-10-18T09:35:41.891 INFO:teuthology.orchestra.run.smithi061.stdout: "op_send": 3270, 2023-10-18T09:35:41.891 INFO:teuthology.orchestra.run.smithi061.stdout: "op_send_bytes": 21, 2023-10-18T09:35:41.891 INFO:teuthology.orchestra.run.smithi061.stdout: "op_resend": 3, 2023-10-18T09:35:41.891 INFO:teuthology.orchestra.run.smithi061.stdout: "op_reply": 3270, 2023-10-18T09:35:41.891 INFO:teuthology.orchestra.run.smithi061.stdout: "op_latency": { 2023-10-18T09:35:41.892 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3270, 2023-10-18T09:35:41.892 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 58.194060473, 2023-10-18T09:35:41.892 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.017796348 2023-10-18T09:35:41.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.892 INFO:teuthology.orchestra.run.smithi061.stdout: "op_inflight": 0, 2023-10-18T09:35:41.893 INFO:teuthology.orchestra.run.smithi061.stdout: "oplen_avg": { 2023-10-18T09:35:41.893 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3270, 2023-10-18T09:35:41.893 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 3271 2023-10-18T09:35:41.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.893 INFO:teuthology.orchestra.run.smithi061.stdout: "op": 3270, 2023-10-18T09:35:41.894 INFO:teuthology.orchestra.run.smithi061.stdout: "op_r": 3270, 2023-10-18T09:35:41.894 INFO:teuthology.orchestra.run.smithi061.stdout: "op_w": 0, 2023-10-18T09:35:41.894 INFO:teuthology.orchestra.run.smithi061.stdout: "op_rmw": 0, 2023-10-18T09:35:41.894 INFO:teuthology.orchestra.run.smithi061.stdout: "op_pg": 0, 2023-10-18T09:35:41.895 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_stat": 2003, 2023-10-18T09:35:41.895 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_create": 0, 2023-10-18T09:35:41.895 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_read": 1265, 2023-10-18T09:35:41.896 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_write": 0, 2023-10-18T09:35:41.896 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_writefull": 0, 2023-10-18T09:35:41.896 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_writesame": 0, 2023-10-18T09:35:41.896 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_append": 0, 2023-10-18T09:35:41.897 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_zero": 0, 2023-10-18T09:35:41.897 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_truncate": 0, 2023-10-18T09:35:41.898 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_delete": 0, 2023-10-18T09:35:41.898 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_mapext": 0, 2023-10-18T09:35:41.898 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_sparse_read": 0, 2023-10-18T09:35:41.899 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_clonerange": 0, 2023-10-18T09:35:41.899 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_getxattr": 1, 2023-10-18T09:35:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_setxattr": 0, 2023-10-18T09:35:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:35:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_rmxattr": 0, 2023-10-18T09:35:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:35:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_call": 0, 2023-10-18T09:35:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_watch": 0, 2023-10-18T09:35:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_notify": 0, 2023-10-18T09:35:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:35:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_pgls": 0, 2023-10-18T09:35:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:35:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_other": 0, 2023-10-18T09:35:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_active": 0, 2023-10-18T09:35:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_send": 0, 2023-10-18T09:35:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_resend": 0, 2023-10-18T09:35:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_ping": 0, 2023-10-18T09:35:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_active": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_send": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_resend": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_active": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_send": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_resend": 0, 2023-10-18T09:35:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_active": 0, 2023-10-18T09:35:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_send": 0, 2023-10-18T09:35:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_resend": 0, 2023-10-18T09:35:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "command_active": 0, 2023-10-18T09:35:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "command_send": 0, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "command_resend": 0, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "map_epoch": 92, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "map_full": 0, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "map_inc": 4, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_sessions": 6, 2023-10-18T09:35:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_session_open": 9, 2023-10-18T09:35:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_session_close": 3, 2023-10-18T09:35:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_laggy": 0, 2023-10-18T09:35:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_wr": 0, 2023-10-18T09:35:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_rd": 2, 2023-10-18T09:35:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_del": 0 2023-10-18T09:35:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "oft": { 2023-10-18T09:35:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_objs": 0, 2023-10-18T09:35:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:35:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_updates": 0, 2023-10-18T09:35:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_removes": 0 2023-10-18T09:35:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "osd-slow-ops": { 2023-10-18T09:35:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "slow_ops_count": 0 2023-10-18T09:35:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "purge_queue": { 2023-10-18T09:35:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_ops": 0, 2023-10-18T09:35:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T09:35:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing": 0, 2023-10-18T09:35:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_high_water": 0, 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executed": 0, 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_item_in_journal": 0 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 3614, 2023-10-18T09:35:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 2411034, 2023-10-18T09:35:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 3614, 2023-10-18T09:35:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 3614, 2023-10-18T09:35:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 2411034, 2023-10-18T09:35:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-objecter_bytes": { 2023-10-18T09:35:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 3270, 2023-10-18T09:35:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 1064311, 2023-10-18T09:35:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 3270, 2023-10-18T09:35:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 1064311, 2023-10-18T09:35:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-objecter_ops": { 2023-10-18T09:35:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 1024, 2023-10-18T09:35:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 3270, 2023-10-18T09:35:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 3270, 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 3270, 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 3270, 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:35:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 3758096384, 2023-10-18T09:35:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 0, 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 0, 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-write_buf_throttle-0x5568fdfc0a00": { 2023-10-18T09:35:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 3758096384, 2023-10-18T09:35:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 0, 2023-10-18T09:35:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 0, 2023-10-18T09:35:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:41.936 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:41.937 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:41 smithi170 ceph-mon[135150]: pgmap v872: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2023-10-18T09:35:42.370 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-10-18T09:35:42.370 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-10-18T09:35:42.371 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-10-18T09:35:42.371 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-10-18T09:35:42.371 DEBUG:tasks.check_counter:Getting stats from d 2023-10-18T09:35:42.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-18T09:35:43.135 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:35:43.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:43 smithi110 ceph-mon[134818]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-10-18T09:35:43.753 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:43 smithi061 ceph-mon[98573]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-10-18T09:35:43.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:43 smithi170 ceph-mon[135150]: pgmap v873: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-10-18T09:35:44.989 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:44.989 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:35:44.989 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 319, 2023-10-18T09:35:44.990 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 316, 2023-10-18T09:35:44.990 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 501328, 2023-10-18T09:35:44.990 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 190352, 2023-10-18T09:35:44.990 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 4, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 0, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.090768240, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.043975615, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.027840097, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.010661707, 2023-10-18T09:35:44.991 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:44.992 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 315, 2023-10-18T09:35:44.992 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.056754729, 2023-10-18T09:35:44.992 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000180173 2023-10-18T09:35:44.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.992 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:44.993 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:44.993 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:44.993 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:44.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.994 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 501328, 2023-10-18T09:35:44.994 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 190352 2023-10-18T09:35:44.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.994 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:35:44.995 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 2, 2023-10-18T09:35:44.995 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 241, 2023-10-18T09:35:44.995 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 320, 2023-10-18T09:35:44.995 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 195824, 2023-10-18T09:35:44.996 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 4, 2023-10-18T09:35:44.996 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 2, 2023-10-18T09:35:44.996 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.037483515, 2023-10-18T09:35:44.996 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.030547848, 2023-10-18T09:35:44.996 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.003440686, 2023-10-18T09:35:44.997 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T09:35:44.997 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:44.997 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 239, 2023-10-18T09:35:44.997 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.052313273, 2023-10-18T09:35:44.997 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000218883 2023-10-18T09:35:44.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.998 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:44.998 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:44.998 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:44.998 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:44.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.999 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T09:35:44.999 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 195824 2023-10-18T09:35:44.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:44.999 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:35:45.000 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 4, 2023-10-18T09:35:45.000 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 4, 2023-10-18T09:35:45.000 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 2992, 2023-10-18T09:35:45.000 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 448, 2023-10-18T09:35:45.000 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 2, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 1, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.001526049, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.000059523, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.001415136, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T09:35:45.001 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:45.002 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3, 2023-10-18T09:35:45.002 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.001160929, 2023-10-18T09:35:45.002 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000386976 2023-10-18T09:35:45.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:45.002 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:35:45.003 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:35:45.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:45.004 INFO:teuthology.orchestra.run.smithi061.stdout: "cct": { 2023-10-18T09:35:45.004 INFO:teuthology.orchestra.run.smithi061.stdout: "total_workers": 0, 2023-10-18T09:35:45.004 INFO:teuthology.orchestra.run.smithi061.stdout: "unhealthy_workers": 0 2023-10-18T09:35:45.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "mempool": { 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_items": 0, 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:35:45.005 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:35:45.006 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:35:45.006 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:35:45.006 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:35:45.006 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:35:45.006 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:35:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:35:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:35:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:35:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:35:45.007 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:35:45.008 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_items": 0, 2023-10-18T09:35:45.008 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:35:45.008 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_items": 0, 2023-10-18T09:35:45.008 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:35:45.009 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:35:45.010 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_items": 0, 2023-10-18T09:35:45.010 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:35:45.010 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:35:45.010 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:35:45.010 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_items": 0, 2023-10-18T09:35:45.011 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_bytes": 0, 2023-10-18T09:35:45.011 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_items": 0, 2023-10-18T09:35:45.011 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:35:45.011 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:35:45.011 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:35:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:35:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_bytes": 148704, 2023-10-18T09:35:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_items": 29, 2023-10-18T09:35:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:35:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_items": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bytes": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_items": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_items": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:35:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, 2023-10-18T09:35:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, 2023-10-18T09:35:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_items": 0, 2023-10-18T09:35:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:35:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:35:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_bytes": 0, 2023-10-18T09:35:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_items": 0, 2023-10-18T09:35:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_bytes": 928, 2023-10-18T09:35:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_items": 2, 2023-10-18T09:35:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_bytes": 0, 2023-10-18T09:35:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_items": 0, 2023-10-18T09:35:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_bytes": 0, 2023-10-18T09:35:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_items": 0 2023-10-18T09:35:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:35:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 321, 2023-10-18T09:35:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 456220, 2023-10-18T09:35:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 321, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 321, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 456220, 2023-10-18T09:35:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:45.020 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:45.575 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:45 smithi061 ceph-mon[98573]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2023-10-18T09:35:45.578 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-18T09:35:45.578 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-18T09:35:45.578 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-10-18T09:35:45.579 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-10-18T09:35:45.579 DEBUG:tasks.check_counter:Getting stats from g 2023-10-18T09:35:45.579 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-18T09:35:45.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:45 smithi110 ceph-mon[134818]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2023-10-18T09:35:45.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:45 smithi170 ceph-mon[135150]: pgmap v874: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2023-10-18T09:35:46.329 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:35:48.004 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:48.004 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:35:48.004 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 1255, 2023-10-18T09:35:48.004 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 1257, 2023-10-18T09:35:48.005 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 4605037, 2023-10-18T09:35:48.005 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 390315, 2023-10-18T09:35:48.005 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 7, 2023-10-18T09:35:48.005 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 2, 2023-10-18T09:35:48.005 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.198787210, 2023-10-18T09:35:48.006 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.055316390, 2023-10-18T09:35:48.006 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.049781536, 2023-10-18T09:35:48.006 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.074104181, 2023-10-18T09:35:48.006 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:48.006 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 1255, 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.220941366, 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000176048 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.007 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.008 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.008 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T09:35:48.008 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T09:35:48.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.009 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:35:48.009 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 1597, 2023-10-18T09:35:48.009 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 1569, 2023-10-18T09:35:48.009 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 1728304, 2023-10-18T09:35:48.010 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 577748, 2023-10-18T09:35:48.010 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 5, 2023-10-18T09:35:48.010 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 3, 2023-10-18T09:35:48.010 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.239957841, 2023-10-18T09:35:48.010 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.065748105, 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.066660261, 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.091143644, 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 1568, 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.276878391, 2023-10-18T09:35:48.011 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000176580 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.013 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 661824, 2023-10-18T09:35:48.013 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 190176 2023-10-18T09:35:48.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.013 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:35:48.013 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 1243, 2023-10-18T09:35:48.014 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 1478, 2023-10-18T09:35:48.014 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 4514485, 2023-10-18T09:35:48.014 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 956596, 2023-10-18T09:35:48.014 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 5, 2023-10-18T09:35:48.014 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 3, 2023-10-18T09:35:48.015 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.234627010, 2023-10-18T09:35:48.015 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.079432043, 2023-10-18T09:35:48.015 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.059407230, 2023-10-18T09:35:48.015 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.076480470, 2023-10-18T09:35:48.016 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:48.016 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 1477, 2023-10-18T09:35:48.016 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.285126195, 2023-10-18T09:35:48.016 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000193044 2023-10-18T09:35:48.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.017 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:48.017 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.017 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.017 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.018 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-18T09:35:48.018 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 571888 2023-10-18T09:35:48.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.018 INFO:teuthology.orchestra.run.smithi061.stdout: "cct": { 2023-10-18T09:35:48.018 INFO:teuthology.orchestra.run.smithi061.stdout: "total_workers": 1, 2023-10-18T09:35:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "unhealthy_workers": 0 2023-10-18T09:35:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "finisher-MDSRank": { 2023-10-18T09:35:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "queue_len": 0, 2023-10-18T09:35:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "complete_latency": { 2023-10-18T09:35:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 6201, 2023-10-18T09:35:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.333598083, 2023-10-18T09:35:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000053797 2023-10-18T09:35:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "finisher-PurgeQueue": { 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "queue_len": 0, 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "complete_latency": { 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds": { 2023-10-18T09:35:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "request": 0, 2023-10-18T09:35:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "reply": 0, 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "reply_latency": { 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "slow_reply": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "forward": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_fetch_complete": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_fetch_keys": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_commit": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_split": 0, 2023-10-18T09:35:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_merge": 0, 2023-10-18T09:35:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes": 394, 2023-10-18T09:35:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_top": 0, 2023-10-18T09:35:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_bottom": 0, 2023-10-18T09:35:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_pin_tail": 394, 2023-10-18T09:35:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_pinned": 394, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_expired": 3, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "inodes_with_caps": 0, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "caps": 0, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "subtrees": 26, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse": 0, 2023-10-18T09:35:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_hit": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_forward": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_discover": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_remote_ino": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "traverse_lock": 0, 2023-10-18T09:35:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "load_cent": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "q": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "exported": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "exported_inodes": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "imported": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "imported_inodes": 0, 2023-10-18T09:35:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_dir_fetch": 0, 2023-10-18T09:35:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_backtrace_fetch": 0, 2023-10-18T09:35:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "openino_peer_discover": 0, 2023-10-18T09:35:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rfiles": 1037, 2023-10-18T09:35:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rbytes": 1636383425, 2023-10-18T09:35:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "root_rsnaps": 0, 2023-10-18T09:35:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T09:35:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_set_tag": 0, 2023-10-18T09:35:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:35:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:35:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:35:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:35:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:35:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "scrub_file_inodes": 0, 2023-10-18T09:35:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_grant": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:35:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_caps": 0, 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_caps_dirty": 0, 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_cap_release": 0, 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "process_request_cap_release": 0 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache": { 2023-10-18T09:35:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_update": 0, 2023-10-18T09:35:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_update_receipt": 0, 2023-10-18T09:35:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_try_discover": 0, 2023-10-18T09:35:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_send_discover": 0, 2023-10-18T09:35:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "dir_handle_discover": 0, 2023-10-18T09:35:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays_delayed": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_created": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_enqueued": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_reintegrated": 0, 2023-10-18T09:35:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "strays_migrated": 0, 2023-10-18T09:35:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_processing": 0, 2023-10-18T09:35:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:35:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:35:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_started": 0, 2023-10-18T09:35:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_completed": 0, 2023-10-18T09:35:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:35:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_exportdir": 0, 2023-10-18T09:35:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_flush": 0, 2023-10-18T09:35:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:35:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_fragstats": 0, 2023-10-18T09:35:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "ireq_inodestats": 0 2023-10-18T09:35:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log": { 2023-10-18T09:35:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "evlrg": 0, 2023-10-18T09:35:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "evadd": 0, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "evex": 0, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "evtrm": 0, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "ev": 3249, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "evexg": 0, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "evexd": 0, 2023-10-18T09:35:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "segadd": 0, 2023-10-18T09:35:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "segex": 0, 2023-10-18T09:35:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "segtrm": 0, 2023-10-18T09:35:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "seg": 6, 2023-10-18T09:35:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "segmjr": 2, 2023-10-18T09:35:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "segexg": 0, 2023-10-18T09:35:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "segexd": 0, 2023-10-18T09:35:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "expos": 4194304, 2023-10-18T09:35:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "wrpos": 13338924, 2023-10-18T09:35:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "rdpos": 13331887, 2023-10-18T09:35:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "jlat": { 2023-10-18T09:35:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "replayed": 3249 2023-10-18T09:35:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_mem": { 2023-10-18T09:35:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "ino": 352, 2023-10-18T09:35:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "ino+": 1339, 2023-10-18T09:35:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "ino-": 987, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dir": 57, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dir+": 69, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dir-": 12, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dn": 394, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dn+": 1385, 2023-10-18T09:35:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "dn-": 991, 2023-10-18T09:35:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "cap": 0, 2023-10-18T09:35:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "cap+": 0, 2023-10-18T09:35:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "cap-": 0, 2023-10-18T09:35:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "rss": 52072, 2023-10-18T09:35:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "heap": 207152 2023-10-18T09:35:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_server": { 2023-10-18T09:35:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "dispatch_client_request": 0, 2023-10-18T09:35:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "dispatch_server_request": 0, 2023-10-18T09:35:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_request": 0, 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_client_session": 0, 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "handle_peer_request": 0, 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "req_create_latency": { 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getattr_latency": { 2023-10-18T09:35:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getfilelock_latency": { 2023-10-18T09:35:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "req_link_latency": { 2023-10-18T09:35:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookup_latency": { 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookuphash_latency": { 2023-10-18T09:35:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupino_latency": { 2023-10-18T09:35:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupname_latency": { 2023-10-18T09:35:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupparent_latency": { 2023-10-18T09:35:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lookupsnap_latency": { 2023-10-18T09:35:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "req_lssnap_latency": { 2023-10-18T09:35:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mkdir_latency": { 2023-10-18T09:35:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mknod_latency": { 2023-10-18T09:35:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "req_mksnap_latency": { 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "req_open_latency": { 2023-10-18T09:35:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "req_readdir_latency": { 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rename_latency": { 2023-10-18T09:35:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "req_renamesnap_latency": { 2023-10-18T09:35:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "req_snapdiff_latency": { 2023-10-18T09:35:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmdir_latency": { 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmsnap_latency": { 2023-10-18T09:35:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "req_rmxattr_latency": { 2023-10-18T09:35:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setattr_latency": { 2023-10-18T09:35:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setdirlayout_latency": { 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setfilelock_latency": { 2023-10-18T09:35:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setlayout_latency": { 2023-10-18T09:35:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "req_setxattr_latency": { 2023-10-18T09:35:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "req_symlink_latency": { 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "req_unlink_latency": { 2023-10-18T09:35:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:35:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:35:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "req_getvxattr_latency": { 2023-10-18T09:35:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_sessions": { 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "session_count": 1, 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "session_add": 3, 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "session_remove": 2, 2023-10-18T09:35:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "sessions_open": 1, 2023-10-18T09:35:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "sessions_stale": 0, 2023-10-18T09:35:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "total_load": 0, 2023-10-18T09:35:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "average_load": 0, 2023-10-18T09:35:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "avg_session_uptime": 2679, 2023-10-18T09:35:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "mdthresh_evicted": 0 2023-10-18T09:35:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "mempool": { 2023-10-18T09:35:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:35:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_items": 0, 2023-10-18T09:35:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:35:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:35:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:35:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:35:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:35:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:35:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:35:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:35:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_items": 0, 2023-10-18T09:35:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:35:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_items": 0, 2023-10-18T09:35:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:35:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:35:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:35:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:35:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:35:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:35:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:35:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_items": 0, 2023-10-18T09:35:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:35:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:35:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:35:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_items": 0, 2023-10-18T09:35:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_bytes": 0, 2023-10-18T09:35:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_items": 0, 2023-10-18T09:35:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_bytes": 203330, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_items": 388, 2023-10-18T09:35:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:35:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_items": 0, 2023-10-18T09:35:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bytes": 0, 2023-10-18T09:35:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_items": 0, 2023-10-18T09:35:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:35:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_items": 0, 2023-10-18T09:35:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:35:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, 2023-10-18T09:35:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 8888, 2023-10-18T09:35:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_items": 170, 2023-10-18T09:35:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:35:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:35:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_bytes": 0, 2023-10-18T09:35:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_items": 0, 2023-10-18T09:35:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_bytes": 1149119, 2023-10-18T09:35:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_items": 10203, 2023-10-18T09:35:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_bytes": 0, 2023-10-18T09:35:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_items": 0, 2023-10-18T09:35:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_bytes": 0, 2023-10-18T09:35:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_items": 0 2023-10-18T09:35:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter": { 2023-10-18T09:35:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "op_active": 0, 2023-10-18T09:35:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "op_laggy": 0, 2023-10-18T09:35:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "op_send": 3738, 2023-10-18T09:35:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "op_send_bytes": 26, 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "op_resend": 4, 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "op_reply": 3738, 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "op_latency": { 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3738, 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 30.003519165, 2023-10-18T09:35:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.008026623 2023-10-18T09:35:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "op_inflight": 0, 2023-10-18T09:35:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "oplen_avg": { 2023-10-18T09:35:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3738, 2023-10-18T09:35:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 3739 2023-10-18T09:35:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "op": 3738, 2023-10-18T09:35:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "op_r": 3738, 2023-10-18T09:35:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "op_w": 0, 2023-10-18T09:35:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "op_rmw": 0, 2023-10-18T09:35:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "op_pg": 0, 2023-10-18T09:35:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_stat": 2206, 2023-10-18T09:35:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_create": 0, 2023-10-18T09:35:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_read": 1529, 2023-10-18T09:35:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_write": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_writefull": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_writesame": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_append": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_zero": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_truncate": 0, 2023-10-18T09:35:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_delete": 0, 2023-10-18T09:35:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_mapext": 0, 2023-10-18T09:35:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_sparse_read": 0, 2023-10-18T09:35:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_clonerange": 0, 2023-10-18T09:35:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_getxattr": 2, 2023-10-18T09:35:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_setxattr": 0, 2023-10-18T09:35:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:35:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_rmxattr": 0, 2023-10-18T09:35:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:35:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_call": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_watch": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_notify": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_pgls": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:35:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "osdop_other": 0, 2023-10-18T09:35:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_active": 0, 2023-10-18T09:35:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_send": 0, 2023-10-18T09:35:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_resend": 0, 2023-10-18T09:35:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "linger_ping": 0, 2023-10-18T09:35:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_active": 0, 2023-10-18T09:35:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_send": 0, 2023-10-18T09:35:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "poolop_resend": 0, 2023-10-18T09:35:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_active": 0, 2023-10-18T09:35:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_send": 0, 2023-10-18T09:35:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "poolstat_resend": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_active": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_send": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "statfs_resend": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "command_active": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "command_send": 0, 2023-10-18T09:35:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "command_resend": 0, 2023-10-18T09:35:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "map_epoch": 92, 2023-10-18T09:35:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "map_full": 0, 2023-10-18T09:35:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "map_inc": 4, 2023-10-18T09:35:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_sessions": 5, 2023-10-18T09:35:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_session_open": 7, 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_session_close": 2, 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_laggy": 0, 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_wr": 0, 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_rd": 2, 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_del": 0 2023-10-18T09:35:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "oft": { 2023-10-18T09:35:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_objs": 0, 2023-10-18T09:35:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:35:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_updates": 0, 2023-10-18T09:35:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "omap_total_removes": 0 2023-10-18T09:35:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "osd-slow-ops": { 2023-10-18T09:35:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "slow_ops_count": 0 2023-10-18T09:35:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "purge_queue": { 2023-10-18T09:35:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_ops": 0, 2023-10-18T09:35:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T09:35:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing": 0, 2023-10-18T09:35:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executing_high_water": 0, 2023-10-18T09:35:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_executed": 0, 2023-10-18T09:35:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "pq_item_in_journal": 0 2023-10-18T09:35:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:35:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 4091, 2023-10-18T09:35:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 10458686, 2023-10-18T09:35:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 4091, 2023-10-18T09:35:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 4091, 2023-10-18T09:35:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 10458686, 2023-10-18T09:35:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-objecter_bytes": { 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 3738, 2023-10-18T09:35:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 9144630, 2023-10-18T09:35:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 3738, 2023-10-18T09:35:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 9144630, 2023-10-18T09:35:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-objecter_ops": { 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 1024, 2023-10-18T09:35:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 3738, 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 3738, 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 3738, 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 3738, 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:35:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:48.120 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 3758096384, 2023-10-18T09:35:48.120 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:48.120 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:48.120 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 0, 2023-10-18T09:35:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 0, 2023-10-18T09:35:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-write_buf_throttle-0x563f8374aa00": { 2023-10-18T09:35:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 3758096384, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 0, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 0, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 0, 2023-10-18T09:35:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 0, 2023-10-18T09:35:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:48.126 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:48.526 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-10-18T09:35:48.527 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-10-18T09:35:48.527 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-10-18T09:35:48.527 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-10-18T09:35:48.528 DEBUG:tasks.check_counter:Getting stats from j 2023-10-18T09:35:48.528 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-18T09:35:49.274 INFO:teuthology.orchestra.run.smithi061.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-18T09:35:49.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:49 smithi110 ceph-mon[134818]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-10-18T09:35:49.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:49 smithi061 ceph-mon[98573]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-10-18T09:35:49.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:49 smithi170 ceph-mon[135150]: pgmap v875: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.5 KiB/s wr, 12 op/s 2023-10-18T09:35:51.019 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-10-18T09:35:51.020 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:35:51.020 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 4, 2023-10-18T09:35:51.020 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 4, 2023-10-18T09:35:51.020 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 2992, 2023-10-18T09:35:51.021 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 448, 2023-10-18T09:35:51.021 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 4, 2023-10-18T09:35:51.021 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 0, 2023-10-18T09:35:51.021 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.004678219, 2023-10-18T09:35:51.021 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.000076869, 2023-10-18T09:35:51.022 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.001327770, 2023-10-18T09:35:51.022 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T09:35:51.022 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:51.022 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 3, 2023-10-18T09:35:51.022 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.001294702, 2023-10-18T09:35:51.023 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000431567 2023-10-18T09:35:51.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.023 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:51.023 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:51.023 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:51.024 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:51.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.024 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:35:51.024 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:35:51.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.025 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:35:51.025 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 321, 2023-10-18T09:35:51.025 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 318, 2023-10-18T09:35:51.025 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 501872, 2023-10-18T09:35:51.025 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 191600, 2023-10-18T09:35:51.026 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 2, 2023-10-18T09:35:51.026 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 1, 2023-10-18T09:35:51.026 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.093086384, 2023-10-18T09:35:51.026 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.046822021, 2023-10-18T09:35:51.026 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.023471930, 2023-10-18T09:35:51.027 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.016280223, 2023-10-18T09:35:51.027 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:51.027 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 317, 2023-10-18T09:35:51.027 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.067231147, 2023-10-18T09:35:51.027 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000212085 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:51.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.029 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 501872, 2023-10-18T09:35:51.029 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 191600 2023-10-18T09:35:51.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.029 INFO:teuthology.orchestra.run.smithi061.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:35:51.029 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_messages": 2, 2023-10-18T09:35:51.030 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages": 243, 2023-10-18T09:35:51.030 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_bytes": 320, 2023-10-18T09:35:51.030 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_bytes": 196176, 2023-10-18T09:35:51.030 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_created_connections": 4, 2023-10-18T09:35:51.030 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_active_connections": 2, 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_total_time": 0.032312301, 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_send_time": 0.024001855, 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_recv_time": 0.002894565, 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:51.031 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 241, 2023-10-18T09:35:51.032 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.044043170, 2023-10-18T09:35:51.032 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000182751 2023-10-18T09:35:51.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.032 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:51.032 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: "msgr_send_encrypted_bytes": 196176 2023-10-18T09:35:51.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.034 INFO:teuthology.orchestra.run.smithi061.stdout: "cct": { 2023-10-18T09:35:51.034 INFO:teuthology.orchestra.run.smithi061.stdout: "total_workers": 0, 2023-10-18T09:35:51.034 INFO:teuthology.orchestra.run.smithi061.stdout: "unhealthy_workers": 0 2023-10-18T09:35:51.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mempool": { 2023-10-18T09:35:51.035 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:35:51.035 INFO:teuthology.orchestra.run.smithi061.stdout: "bloom_filter_items": 0, 2023-10-18T09:35:51.035 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:35:51.035 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:35:51.035 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:35:51.036 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:35:51.037 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:35:51.037 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:35:51.037 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:35:51.037 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:35:51.037 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_items": 0, 2023-10-18T09:35:51.038 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:35:51.038 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blob_items": 0, 2023-10-18T09:35:51.038 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:35:51.038 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:35:51.038 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:35:51.039 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:35:51.039 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:35:51.039 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:35:51.039 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:35:51.039 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_txc_items": 0, 2023-10-18T09:35:51.040 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:35:51.040 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:35:51.040 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:35:51.040 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_writing_items": 0, 2023-10-18T09:35:51.040 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_bytes": 0, 2023-10-18T09:35:51.041 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_items": 0, 2023-10-18T09:35:51.041 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:35:51.041 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:35:51.041 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:35:51.041 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_bytes": 148704, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_anon_items": 29, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "buffer_meta_items": 0, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bytes": 0, 2023-10-18T09:35:51.042 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_items": 0, 2023-10-18T09:35:51.043 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:35:51.043 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mapbl_items": 0, 2023-10-18T09:35:51.043 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:35:51.043 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pglog_items": 0, 2023-10-18T09:35:51.043 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_bytes": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_items": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_bytes": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "pgmap_items": 0, 2023-10-18T09:35:51.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_bytes": 928, 2023-10-18T09:35:51.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_co_items": 2, 2023-10-18T09:35:51.045 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_bytes": 0, 2023-10-18T09:35:51.045 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_1_items": 0, 2023-10-18T09:35:51.045 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_bytes": 0, 2023-10-18T09:35:51.045 INFO:teuthology.orchestra.run.smithi061.stdout: "unittest_2_items": 0 2023-10-18T09:35:51.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-10-18T09:35:51.046 INFO:teuthology.orchestra.run.smithi061.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:35:51.046 INFO:teuthology.orchestra.run.smithi061.stdout: "val": 0, 2023-10-18T09:35:51.046 INFO:teuthology.orchestra.run.smithi061.stdout: "max": 104857600, 2023-10-18T09:35:51.046 INFO:teuthology.orchestra.run.smithi061.stdout: "get_started": 0, 2023-10-18T09:35:51.047 INFO:teuthology.orchestra.run.smithi061.stdout: "get": 323, 2023-10-18T09:35:51.047 INFO:teuthology.orchestra.run.smithi061.stdout: "get_sum": 456480, 2023-10-18T09:35:51.049 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:51.049 INFO:teuthology.orchestra.run.smithi061.stdout: "get_or_fail_success": 323, 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "take": 0, 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "take_sum": 0, 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "put": 323, 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "put_sum": 456480, 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "wait": { 2023-10-18T09:35:51.050 INFO:teuthology.orchestra.run.smithi061.stdout: "avgcount": 0, 2023-10-18T09:35:51.051 INFO:teuthology.orchestra.run.smithi061.stdout: "sum": 0.000000000, 2023-10-18T09:35:51.051 INFO:teuthology.orchestra.run.smithi061.stdout: "avgtime": 0.000000000 2023-10-18T09:35:51.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:51.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-10-18T09:35:51.051 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-10-18T09:35:51.289 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:51 smithi061 ceph-mon[98573]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-10-18T09:35:51.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:51 smithi110 ceph-mon[134818]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-10-18T09:35:51.582 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-18T09:35:51.582 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-18T09:35:51.583 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-10-18T09:35:51.583 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-10-18T09:35:51.583 DEBUG:tasks.check_counter:Getting stats from b 2023-10-18T09:35:51.583 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-18T09:35:51.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:51 smithi170 ceph-mon[135150]: pgmap v876: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2023-10-18T09:35:52.234 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:35:53.332 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:53 smithi110 ceph-mon[134818]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-18T09:35:53.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:53 smithi061 ceph-mon[98573]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-18T09:35:53.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:53 smithi170 ceph-mon[135150]: pgmap v877: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2023-10-18T09:35:53.766 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2023-10-18T09:35:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:35:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4301, 2023-10-18T09:35:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 5321, 2023-10-18T09:35:53.767 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1360765, 2023-10-18T09:35:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 9605296, 2023-10-18T09:35:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 14, 2023-10-18T09:35:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2023-10-18T09:35:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.428876438, 2023-10-18T09:35:53.768 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.107971151, 2023-10-18T09:35:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.118116133, 2023-10-18T09:35:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.194880871, 2023-10-18T09:35:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 5317, 2023-10-18T09:35:53.769 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2.657379449, 2023-10-18T09:35:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000499789 2023-10-18T09:35:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 5119, 2023-10-18T09:35:53.770 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001072159, 2023-10-18T09:35:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000209 2023-10-18T09:35:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T09:35:53.771 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4898483, 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4900675, 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 775557960, 2023-10-18T09:35:53.772 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 796633014, 2023-10-18T09:35:53.773 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2023-10-18T09:35:53.773 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2023-10-18T09:35:53.773 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 185.496221242, 2023-10-18T09:35:53.773 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 80.234720119, 2023-10-18T09:35:53.773 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 130.894184787, 2023-10-18T09:35:53.774 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.135097534, 2023-10-18T09:35:53.774 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:53.774 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4900678, 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 90.456607841, 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000018457 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9795758, 2023-10-18T09:35:53.775 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.931360800, 2023-10-18T09:35:53.776 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000095 2023-10-18T09:35:53.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.776 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 681104, 2023-10-18T09:35:53.776 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 201648 2023-10-18T09:35:53.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:35:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 7909, 2023-10-18T09:35:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 10884, 2023-10-18T09:35:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2519608, 2023-10-18T09:35:53.777 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 6539984, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 102, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 7, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.808652939, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.190066774, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.288193671, 2023-10-18T09:35:53.778 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.360542638, 2023-10-18T09:35:53.779 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:35:53.779 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 10881, 2023-10-18T09:35:53.779 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.670127681, 2023-10-18T09:35:53.779 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000153490 2023-10-18T09:35:53.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:35:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 11364, 2023-10-18T09:35:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002552978, 2023-10-18T09:35:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000224 2023-10-18T09:35:53.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.781 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 57520, 2023-10-18T09:35:53.781 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 539200 2023-10-18T09:35:53.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.781 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2023-10-18T09:35:53.781 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2023-10-18T09:35:53.782 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2023-10-18T09:35:53.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.782 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2023-10-18T09:35:53.782 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:35:53.782 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:35:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3280, 2023-10-18T09:35:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.966912595, 2023-10-18T09:35:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000599668 2023-10-18T09:35:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.784 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2023-10-18T09:35:53.784 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:35:53.784 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:35:53.784 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 135, 2023-10-18T09:35:53.784 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.005502552, 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000040759 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 2183, 2023-10-18T09:35:53.785 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 1968, 2023-10-18T09:35:53.786 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2023-10-18T09:35:53.786 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1968, 2023-10-18T09:35:53.786 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 7.412846079, 2023-10-18T09:35:53.786 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.003766690 2023-10-18T09:35:53.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.787 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2023-10-18T09:35:53.787 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 215, 2023-10-18T09:35:53.787 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2023-10-18T09:35:53.787 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2023-10-18T09:35:53.787 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 39, 2023-10-18T09:35:53.788 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 3, 2023-10-18T09:35:53.788 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2023-10-18T09:35:53.788 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 384, 2023-10-18T09:35:53.788 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 7, 2023-10-18T09:35:53.788 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 4, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 373, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 373, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 988, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 270, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 270, 2023-10-18T09:35:53.789 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 26, 2023-10-18T09:35:53.790 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 2577, 2023-10-18T09:35:53.790 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 1807, 2023-10-18T09:35:53.790 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 197, 2023-10-18T09:35:53.790 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 1, 2023-10-18T09:35:53.790 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:35:53.791 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2023-10-18T09:35:53.791 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 5, 2023-10-18T09:35:53.791 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 2, 2023-10-18T09:35:53.791 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2023-10-18T09:35:53.791 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 37, 2023-10-18T09:35:53.792 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 988, 2023-10-18T09:35:53.792 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 13, 2023-10-18T09:35:53.792 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 325, 2023-10-18T09:35:53.792 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2023-10-18T09:35:53.792 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 957, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1037, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1636383425, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 957, 2023-10-18T09:35:53.793 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2023-10-18T09:35:53.794 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:35:53.794 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:35:53.794 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 102, 2023-10-18T09:35:53.794 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 4, 2023-10-18T09:35:53.794 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 98, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 855, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 22, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 124, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 1205, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:35:53.795 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 1237, 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 1035, 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 28, 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 176 2023-10-18T09:35:53.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.797 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2023-10-18T09:35:53.797 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 191, 2023-10-18T09:35:53.797 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 7, 2023-10-18T09:35:53.797 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 4, 2023-10-18T09:35:53.797 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 11, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 866, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 28, 2023-10-18T09:35:53.798 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 27, 2023-10-18T09:35:53.799 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 1, 2023-10-18T09:35:53.799 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2023-10-18T09:35:53.799 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2023-10-18T09:35:53.799 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:35:53.799 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:35:53.800 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2023-10-18T09:35:53.800 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2023-10-18T09:35:53.800 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 4, 2023-10-18T09:35:53.800 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 37, 2023-10-18T09:35:53.800 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2023-10-18T09:35:53.801 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 3, 2023-10-18T09:35:53.801 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2023-10-18T09:35:53.801 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2023-10-18T09:35:53.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.801 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2023-10-18T09:35:53.802 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2023-10-18T09:35:53.802 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 3250, 2023-10-18T09:35:53.802 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2023-10-18T09:35:53.802 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2023-10-18T09:35:53.802 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 3250, 2023-10-18T09:35:53.803 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2023-10-18T09:35:53.803 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2023-10-18T09:35:53.803 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 6, 2023-10-18T09:35:53.803 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2023-10-18T09:35:53.803 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 6, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2023-10-18T09:35:53.804 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 13345961, 2023-10-18T09:35:53.805 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2023-10-18T09:35:53.805 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2023-10-18T09:35:53.805 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1572, 2023-10-18T09:35:53.805 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 14.527315222, 2023-10-18T09:35:53.805 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.009241294 2023-10-18T09:35:53.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.806 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2023-10-18T09:35:53.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.806 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2023-10-18T09:35:53.806 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 370, 2023-10-18T09:35:53.807 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 1351, 2023-10-18T09:35:53.807 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 981, 2023-10-18T09:35:53.807 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 63, 2023-10-18T09:35:53.807 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 80, 2023-10-18T09:35:53.807 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 17, 2023-10-18T09:35:53.808 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 384, 2023-10-18T09:35:53.808 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 1710, 2023-10-18T09:35:53.808 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 1326, 2023-10-18T09:35:53.808 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 270, 2023-10-18T09:35:53.808 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 1519, 2023-10-18T09:35:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 1249, 2023-10-18T09:35:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 65116, 2023-10-18T09:35:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 207152 2023-10-18T09:35:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.809 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2023-10-18T09:35:53.810 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 2310, 2023-10-18T09:35:53.810 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 78, 2023-10-18T09:35:53.810 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 2183, 2023-10-18T09:35:53.810 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 317, 2023-10-18T09:35:53.810 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 164, 2023-10-18T09:35:53.811 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2023-10-18T09:35:53.811 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 997, 2023-10-18T09:35:53.811 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 5.990660130, 2023-10-18T09:35:53.811 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.006008686 2023-10-18T09:35:53.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 80, 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.124061525, 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001550769 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.812 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2023-10-18T09:35:53.813 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.813 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.813 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.813 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2023-10-18T09:35:53.814 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2023-10-18T09:35:53.814 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002012670, 2023-10-18T09:35:53.814 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002012670 2023-10-18T09:35:53.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.814 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2023-10-18T09:35:53.815 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 253, 2023-10-18T09:35:53.815 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.182870944, 2023-10-18T09:35:53.815 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000722810 2023-10-18T09:35:53.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.815 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2023-10-18T09:35:53.816 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2023-10-18T09:35:53.817 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000167840, 2023-10-18T09:35:53.817 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000167840 2023-10-18T09:35:53.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.817 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2023-10-18T09:35:53.817 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.818 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.818 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.818 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2023-10-18T09:35:53.818 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.819 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.820 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.820 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2023-10-18T09:35:53.820 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.820 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2023-10-18T09:35:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 35, 2023-10-18T09:35:53.821 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.617816853, 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.017651910 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.822 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.823 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2023-10-18T09:35:53.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.823 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.823 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.824 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2023-10-18T09:35:53.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 493, 2023-10-18T09:35:53.824 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.224720884, 2023-10-18T09:35:53.824 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000455823 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 59, 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.122370890, 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002074082 2023-10-18T09:35:53.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.826 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2023-10-18T09:35:53.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 21, 2023-10-18T09:35:53.826 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.096289649, 2023-10-18T09:35:53.826 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004585221 2023-10-18T09:35:53.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.827 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2023-10-18T09:35:53.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.827 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.827 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.828 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2023-10-18T09:35:53.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.828 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.828 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.829 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2023-10-18T09:35:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.830 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2023-10-18T09:35:53.831 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001530470, 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000765235 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.832 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.833 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2023-10-18T09:35:53.833 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.833 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.834 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.834 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2023-10-18T09:35:53.834 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.834 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7, 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.018470361, 2023-10-18T09:35:53.835 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002638623 2023-10-18T09:35:53.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.836 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2023-10-18T09:35:53.836 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.836 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.836 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.837 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2023-10-18T09:35:53.837 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 19, 2023-10-18T09:35:53.837 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.031873863, 2023-10-18T09:35:53.837 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001677571 2023-10-18T09:35:53.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.838 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:35:53.838 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:35:53.838 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2023-10-18T09:35:53.838 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.839 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.839 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.839 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.839 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2023-10-18T09:35:53.840 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2023-10-18T09:35:53.840 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 3, 2023-10-18T09:35:53.840 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2023-10-18T09:35:53.840 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2023-10-18T09:35:53.840 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 69, 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 69, 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1032, 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.841 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2023-10-18T09:35:53.842 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:35:53.842 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2023-10-18T09:35:53.842 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:35:53.842 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:35:53.842 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:35:53.843 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:35:53.843 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:35:53.843 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:35:53.843 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:35:53.843 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:35:53.844 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:35:53.844 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:35:53.844 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:35:53.844 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:35:53.844 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:35:53.845 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:35:53.846 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:35:53.846 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:35:53.846 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:35:53.846 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:35:53.846 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2023-10-18T09:35:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:35:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:35:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:35:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2023-10-18T09:35:53.847 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:35:53.848 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 300901, 2023-10-18T09:35:53.849 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 449, 2023-10-18T09:35:53.849 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 264, 2023-10-18T09:35:53.849 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 3, 2023-10-18T09:35:53.849 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2023-10-18T09:35:53.849 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2023-10-18T09:35:53.850 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:35:53.850 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2023-10-18T09:35:53.850 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:35:53.850 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2023-10-18T09:35:53.850 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2023-10-18T09:35:53.851 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2023-10-18T09:35:53.851 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:35:53.851 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:35:53.851 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2023-10-18T09:35:53.851 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 1326505, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 14670, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2023-10-18T09:35:53.852 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2023-10-18T09:35:53.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.853 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2023-10-18T09:35:53.853 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2023-10-18T09:35:53.853 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2023-10-18T09:35:53.853 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 3327, 2023-10-18T09:35:53.854 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 9575957, 2023-10-18T09:35:53.854 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 37, 2023-10-18T09:35:53.854 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 3327, 2023-10-18T09:35:53.854 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2023-10-18T09:35:53.854 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3327, 2023-10-18T09:35:53.855 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 39.068354336, 2023-10-18T09:35:53.855 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.011742817 2023-10-18T09:35:53.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.855 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2023-10-18T09:35:53.855 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2023-10-18T09:35:53.856 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3327, 2023-10-18T09:35:53.856 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 3422 2023-10-18T09:35:53.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.856 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 3327, 2023-10-18T09:35:53.856 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 978, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 2349, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 11, 2023-10-18T09:35:53.857 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 17, 2023-10-18T09:35:53.858 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 1585, 2023-10-18T09:35:53.858 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 86, 2023-10-18T09:35:53.858 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2023-10-18T09:35:53.858 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2023-10-18T09:35:53.858 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 3, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 583, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2023-10-18T09:35:53.859 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 961, 2023-10-18T09:35:53.860 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 22, 2023-10-18T09:35:53.860 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:35:53.860 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2023-10-18T09:35:53.860 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:35:53.860 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2023-10-18T09:35:53.861 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2023-10-18T09:35:53.861 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2023-10-18T09:35:53.861 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:35:53.861 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2023-10-18T09:35:53.861 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:35:53.862 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2023-10-18T09:35:53.862 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2023-10-18T09:35:53.862 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2023-10-18T09:35:53.862 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2023-10-18T09:35:53.862 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2023-10-18T09:35:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2023-10-18T09:35:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2023-10-18T09:35:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2023-10-18T09:35:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2023-10-18T09:35:53.863 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2023-10-18T09:35:53.864 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2023-10-18T09:35:53.865 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2023-10-18T09:35:53.865 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 92, 2023-10-18T09:35:53.865 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2023-10-18T09:35:53.865 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 9, 2023-10-18T09:35:53.865 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 126, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2023-10-18T09:35:53.866 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 28 2023-10-18T09:35:53.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.867 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2023-10-18T09:35:53.867 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2023-10-18T09:35:53.867 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:35:53.867 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2023-10-18T09:35:53.868 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2023-10-18T09:35:53.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.868 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2023-10-18T09:35:53.868 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2023-10-18T09:35:53.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.869 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2023-10-18T09:35:53.869 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2023-10-18T09:35:53.869 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 17, 2023-10-18T09:35:53.869 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2023-10-18T09:35:53.869 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 5, 2023-10-18T09:35:53.870 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 27, 2023-10-18T09:35:53.870 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2023-10-18T09:35:53.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.870 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:35:53.870 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 68, 2023-10-18T09:35:53.871 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:35:53.871 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:35:53.871 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 4910691, 2023-10-18T09:35:53.871 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 337524286, 2023-10-18T09:35:53.871 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:53.872 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 4910691, 2023-10-18T09:35:53.872 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:35:53.872 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:35:53.872 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4910691, 2023-10-18T09:35:53.872 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 337524286, 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.874 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2023-10-18T09:35:53.874 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:35:53.874 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:35:53.874 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:35:53.874 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:35:53.875 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:35:53.875 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:53.875 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:53.875 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 3327, 2023-10-18T09:35:53.875 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 9578900, 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 3327, 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 9578900, 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.876 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.877 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2023-10-18T09:35:53.877 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:35:53.877 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:35:53.878 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 3327, 2023-10-18T09:35:53.879 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 3327, 2023-10-18T09:35:53.879 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 3327, 2023-10-18T09:35:53.879 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 3327, 2023-10-18T09:35:53.879 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:35:53.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.880 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.880 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.880 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 27, 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2943, 2023-10-18T09:35:53.881 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:53.882 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 27, 2023-10-18T09:35:53.882 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:35:53.882 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:35:53.882 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 13, 2023-10-18T09:35:53.882 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2943, 2023-10-18T09:35:53.883 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:35:53.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.883 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55ed512f26e0": { 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:35:53.884 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 3250, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 9151657, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 3250, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:35:53.885 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1572, 2023-10-18T09:35:53.886 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 9151657, 2023-10-18T09:35:53.886 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:35:53.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:35:53.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:35:53.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:35:53.887 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.887 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:35:53.887 INFO:teuthology.orchestra.run.smithi110.stdout:} 2023-10-18T09:35:54.314 INFO:tasks.check_counter:Daemon mds.b mds.exported=37 2023-10-18T09:35:54.315 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2023-10-18T09:35:54.315 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=191 2023-10-18T09:35:54.315 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=7 2023-10-18T09:35:54.315 DEBUG:tasks.check_counter:Getting stats from e 2023-10-18T09:35:54.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-18T09:35:55.004 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:35:55.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:55 smithi061 ceph-mon[98573]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:55.568 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:55 smithi110 ceph-mon[134818]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:55.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:55 smithi170 ceph-mon[135150]: pgmap v878: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:57.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:57 smithi110 ceph-mon[134818]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:57.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:57 smithi061 ceph-mon[98573]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:57.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:57 smithi170 ceph-mon[135150]: pgmap v879: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2023-10-18T09:35:59.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:35:59 smithi061 ceph-mon[98573]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-10-18T09:35:59.604 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:35:59 smithi110 ceph-mon[134818]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-10-18T09:35:59.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:35:59 smithi170 ceph-mon[135150]: pgmap v880: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 10 op/s 2023-10-18T09:36:01.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:01 smithi110 ceph-mon[134818]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:36:01.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:01 smithi061 ceph-mon[98573]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:36:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:01 smithi170 ceph-mon[135150]: pgmap v881: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1023 B/s wr, 11 op/s 2023-10-18T09:36:03.483 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:03 smithi110 ceph-mon[134818]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-18T09:36:03.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:03 smithi061 ceph-mon[98573]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-18T09:36:03.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:03 smithi170 ceph-mon[135150]: pgmap v882: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 9 op/s 2023-10-18T09:36:03.833 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2023-10-18T09:36:03.833 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:03.833 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4981012, 2023-10-18T09:36:03.834 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 4979866, 2023-10-18T09:36:03.834 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 807869231, 2023-10-18T09:36:03.834 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 788428123, 2023-10-18T09:36:03.834 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 11, 2023-10-18T09:36:03.834 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 188.960151193, 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 81.605159941, 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 135.557629987, 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.021948746, 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:03.835 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4979865, 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 71.994444682, 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000014457 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 9958636, 2023-10-18T09:36:03.836 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.065450928, 2023-10-18T09:36:03.837 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000106 2023-10-18T09:36:03.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.837 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:36:03.837 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:36:03.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1297, 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1765, 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 474427, 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 1387511, 2023-10-18T09:36:03.838 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 15, 2023-10-18T09:36:03.839 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2023-10-18T09:36:03.839 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.094775158, 2023-10-18T09:36:03.839 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.038621327, 2023-10-18T09:36:03.839 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.057004210, 2023-10-18T09:36:03.839 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006495265, 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1763, 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.289658407, 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000164298 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.840 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1359, 2023-10-18T09:36:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000506823, 2023-10-18T09:36:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000372 2023-10-18T09:36:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.841 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 497248 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 845, 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 827, 2023-10-18T09:36:03.842 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 874199, 2023-10-18T09:36:03.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 501907, 2023-10-18T09:36:03.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 14, 2023-10-18T09:36:03.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2023-10-18T09:36:03.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.073373353, 2023-10-18T09:36:03.843 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.025734441, 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.036432168, 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.013476790, 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 826, 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.156866311, 2023-10-18T09:36:03.844 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000189910 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 399, 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000076119, 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000190 2023-10-18T09:36:03.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 650832, 2023-10-18T09:36:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 201344 2023-10-18T09:36:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2023-10-18T09:36:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2023-10-18T09:36:03.847 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2023-10-18T09:36:03.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.847 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2023-10-18T09:36:03.847 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:03.847 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1141, 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.216750956, 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000189965 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.848 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 16, 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000272005, 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000017000 2023-10-18T09:36:03.849 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.850 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2023-10-18T09:36:03.850 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 247, 2023-10-18T09:36:03.850 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 228, 2023-10-18T09:36:03.850 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 228, 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.121027877, 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000530824 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2023-10-18T09:36:03.851 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 19, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2023-10-18T09:36:03.852 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 338, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 4, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 334, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 327, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 229, 2023-10-18T09:36:03.853 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 229, 2023-10-18T09:36:03.854 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 18, 2023-10-18T09:36:03.854 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 693, 2023-10-18T09:36:03.854 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 633, 2023-10-18T09:36:03.854 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 19, 2023-10-18T09:36:03.854 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 21, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2023-10-18T09:36:03.855 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 8, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 301, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 669, 2023-10-18T09:36:03.856 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1037, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1636383425, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 669, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:03.857 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:03.858 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:03.858 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:36:03.858 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:03.858 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:36:03.858 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 669, 2023-10-18T09:36:03.859 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:03.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 3, 2023-10-18T09:36:03.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 24, 2023-10-18T09:36:03.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:03.859 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 260, 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 242, 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 7, 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 13 2023-10-18T09:36:03.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 53, 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 451, 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 235, 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 410, 2023-10-18T09:36:03.861 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 53, 2023-10-18T09:36:03.862 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2023-10-18T09:36:03.862 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:03.862 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:03.862 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 1, 2023-10-18T09:36:03.862 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 1, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:03.863 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:03.864 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:03.865 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2023-10-18T09:36:03.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.865 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2023-10-18T09:36:03.865 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2023-10-18T09:36:03.865 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 714, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 714, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2023-10-18T09:36:03.866 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 2, 2023-10-18T09:36:03.867 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2023-10-18T09:36:03.867 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2023-10-18T09:36:03.867 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2023-10-18T09:36:03.867 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2023-10-18T09:36:03.867 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2023-10-18T09:36:03.868 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2023-10-18T09:36:03.868 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2023-10-18T09:36:03.868 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 5258610, 2023-10-18T09:36:03.868 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2023-10-18T09:36:03.868 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 267, 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 8.488012372, 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.031790308 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2023-10-18T09:36:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2023-10-18T09:36:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 337, 2023-10-18T09:36:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 338, 2023-10-18T09:36:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 1, 2023-10-18T09:36:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 44, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 234, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 190, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 338, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 341, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 3, 2023-10-18T09:36:03.871 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 229, 2023-10-18T09:36:03.872 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 517, 2023-10-18T09:36:03.872 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 288, 2023-10-18T09:36:03.872 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 54444, 2023-10-18T09:36:03.872 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 198960 2023-10-18T09:36:03.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 262, 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 247, 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 15, 2023-10-18T09:36:03.873 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 5, 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 5, 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.011526910, 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002305382 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.874 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:03.875 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.876 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 196, 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.092964610, 2023-10-18T09:36:03.877 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000474309 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.879 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:03.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.881 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.882 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.882 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2023-10-18T09:36:03.882 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.882 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2023-10-18T09:36:03.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 18, 2023-10-18T09:36:03.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.007783184, 2023-10-18T09:36:03.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000432399 2023-10-18T09:36:03.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.886 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2023-10-18T09:36:03.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 8, 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.007857512, 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000982189 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.890 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:03.891 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.892 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.893 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.893 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:03.893 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.893 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.894 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.894 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:03.894 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.894 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.896 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2023-10-18T09:36:03.897 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000895661, 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000895661 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:03.898 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:03.899 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.899 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.899 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.899 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 1, 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 0, 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2023-10-18T09:36:03.900 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 25, 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 25, 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2940, 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.901 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2023-10-18T09:36:03.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:03.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:03.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:03.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:03.902 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:03.903 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:03.904 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:03.905 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:03.906 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:03.907 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 327312, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 449, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2023-10-18T09:36:03.908 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2023-10-18T09:36:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2023-10-18T09:36:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:03.909 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:03.910 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2023-10-18T09:36:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 1159645, 2023-10-18T09:36:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 12007, 2023-10-18T09:36:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2023-10-18T09:36:03.911 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2023-10-18T09:36:03.912 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 1052, 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 1102656, 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 33, 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 1052, 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1052, 2023-10-18T09:36:03.913 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 21.934652025, 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.020850429 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1052, 2023-10-18T09:36:03.914 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1089 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 1052, 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 670, 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 382, 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2023-10-18T09:36:03.915 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2023-10-18T09:36:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2023-10-18T09:36:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2023-10-18T09:36:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 1, 2023-10-18T09:36:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 268, 2023-10-18T09:36:03.916 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 63, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 14, 2023-10-18T09:36:03.917 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 669, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:03.918 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:03.919 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2023-10-18T09:36:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2023-10-18T09:36:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2023-10-18T09:36:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2023-10-18T09:36:03.920 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2023-10-18T09:36:03.921 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2023-10-18T09:36:03.922 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2023-10-18T09:36:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2023-10-18T09:36:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 92, 2023-10-18T09:36:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2023-10-18T09:36:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2023-10-18T09:36:03.923 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 34, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2023-10-18T09:36:03.924 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 10 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2023-10-18T09:36:03.925 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2023-10-18T09:36:03.926 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 2, 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 1, 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 1, 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:03.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 4983152, 2023-10-18T09:36:03.928 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 360721513, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 4983152, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 4983152, 2023-10-18T09:36:03.929 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 360721513, 2023-10-18T09:36:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:03.931 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1052, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1102765, 2023-10-18T09:36:03.932 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1052, 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1102765, 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:03.934 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1052, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1052, 2023-10-18T09:36:03.935 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1052, 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1052, 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.936 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:03.937 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 109, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:03.938 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1, 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 109, 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x5607530406e0": { 2023-10-18T09:36:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:03.940 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 714, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1064306, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 714, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:03.941 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 267, 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1064306, 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:03.942 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:03.943 INFO:teuthology.orchestra.run.smithi110.stdout:} 2023-10-18T09:36:04.305 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-18T09:36:04.305 INFO:tasks.check_counter:Daemon mds.e mds.imported=8 2023-10-18T09:36:04.305 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=53 2023-10-18T09:36:04.305 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=451 2023-10-18T09:36:04.306 DEBUG:tasks.check_counter:Getting stats from h 2023-10-18T09:36:04.306 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-18T09:36:05.034 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:36:05.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:05 smithi110 ceph-mon[134818]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-18T09:36:05.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:05 smithi061 ceph-mon[98573]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-18T09:36:05.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:05 smithi170 ceph-mon[135150]: pgmap v883: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2023-10-18T09:36:06.434 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2023-10-18T09:36:06.434 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:06.434 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2448, 2023-10-18T09:36:06.435 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 3475, 2023-10-18T09:36:06.435 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 715163, 2023-10-18T09:36:06.435 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 4087471, 2023-10-18T09:36:06.435 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 20, 2023-10-18T09:36:06.435 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2023-10-18T09:36:06.436 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.167800920, 2023-10-18T09:36:06.436 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.062903571, 2023-10-18T09:36:06.436 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.087394470, 2023-10-18T09:36:06.436 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.026664766, 2023-10-18T09:36:06.436 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:06.437 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3471, 2023-10-18T09:36:06.437 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.546172177, 2023-10-18T09:36:06.437 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000157352 2023-10-18T09:36:06.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.437 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:06.438 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1840, 2023-10-18T09:36:06.438 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000866918, 2023-10-18T09:36:06.438 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000471 2023-10-18T09:36:06.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.438 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1920, 2023-10-18T09:36:06.439 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 1568 2023-10-18T09:36:06.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.439 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:06.439 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1126, 2023-10-18T09:36:06.439 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1098, 2023-10-18T09:36:06.440 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 948828, 2023-10-18T09:36:06.440 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 593817, 2023-10-18T09:36:06.440 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 9, 2023-10-18T09:36:06.440 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 7, 2023-10-18T09:36:06.440 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.097751750, 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.031511394, 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.043126365, 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.023025335, 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1098, 2023-10-18T09:36:06.441 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.163959128, 2023-10-18T09:36:06.442 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000149325 2023-10-18T09:36:06.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.442 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:06.442 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 388, 2023-10-18T09:36:06.442 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000080076, 2023-10-18T09:36:06.443 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000206 2023-10-18T09:36:06.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 669072, 2023-10-18T09:36:06.443 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 205360 2023-10-18T09:36:06.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4361, 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 3518, 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1854227, 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 2019309, 2023-10-18T09:36:06.444 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 13, 2023-10-18T09:36:06.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 7, 2023-10-18T09:36:06.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.270800174, 2023-10-18T09:36:06.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.113070593, 2023-10-18T09:36:06.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.185229301, 2023-10-18T09:36:06.445 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.012387636, 2023-10-18T09:36:06.446 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:06.446 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3518, 2023-10-18T09:36:06.446 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.535501924, 2023-10-18T09:36:06.446 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000152217 2023-10-18T09:36:06.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 6816, 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001423307, 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000208 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.447 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T09:36:06.448 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 584736 2023-10-18T09:36:06.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.448 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2023-10-18T09:36:06.448 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2023-10-18T09:36:06.448 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1183, 2023-10-18T09:36:06.449 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.582685507, 2023-10-18T09:36:06.450 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000492549 2023-10-18T09:36:06.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.450 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:06.450 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:06.451 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:06.451 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 729, 2023-10-18T09:36:06.451 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.037740925, 2023-10-18T09:36:06.451 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000051770 2023-10-18T09:36:06.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.452 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2023-10-18T09:36:06.452 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 930, 2023-10-18T09:36:06.452 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 843, 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 843, 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 4.334577039, 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005141847 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.453 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2023-10-18T09:36:06.454 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 87, 2023-10-18T09:36:06.454 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:06.454 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:06.454 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2023-10-18T09:36:06.454 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2023-10-18T09:36:06.455 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2023-10-18T09:36:06.455 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 152, 2023-10-18T09:36:06.455 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 2, 2023-10-18T09:36:06.455 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 132, 2023-10-18T09:36:06.456 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 18, 2023-10-18T09:36:06.456 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 149, 2023-10-18T09:36:06.456 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 212, 2023-10-18T09:36:06.456 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 10, 2023-10-18T09:36:06.456 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 10, 2023-10-18T09:36:06.457 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 16, 2023-10-18T09:36:06.457 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 1798, 2023-10-18T09:36:06.457 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 1491, 2023-10-18T09:36:06.457 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 85, 2023-10-18T09:36:06.457 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 32, 2023-10-18T09:36:06.458 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:06.458 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:06.458 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 78, 2023-10-18T09:36:06.458 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 133, 2023-10-18T09:36:06.458 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 7, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 198, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 15, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 313, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:06.459 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 550, 2023-10-18T09:36:06.460 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:06.460 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 1037, 2023-10-18T09:36:06.460 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 1636383425, 2023-10-18T09:36:06.460 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2023-10-18T09:36:06.460 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 550, 2023-10-18T09:36:06.461 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:06.461 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:06.461 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:06.461 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 6, 2023-10-18T09:36:06.461 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 6, 2023-10-18T09:36:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 544, 2023-10-18T09:36:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 23, 2023-10-18T09:36:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 552, 2023-10-18T09:36:06.463 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:06.463 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:06.463 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:06.463 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 413, 2023-10-18T09:36:06.463 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 248, 2023-10-18T09:36:06.464 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 115, 2023-10-18T09:36:06.464 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 868 2023-10-18T09:36:06.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.464 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2023-10-18T09:36:06.464 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 140, 2023-10-18T09:36:06.465 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 399, 2023-10-18T09:36:06.465 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 212, 2023-10-18T09:36:06.465 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 355, 2023-10-18T09:36:06.465 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 148, 2023-10-18T09:36:06.465 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 211, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 211, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:06.466 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2023-10-18T09:36:06.467 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:06.467 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:06.467 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:06.467 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2023-10-18T09:36:06.467 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2023-10-18T09:36:06.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:06.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 7, 2023-10-18T09:36:06.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2023-10-18T09:36:06.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:06.468 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 1410, 2023-10-18T09:36:06.469 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2023-10-18T09:36:06.470 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2023-10-18T09:36:06.470 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 1410, 2023-10-18T09:36:06.470 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2023-10-18T09:36:06.470 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2023-10-18T09:36:06.470 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 3, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 3, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2023-10-18T09:36:06.471 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2023-10-18T09:36:06.472 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2023-10-18T09:36:06.472 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 6611355, 2023-10-18T09:36:06.472 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2023-10-18T09:36:06.472 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2023-10-18T09:36:06.472 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 281, 2023-10-18T09:36:06.473 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2.316485849, 2023-10-18T09:36:06.473 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.008243721 2023-10-18T09:36:06.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.473 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 150, 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 549, 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 399, 2023-10-18T09:36:06.474 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 38, 2023-10-18T09:36:06.475 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 214, 2023-10-18T09:36:06.475 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 176, 2023-10-18T09:36:06.475 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 152, 2023-10-18T09:36:06.475 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 575, 2023-10-18T09:36:06.475 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 423, 2023-10-18T09:36:06.476 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 10, 2023-10-18T09:36:06.476 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 521, 2023-10-18T09:36:06.476 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 511, 2023-10-18T09:36:06.476 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 55736, 2023-10-18T09:36:06.476 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 198960 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 1090, 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 8, 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 930, 2023-10-18T09:36:06.477 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 29, 2023-10-18T09:36:06.478 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 35, 2023-10-18T09:36:06.478 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2023-10-18T09:36:06.478 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 209, 2023-10-18T09:36:06.478 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.285953305, 2023-10-18T09:36:06.478 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001368197 2023-10-18T09:36:06.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.479 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2023-10-18T09:36:06.479 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2023-10-18T09:36:06.479 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.004524660, 2023-10-18T09:36:06.479 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002262330 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.481 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2023-10-18T09:36:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 273, 2023-10-18T09:36:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 3.513954934, 2023-10-18T09:36:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.012871629 2023-10-18T09:36:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.482 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.483 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2023-10-18T09:36:06.484 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.485 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.486 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2023-10-18T09:36:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.487 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2023-10-18T09:36:06.488 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.489 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2023-10-18T09:36:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.490 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2023-10-18T09:36:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 143, 2023-10-18T09:36:06.491 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.064640379, 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000452030 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3, 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.001240291, 2023-10-18T09:36:06.492 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000413430 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 209, 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.456347731, 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002183481 2023-10-18T09:36:06.493 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.494 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.495 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2023-10-18T09:36:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.497 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.498 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2023-10-18T09:36:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.499 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.500 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.501 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2023-10-18T09:36:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.502 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2023-10-18T09:36:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2023-10-18T09:36:06.504 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4, 2023-10-18T09:36:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.007915739, 2023-10-18T09:36:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.001978934 2023-10-18T09:36:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:06.505 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2023-10-18T09:36:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2023-10-18T09:36:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2023-10-18T09:36:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2023-10-18T09:36:06.507 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2023-10-18T09:36:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2023-10-18T09:36:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 70, 2023-10-18T09:36:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 70, 2023-10-18T09:36:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 1046, 2023-10-18T09:36:06.508 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:06.509 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:06.510 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:06.511 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:06.512 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:06.513 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:06.515 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 306785, 2023-10-18T09:36:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 263, 2023-10-18T09:36:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 616, 2023-10-18T09:36:06.516 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 7, 2023-10-18T09:36:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2023-10-18T09:36:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2023-10-18T09:36:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:06.517 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2023-10-18T09:36:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:06.518 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 508758, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 5231, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:06.519 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2023-10-18T09:36:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2023-10-18T09:36:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2023-10-18T09:36:06.520 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 1661, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 2510055, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 33, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 1661, 2023-10-18T09:36:06.521 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2023-10-18T09:36:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1661, 2023-10-18T09:36:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 25.270587728, 2023-10-18T09:36:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.015214080 2023-10-18T09:36:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.522 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1661, 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1784 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 1661, 2023-10-18T09:36:06.523 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 634, 2023-10-18T09:36:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 1027, 2023-10-18T09:36:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2023-10-18T09:36:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2023-10-18T09:36:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2023-10-18T09:36:06.524 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 84, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 365, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 123, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2023-10-18T09:36:06.525 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2023-10-18T09:36:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2023-10-18T09:36:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 434, 2023-10-18T09:36:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2023-10-18T09:36:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:06.526 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 550, 2023-10-18T09:36:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2023-10-18T09:36:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:06.527 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2023-10-18T09:36:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2023-10-18T09:36:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2023-10-18T09:36:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:06.528 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2023-10-18T09:36:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2023-10-18T09:36:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2023-10-18T09:36:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2023-10-18T09:36:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2023-10-18T09:36:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2023-10-18T09:36:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2023-10-18T09:36:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2023-10-18T09:36:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2023-10-18T09:36:06.530 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2023-10-18T09:36:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2023-10-18T09:36:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2023-10-18T09:36:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2023-10-18T09:36:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2023-10-18T09:36:06.531 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 92, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2023-10-18T09:36:06.532 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2023-10-18T09:36:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 12, 2023-10-18T09:36:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 12, 2023-10-18T09:36:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2023-10-18T09:36:06.533 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 147, 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 51 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2023-10-18T09:36:06.534 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2023-10-18T09:36:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2023-10-18T09:36:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2023-10-18T09:36:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.535 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2023-10-18T09:36:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2023-10-18T09:36:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2023-10-18T09:36:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:06.536 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 10, 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 5, 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 211, 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.537 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:06.538 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 7931, 2023-10-18T09:36:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2805580, 2023-10-18T09:36:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 7931, 2023-10-18T09:36:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:06.539 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7931, 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2805580, 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.540 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:06.541 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1661, 2023-10-18T09:36:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2533054, 2023-10-18T09:36:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1661, 2023-10-18T09:36:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2533054, 2023-10-18T09:36:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:06.543 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:06.544 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2023-10-18T09:36:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:06.545 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 1661, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 1661, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 1661, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1661, 2023-10-18T09:36:06.546 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:06.548 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 211, 2023-10-18T09:36:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 22999, 2023-10-18T09:36:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 211, 2023-10-18T09:36:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:06.549 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 84, 2023-10-18T09:36:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 22999, 2023-10-18T09:36:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.550 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x55d1199a06e0": { 2023-10-18T09:36:06.551 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 1410, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2417051, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:06.552 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 1410, 2023-10-18T09:36:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 281, 2023-10-18T09:36:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2417051, 2023-10-18T09:36:06.553 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:06.555 INFO:teuthology.orchestra.run.smithi110.stdout:} 2023-10-18T09:36:06.899 INFO:tasks.check_counter:Daemon mds.h mds.exported=7 2023-10-18T09:36:06.900 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2023-10-18T09:36:06.900 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=140 2023-10-18T09:36:06.900 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=399 2023-10-18T09:36:06.901 DEBUG:tasks.check_counter:Getting stats from k 2023-10-18T09:36:06.901 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-18T09:36:07.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:07 smithi110 ceph-mon[134818]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:07.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:07 smithi061 ceph-mon[98573]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:07.515 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-18T09:36:07.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:07 smithi170 ceph-mon[135150]: pgmap v884: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:08.939 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2023-10-18T09:36:08.939 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:08.940 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 354, 2023-10-18T09:36:08.940 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 333, 2023-10-18T09:36:08.940 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 637474, 2023-10-18T09:36:08.940 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 196557, 2023-10-18T09:36:08.940 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2023-10-18T09:36:08.941 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2023-10-18T09:36:08.941 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.043585565, 2023-10-18T09:36:08.941 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.015105664, 2023-10-18T09:36:08.941 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.018945746, 2023-10-18T09:36:08.941 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.008354207, 2023-10-18T09:36:08.942 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:08.942 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 332, 2023-10-18T09:36:08.942 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.042530112, 2023-10-18T09:36:08.942 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000128102 2023-10-18T09:36:08.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.943 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 636640, 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 196240 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1053, 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1298, 2023-10-18T09:36:08.944 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 258974, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 820313, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.095315614, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.026289868, 2023-10-18T09:36:08.945 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.020160560, 2023-10-18T09:36:08.946 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.040660505, 2023-10-18T09:36:08.946 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:08.946 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1296, 2023-10-18T09:36:08.946 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.364732886, 2023-10-18T09:36:08.946 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000281429 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 493136 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2438, 2023-10-18T09:36:08.948 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2438, 2023-10-18T09:36:08.949 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 3138898, 2023-10-18T09:36:08.949 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 757480, 2023-10-18T09:36:08.949 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2023-10-18T09:36:08.949 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2023-10-18T09:36:08.949 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.198345874, 2023-10-18T09:36:08.950 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.055081728, 2023-10-18T09:36:08.950 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.050524222, 2023-10-18T09:36:08.950 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.077391131, 2023-10-18T09:36:08.950 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:08.950 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2437, 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.602260044, 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000247131 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.951 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.952 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2023-10-18T09:36:08.953 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2023-10-18T09:36:08.953 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2023-10-18T09:36:08.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.953 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2023-10-18T09:36:08.953 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 5884, 2023-10-18T09:36:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.150935181, 2023-10-18T09:36:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000025651 2023-10-18T09:36:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.955 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2023-10-18T09:36:08.956 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2023-10-18T09:36:08.958 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2023-10-18T09:36:08.958 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:08.958 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:08.958 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2023-10-18T09:36:08.958 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 351, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 0, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 351, 2023-10-18T09:36:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 351, 2023-10-18T09:36:08.960 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2023-10-18T09:36:08.960 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2023-10-18T09:36:08.960 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2023-10-18T09:36:08.960 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 10, 2023-10-18T09:36:08.960 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:08.961 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2023-10-18T09:36:08.962 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2023-10-18T09:36:08.962 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2023-10-18T09:36:08.962 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2023-10-18T09:36:08.962 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2023-10-18T09:36:08.962 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2023-10-18T09:36:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2023-10-18T09:36:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2023-10-18T09:36:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:08.963 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2023-10-18T09:36:08.964 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2023-10-18T09:36:08.964 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2023-10-18T09:36:08.964 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T09:36:08.964 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:08.964 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2023-10-18T09:36:08.965 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:08.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T09:36:08.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2023-10-18T09:36:08.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:08.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:08.966 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.967 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2023-10-18T09:36:08.968 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:08.969 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2023-10-18T09:36:08.970 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:08.970 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:08.970 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:08.970 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2023-10-18T09:36:08.970 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:08.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2023-10-18T09:36:08.972 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2023-10-18T09:36:08.973 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 1410, 2023-10-18T09:36:08.973 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2023-10-18T09:36:08.973 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2023-10-18T09:36:08.973 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2023-10-18T09:36:08.973 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2023-10-18T09:36:08.974 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2023-10-18T09:36:08.974 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 3, 2023-10-18T09:36:08.974 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2023-10-18T09:36:08.974 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2023-10-18T09:36:08.974 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 6611355, 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 6610781, 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.975 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.976 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.976 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 1410 2023-10-18T09:36:08.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.976 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 137, 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 531, 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 394, 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 22, 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 32, 2023-10-18T09:36:08.977 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 10, 2023-10-18T09:36:08.978 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 351, 2023-10-18T09:36:08.978 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 552, 2023-10-18T09:36:08.978 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 201, 2023-10-18T09:36:08.978 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 51372, 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 207152 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.979 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2023-10-18T09:36:08.980 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2023-10-18T09:36:08.980 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:08.980 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2023-10-18T09:36:08.980 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2023-10-18T09:36:08.980 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.981 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2023-10-18T09:36:08.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.982 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.982 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.982 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:08.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.983 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.983 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.983 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2023-10-18T09:36:08.984 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.985 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.985 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:08.985 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.986 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.986 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2023-10-18T09:36:08.986 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.987 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.988 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.989 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:08.989 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.989 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.989 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.990 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2023-10-18T09:36:08.990 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.990 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.990 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.992 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2023-10-18T09:36:08.992 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.992 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.992 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.993 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2023-10-18T09:36:08.993 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.993 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.993 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.994 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2023-10-18T09:36:08.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.994 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.994 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.995 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2023-10-18T09:36:08.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.995 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.995 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.996 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:08.997 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.997 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.997 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.997 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2023-10-18T09:36:08.998 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:08.999 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:08.999 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:08.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:08.999 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:08.999 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.000 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.001 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.001 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2023-10-18T09:36:09.001 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.001 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.002 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.002 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:09.002 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.002 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.003 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.004 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2023-10-18T09:36:09.004 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.004 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.004 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.006 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2023-10-18T09:36:09.006 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.006 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.006 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.007 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2023-10-18T09:36:09.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.007 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.007 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.008 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:09.008 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:09.008 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:09.008 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.008 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.009 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.009 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2023-10-18T09:36:09.009 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 1, 2023-10-18T09:36:09.010 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 2, 2023-10-18T09:36:09.010 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 1, 2023-10-18T09:36:09.010 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 1, 2023-10-18T09:36:09.010 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2023-10-18T09:36:09.010 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2941, 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2023-10-18T09:36:09.011 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:09.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:09.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:09.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:09.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:09.012 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:09.013 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:09.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:09.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:09.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:09.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:09.014 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:09.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:09.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:09.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:09.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:09.015 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:09.016 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:09.017 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:09.018 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:09.018 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:09.018 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 194903, 2023-10-18T09:36:09.018 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 180, 2023-10-18T09:36:09.019 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:36:09.019 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2023-10-18T09:36:09.019 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2023-10-18T09:36:09.019 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2023-10-18T09:36:09.019 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:09.020 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:09.020 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:09.020 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:09.020 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:09.020 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 170, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 585522, 2023-10-18T09:36:09.021 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 4764, 2023-10-18T09:36:09.022 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:09.022 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2023-10-18T09:36:09.022 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:09.022 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2023-10-18T09:36:09.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.023 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2023-10-18T09:36:09.023 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2023-10-18T09:36:09.023 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2023-10-18T09:36:09.023 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 3487, 2023-10-18T09:36:09.023 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 3487, 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3487, 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 18.665699337, 2023-10-18T09:36:09.024 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.005352939 2023-10-18T09:36:09.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.025 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2023-10-18T09:36:09.025 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2023-10-18T09:36:09.025 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 3487, 2023-10-18T09:36:09.025 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 3488 2023-10-18T09:36:09.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.026 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 3487, 2023-10-18T09:36:09.026 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 3487, 2023-10-18T09:36:09.026 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2023-10-18T09:36:09.026 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 2182, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 1303, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2023-10-18T09:36:09.027 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2023-10-18T09:36:09.028 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2023-10-18T09:36:09.028 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2023-10-18T09:36:09.028 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2023-10-18T09:36:09.028 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2023-10-18T09:36:09.028 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2023-10-18T09:36:09.029 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2023-10-18T09:36:09.029 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:09.029 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:09.029 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2023-10-18T09:36:09.029 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2023-10-18T09:36:09.030 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:09.030 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:09.030 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:09.030 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2023-10-18T09:36:09.030 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2023-10-18T09:36:09.031 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2023-10-18T09:36:09.031 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:09.031 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2023-10-18T09:36:09.031 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:09.031 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2023-10-18T09:36:09.032 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2023-10-18T09:36:09.032 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2023-10-18T09:36:09.032 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2023-10-18T09:36:09.032 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2023-10-18T09:36:09.032 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2023-10-18T09:36:09.033 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2023-10-18T09:36:09.034 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2023-10-18T09:36:09.034 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2023-10-18T09:36:09.034 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2023-10-18T09:36:09.034 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2023-10-18T09:36:09.034 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2023-10-18T09:36:09.035 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 92, 2023-10-18T09:36:09.035 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2023-10-18T09:36:09.035 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 4, 2023-10-18T09:36:09.035 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 5, 2023-10-18T09:36:09.035 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 6, 2023-10-18T09:36:09.036 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 1, 2023-10-18T09:36:09.036 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2023-10-18T09:36:09.036 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2023-10-18T09:36:09.036 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2023-10-18T09:36:09.036 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2023-10-18T09:36:09.037 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2023-10-18T09:36:09.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.038 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2023-10-18T09:36:09.038 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2023-10-18T09:36:09.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.038 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2023-10-18T09:36:09.039 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:09.039 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T09:36:09.039 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2023-10-18T09:36:09.039 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2023-10-18T09:36:09.039 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2023-10-18T09:36:09.040 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:09.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.040 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:09.040 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:09.040 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 3841, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 3668574, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 3841, 2023-10-18T09:36:09.041 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:09.042 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:09.042 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 3841, 2023-10-18T09:36:09.042 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 3668574, 2023-10-18T09:36:09.042 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:09.042 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:09.043 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:09.044 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:09.045 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 3487, 2023-10-18T09:36:09.045 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2417056, 2023-10-18T09:36:09.045 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 3487, 2023-10-18T09:36:09.045 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2417056, 2023-10-18T09:36:09.045 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:09.046 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.046 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.046 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.046 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 3487, 2023-10-18T09:36:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 3487, 2023-10-18T09:36:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 3487, 2023-10-18T09:36:09.049 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 3487, 2023-10-18T09:36:09.049 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:09.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.049 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.050 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.050 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:09.050 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:09.050 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:09.051 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:09.052 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:09.052 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2023-10-18T09:36:09.052 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2023-10-18T09:36:09.052 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:09.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.053 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.053 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.053 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2023-10-18T09:36:09.053 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x556d5c3a2960": { 2023-10-18T09:36:09.054 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2023-10-18T09:36:09.054 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2023-10-18T09:36:09.054 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2023-10-18T09:36:09.054 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2023-10-18T09:36:09.054 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2023-10-18T09:36:09.055 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2023-10-18T09:36:09.056 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2023-10-18T09:36:09.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2023-10-18T09:36:09.056 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2023-10-18T09:36:09.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2023-10-18T09:36:09.056 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.057 INFO:teuthology.orchestra.run.smithi110.stdout: } 2023-10-18T09:36:09.057 INFO:teuthology.orchestra.run.smithi110.stdout:} 2023-10-18T09:36:09.301 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:09 smithi110 ceph-mon[134818]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:09.378 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-10-18T09:36:09.378 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-10-18T09:36:09.379 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-10-18T09:36:09.379 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-10-18T09:36:09.379 DEBUG:tasks.check_counter:Getting stats from c 2023-10-18T09:36:09.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-18T09:36:09.460 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:09 smithi170 ceph-mon[135150]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:09.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:09 smithi061 ceph-mon[98573]: pgmap v885: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2023-10-18T09:36:10.001 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:36:11.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:11 smithi170 ceph-mon[135150]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:11.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:11 smithi110 ceph-mon[134818]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:11.470 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-18T09:36:11.470 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:11.470 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 3579, 2023-10-18T09:36:11.471 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 3579, 2023-10-18T09:36:11.471 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4527564, 2023-10-18T09:36:11.471 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 1112280, 2023-10-18T09:36:11.471 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2023-10-18T09:36:11.471 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 2, 2023-10-18T09:36:11.472 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.285392480, 2023-10-18T09:36:11.472 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.076055637, 2023-10-18T09:36:11.472 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.074190028, 2023-10-18T09:36:11.472 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.115723342, 2023-10-18T09:36:11.472 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:11.473 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3578, 2023-10-18T09:36:11.473 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.100428153, 2023-10-18T09:36:11.473 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000307553 2023-10-18T09:36:11.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.473 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:11.474 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.474 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.474 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.474 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:36:11.475 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:36:11.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.475 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:11.475 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4, 2023-10-18T09:36:11.475 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 247, 2023-10-18T09:36:11.476 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 876, 2023-10-18T09:36:11.476 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 581100, 2023-10-18T09:36:11.476 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2023-10-18T09:36:11.476 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2023-10-18T09:36:11.476 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.013331444, 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.009399526, 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.001780808, 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000065145, 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 245, 2023-10-18T09:36:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.076546227, 2023-10-18T09:36:11.478 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000312433 2023-10-18T09:36:11.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.478 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:11.478 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.478 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.479 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2023-10-18T09:36:11.479 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 580416 2023-10-18T09:36:11.479 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.480 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:11.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 359, 2023-10-18T09:36:11.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 331, 2023-10-18T09:36:11.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 664108, 2023-10-18T09:36:11.480 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 195303, 2023-10-18T09:36:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2023-10-18T09:36:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 2, 2023-10-18T09:36:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.048483816, 2023-10-18T09:36:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.017180776, 2023-10-18T09:36:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.021015912, 2023-10-18T09:36:11.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.009360098, 2023-10-18T09:36:11.482 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:11.482 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 330, 2023-10-18T09:36:11.482 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.094669487, 2023-10-18T09:36:11.482 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000286877 2023-10-18T09:36:11.483 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.483 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:11.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.483 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.483 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 663840, 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 194992 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-10-18T09:36:11.484 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-10-18T09:36:11.485 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-10-18T09:36:11.485 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.485 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-10-18T09:36:11.485 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:11.485 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:11.486 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5745, 2023-10-18T09:36:11.486 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.152770203, 2023-10-18T09:36:11.486 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000026591 2023-10-18T09:36:11.486 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.486 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.487 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.488 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.488 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.488 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.488 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-10-18T09:36:11.488 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2023-10-18T09:36:11.489 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2023-10-18T09:36:11.489 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-10-18T09:36:11.489 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.489 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.489 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-10-18T09:36:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2023-10-18T09:36:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:11.490 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 436, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2023-10-18T09:36:11.491 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 0, 2023-10-18T09:36:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 436, 2023-10-18T09:36:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 436, 2023-10-18T09:36:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2023-10-18T09:36:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2023-10-18T09:36:11.492 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2023-10-18T09:36:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 3, 2023-10-18T09:36:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2023-10-18T09:36:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2023-10-18T09:36:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2023-10-18T09:36:11.493 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2023-10-18T09:36:11.494 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2023-10-18T09:36:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2023-10-18T09:36:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2023-10-18T09:36:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2023-10-18T09:36:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:11.495 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2023-10-18T09:36:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2023-10-18T09:36:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2023-10-18T09:36:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2023-10-18T09:36:11.496 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T09:36:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:36:11.497 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2023-10-18T09:36:11.498 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2023-10-18T09:36:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2023-10-18T09:36:11.499 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2023-10-18T09:36:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2023-10-18T09:36:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-10-18T09:36:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2023-10-18T09:36:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2023-10-18T09:36:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2023-10-18T09:36:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2023-10-18T09:36:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2023-10-18T09:36:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2023-10-18T09:36:11.501 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2023-10-18T09:36:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2023-10-18T09:36:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:11.502 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-10-18T09:36:11.503 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2023-10-18T09:36:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-10-18T09:36:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:11.504 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-10-18T09:36:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-10-18T09:36:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-10-18T09:36:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2023-10-18T09:36:11.505 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-10-18T09:36:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-10-18T09:36:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 2043, 2023-10-18T09:36:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-10-18T09:36:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-10-18T09:36:11.506 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 3, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-10-18T09:36:11.507 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-10-18T09:36:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-10-18T09:36:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 7718069, 2023-10-18T09:36:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 7717495, 2023-10-18T09:36:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-10-18T09:36:11.508 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 2043 2023-10-18T09:36:11.509 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-10-18T09:36:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 76, 2023-10-18T09:36:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 212, 2023-10-18T09:36:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 136, 2023-10-18T09:36:11.510 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 12, 2023-10-18T09:36:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 23, 2023-10-18T09:36:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 11, 2023-10-18T09:36:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 436, 2023-10-18T09:36:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 576, 2023-10-18T09:36:11.511 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 140, 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 50884, 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 207152 2023-10-18T09:36:11.512 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-10-18T09:36:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2023-10-18T09:36:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2023-10-18T09:36:11.513 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2023-10-18T09:36:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2023-10-18T09:36:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-10-18T09:36:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.514 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-10-18T09:36:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.515 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.516 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-10-18T09:36:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.517 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-10-18T09:36:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.518 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.519 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-10-18T09:36:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.520 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.521 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.522 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.523 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-10-18T09:36:11.524 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-10-18T09:36:11.525 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-10-18T09:36:11.526 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-10-18T09:36:11.527 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.528 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-10-18T09:36:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.529 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.530 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.531 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-10-18T09:36:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.533 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.534 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-10-18T09:36:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.535 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.536 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.537 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-10-18T09:36:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.538 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-10-18T09:36:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.539 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-10-18T09:36:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.540 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.541 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:11.542 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.543 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-10-18T09:36:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2023-10-18T09:36:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2023-10-18T09:36:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2023-10-18T09:36:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2023-10-18T09:36:11.544 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 2943, 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.545 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-10-18T09:36:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:11.546 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:11.547 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:11.548 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:11.549 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:11.550 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:11.551 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:11.552 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 184646, 2023-10-18T09:36:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 117, 2023-10-18T09:36:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:36:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2023-10-18T09:36:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-10-18T09:36:11.553 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:11.554 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2023-10-18T09:36:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-10-18T09:36:11.555 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 507141, 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 16718, 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-10-18T09:36:11.556 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 3578, 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2023-10-18T09:36:11.557 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 3578, 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3578, 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 22.826480202, 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006379675 2023-10-18T09:36:11.558 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-10-18T09:36:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-10-18T09:36:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3578, 2023-10-18T09:36:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3579 2023-10-18T09:36:11.559 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 3578, 2023-10-18T09:36:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 3578, 2023-10-18T09:36:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2023-10-18T09:36:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-10-18T09:36:11.560 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-10-18T09:36:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 2193, 2023-10-18T09:36:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2023-10-18T09:36:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1383, 2023-10-18T09:36:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2023-10-18T09:36:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2023-10-18T09:36:11.562 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-10-18T09:36:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2023-10-18T09:36:11.563 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2023-10-18T09:36:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-10-18T09:36:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-10-18T09:36:11.566 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-10-18T09:36:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-10-18T09:36:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-10-18T09:36:11.567 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-10-18T09:36:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-10-18T09:36:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-10-18T09:36:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-10-18T09:36:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-10-18T09:36:11.568 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-10-18T09:36:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-10-18T09:36:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-10-18T09:36:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-10-18T09:36:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-10-18T09:36:11.569 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-10-18T09:36:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-10-18T09:36:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-10-18T09:36:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-10-18T09:36:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-10-18T09:36:11.570 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-10-18T09:36:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 92, 2023-10-18T09:36:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-10-18T09:36:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 4, 2023-10-18T09:36:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 4, 2023-10-18T09:36:11.571 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 6, 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 2, 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2023-10-18T09:36:11.572 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-10-18T09:36:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2023-10-18T09:36:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-10-18T09:36:11.573 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-10-18T09:36:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-10-18T09:36:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2023-10-18T09:36:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.574 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-10-18T09:36:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T09:36:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-10-18T09:36:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2023-10-18T09:36:11.575 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:11.576 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3938, 2023-10-18T09:36:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 4816808, 2023-10-18T09:36:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3938, 2023-10-18T09:36:11.577 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3938, 2023-10-18T09:36:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 4816808, 2023-10-18T09:36:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:11.578 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.579 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.579 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.579 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.579 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.579 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:11.580 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:11.580 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:11.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:11.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:11.580 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3578, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3523770, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3578, 2023-10-18T09:36:11.581 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3523770, 2023-10-18T09:36:11.582 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:11.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.582 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.582 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.583 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.583 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.583 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:11.583 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:11.583 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:11.584 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3578, 2023-10-18T09:36:11.585 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3578, 2023-10-18T09:36:11.585 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3578, 2023-10-18T09:36:11.585 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3578, 2023-10-18T09:36:11.585 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:11.585 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.586 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.586 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.586 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.586 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.586 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:11.588 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:11.588 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:11.588 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:11.588 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-10-18T09:36:11.588 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-10-18T09:36:11.589 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:11.589 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.589 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.589 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55cd22e84960": { 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:11.590 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:11.591 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:11.591 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:11.591 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:11.591 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:11.591 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:11.592 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:11.592 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-10-18T09:36:11.592 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-10-18T09:36:11.592 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:11.592 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:11.593 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:11.593 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:11.593 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.593 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:11.593 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-18T09:36:11.594 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:11 smithi061 ceph-mon[98573]: pgmap v886: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:11.948 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-18T09:36:11.949 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-10-18T09:36:11.949 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-10-18T09:36:11.949 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-10-18T09:36:11.950 DEBUG:tasks.check_counter:Getting stats from f 2023-10-18T09:36:11.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-18T09:36:12.513 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:36:13.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:13 smithi170 ceph-mon[135150]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:13.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:13 smithi110 ceph-mon[134818]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:13.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:13 smithi061 ceph-mon[98573]: pgmap v887: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:13.772 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-18T09:36:13.772 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:13.772 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 954, 2023-10-18T09:36:13.772 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 966, 2023-10-18T09:36:13.773 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 347526, 2023-10-18T09:36:13.773 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 1727571, 2023-10-18T09:36:13.773 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 14, 2023-10-18T09:36:13.773 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2023-10-18T09:36:13.773 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.082030922, 2023-10-18T09:36:13.774 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.025397068, 2023-10-18T09:36:13.774 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.031243418, 2023-10-18T09:36:13.774 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.022655700, 2023-10-18T09:36:13.774 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:13.774 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 965, 2023-10-18T09:36:13.775 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.300466953, 2023-10-18T09:36:13.775 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000311364 2023-10-18T09:36:13.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.775 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:13.775 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 506, 2023-10-18T09:36:13.776 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000128038, 2023-10-18T09:36:13.776 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000253 2023-10-18T09:36:13.776 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.776 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-10-18T09:36:13.776 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1645, 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1889, 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1028470, 2023-10-18T09:36:13.777 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 1030885, 2023-10-18T09:36:13.778 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 14, 2023-10-18T09:36:13.778 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2023-10-18T09:36:13.778 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.139331826, 2023-10-18T09:36:13.778 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.048659879, 2023-10-18T09:36:13.778 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.088991553, 2023-10-18T09:36:13.779 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.016424230, 2023-10-18T09:36:13.779 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:13.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1884, 2023-10-18T09:36:13.779 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.350415693, 2023-10-18T09:36:13.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000185995 2023-10-18T09:36:13.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.780 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:13.780 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 888, 2023-10-18T09:36:13.780 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000734374, 2023-10-18T09:36:13.780 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000826 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 659904, 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 203872 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:13.781 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4052, 2023-10-18T09:36:13.782 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 3258, 2023-10-18T09:36:13.782 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1676194, 2023-10-18T09:36:13.782 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 1675609, 2023-10-18T09:36:13.782 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 12, 2023-10-18T09:36:13.782 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2023-10-18T09:36:13.783 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.256578306, 2023-10-18T09:36:13.783 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.093459678, 2023-10-18T09:36:13.783 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.220063761, 2023-10-18T09:36:13.783 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.011742960, 2023-10-18T09:36:13.783 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3260, 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.844907610, 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000259174 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:13.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6380, 2023-10-18T09:36:13.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001434165, 2023-10-18T09:36:13.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000224 2023-10-18T09:36:13.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.785 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-18T09:36:13.785 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 500656 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.786 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-10-18T09:36:13.787 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:13.787 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:13.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1280, 2023-10-18T09:36:13.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.235058604, 2023-10-18T09:36:13.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000183639 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:13.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 77, 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.003077239, 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000039964 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-10-18T09:36:13.789 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 429, 2023-10-18T09:36:13.790 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 300, 2023-10-18T09:36:13.790 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-10-18T09:36:13.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 300, 2023-10-18T09:36:13.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.222538722, 2023-10-18T09:36:13.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000741795 2023-10-18T09:36:13.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.791 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-10-18T09:36:13.791 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 129, 2023-10-18T09:36:13.791 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:13.791 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 288, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 5, 2023-10-18T09:36:13.792 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 283, 2023-10-18T09:36:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2023-10-18T09:36:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 280, 2023-10-18T09:36:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2023-10-18T09:36:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 134, 2023-10-18T09:36:13.793 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 134, 2023-10-18T09:36:13.794 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 22, 2023-10-18T09:36:13.794 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 955, 2023-10-18T09:36:13.794 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 739, 2023-10-18T09:36:13.794 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 128, 2023-10-18T09:36:13.794 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 22, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 2007, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 2, 2023-10-18T09:36:13.795 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2023-10-18T09:36:13.796 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2023-10-18T09:36:13.796 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 9, 2023-10-18T09:36:13.796 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 245, 2023-10-18T09:36:13.796 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:13.796 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 659, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 1037, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 1636383425, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 659, 2023-10-18T09:36:13.797 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:13.798 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:13.798 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:13.798 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 2, 2023-10-18T09:36:13.798 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:13.798 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 2, 2023-10-18T09:36:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 657, 2023-10-18T09:36:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 2, 2023-10-18T09:36:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 3, 2023-10-18T09:36:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 106, 2023-10-18T09:36:13.799 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:13.800 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:13.800 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:13.800 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 399, 2023-10-18T09:36:13.800 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 340, 2023-10-18T09:36:13.800 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 21, 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 50 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 77, 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 422, 2023-10-18T09:36:13.801 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 221, 2023-10-18T09:36:13.802 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 384, 2023-10-18T09:36:13.802 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 85, 2023-10-18T09:36:13.802 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2023-10-18T09:36:13.802 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:13.802 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 10, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 10, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:13.803 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2023-10-18T09:36:13.804 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-10-18T09:36:13.805 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:13.805 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:13.805 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-10-18T09:36:13.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.805 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-10-18T09:36:13.806 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-10-18T09:36:13.806 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 998, 2023-10-18T09:36:13.806 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-10-18T09:36:13.806 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-10-18T09:36:13.806 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 998, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 2, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-10-18T09:36:13.807 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2023-10-18T09:36:13.808 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2023-10-18T09:36:13.808 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-10-18T09:36:13.808 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-10-18T09:36:13.808 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-10-18T09:36:13.808 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 5489548, 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 318, 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12.198239910, 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.038359245 2023-10-18T09:36:13.809 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.810 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2023-10-18T09:36:13.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.810 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-10-18T09:36:13.810 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 279, 2023-10-18T09:36:13.810 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 289, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 10, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 49, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 221, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 172, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 288, 2023-10-18T09:36:13.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 300, 2023-10-18T09:36:13.812 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 12, 2023-10-18T09:36:13.812 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 134, 2023-10-18T09:36:13.812 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 459, 2023-10-18T09:36:13.812 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 325, 2023-10-18T09:36:13.812 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 58000, 2023-10-18T09:36:13.813 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198960 2023-10-18T09:36:13.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.813 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-10-18T09:36:13.813 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 500, 2023-10-18T09:36:13.813 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:13.814 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 429, 2023-10-18T09:36:13.814 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 15, 2023-10-18T09:36:13.814 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 20, 2023-10-18T09:36:13.814 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-10-18T09:36:13.814 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15, 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.047929281, 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003195285 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.815 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.816 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.816 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:13.816 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.816 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.817 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.818 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-10-18T09:36:13.818 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 243, 2023-10-18T09:36:13.818 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.119357414, 2023-10-18T09:36:13.818 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000491182 2023-10-18T09:36:13.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.819 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:13.819 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.819 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.819 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.820 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-10-18T09:36:13.820 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2023-10-18T09:36:13.820 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000105531, 2023-10-18T09:36:13.820 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000105531 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.822 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:13.822 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.822 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.822 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.823 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:13.823 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.823 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.823 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.824 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-10-18T09:36:13.825 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.825 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.825 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.825 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-10-18T09:36:13.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.826 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.826 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-10-18T09:36:13.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14, 2023-10-18T09:36:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.005525606, 2023-10-18T09:36:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000394686 2023-10-18T09:36:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-10-18T09:36:13.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 17, 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.010382883, 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000610757 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.829 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.830 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:13.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.830 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.832 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-10-18T09:36:13.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.832 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.834 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:13.834 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.834 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.834 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.835 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:13.836 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.837 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.838 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.838 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-10-18T09:36:13.838 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.838 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.839 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-10-18T09:36:13.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.839 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 10, 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.039238007, 2023-10-18T09:36:13.840 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003923800 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.841 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.842 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.842 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.842 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.842 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-10-18T09:36:13.842 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 1, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 0, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 36, 2023-10-18T09:36:13.843 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 36, 2023-10-18T09:36:13.844 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1041, 2023-10-18T09:36:13.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:13.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.844 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-10-18T09:36:13.844 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:13.845 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:13.846 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:13.847 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:13.848 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:13.848 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:13.848 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:13.848 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:13.848 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:13.849 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:13.850 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:13.850 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:13.850 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-10-18T09:36:13.850 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:13.850 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 301849, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 387, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 528, 2023-10-18T09:36:13.851 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 6, 2023-10-18T09:36:13.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-10-18T09:36:13.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-10-18T09:36:13.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:13.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:13.852 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:13.853 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:13.854 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-10-18T09:36:13.854 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 966626, 2023-10-18T09:36:13.854 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 10317, 2023-10-18T09:36:13.854 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:13.854 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-10-18T09:36:13.855 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:13.855 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-10-18T09:36:13.855 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.855 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-10-18T09:36:13.855 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 1179, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 1340077, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 1179, 2023-10-18T09:36:13.856 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-10-18T09:36:13.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1179, 2023-10-18T09:36:13.857 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 17.605599709, 2023-10-18T09:36:13.857 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.014932654 2023-10-18T09:36:13.857 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.857 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-10-18T09:36:13.858 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-10-18T09:36:13.858 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1179, 2023-10-18T09:36:13.858 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1260 2023-10-18T09:36:13.858 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.858 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 1179, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 669, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 510, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2023-10-18T09:36:13.859 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2023-10-18T09:36:13.860 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 10, 2023-10-18T09:36:13.860 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 328, 2023-10-18T09:36:13.860 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 76, 2023-10-18T09:36:13.860 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-10-18T09:36:13.860 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 32, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:13.861 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:13.862 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 659, 2023-10-18T09:36:13.862 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2023-10-18T09:36:13.862 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:13.862 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:13.862 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-10-18T09:36:13.863 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-10-18T09:36:13.864 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-10-18T09:36:13.865 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-10-18T09:36:13.865 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-10-18T09:36:13.865 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-10-18T09:36:13.865 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-10-18T09:36:13.865 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-10-18T09:36:13.866 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-10-18T09:36:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 92, 2023-10-18T09:36:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-10-18T09:36:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 4, 2023-10-18T09:36:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2023-10-18T09:36:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 99, 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 26 2023-10-18T09:36:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.869 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-10-18T09:36:13.869 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2023-10-18T09:36:13.869 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:13.869 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-10-18T09:36:13.869 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-10-18T09:36:13.870 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:13.871 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 4, 2023-10-18T09:36:13.871 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-10-18T09:36:13.871 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 2, 2023-10-18T09:36:13.871 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 10, 2023-10-18T09:36:13.871 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:13.872 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 6647, 2023-10-18T09:36:13.873 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2446010, 2023-10-18T09:36:13.873 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:13.873 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 6647, 2023-10-18T09:36:13.873 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:13.873 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:13.874 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 6647, 2023-10-18T09:36:13.874 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2446010, 2023-10-18T09:36:13.874 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:13.874 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.874 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.875 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.875 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.875 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.875 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:13.875 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:13.876 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1179, 2023-10-18T09:36:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1341167, 2023-10-18T09:36:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1179, 2023-10-18T09:36:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1341167, 2023-10-18T09:36:13.877 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.878 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:13.879 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:13.880 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:13.880 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:13.880 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1179, 2023-10-18T09:36:13.880 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1179, 2023-10-18T09:36:13.880 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1179, 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1179, 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.882 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.882 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:13.882 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:13.882 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:13.882 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:13.883 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 10, 2023-10-18T09:36:13.883 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1090, 2023-10-18T09:36:13.883 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:13.883 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 10, 2023-10-18T09:36:13.883 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 10, 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1090, 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.884 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.885 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.885 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.885 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:13.885 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55844cd1e6e0": { 2023-10-18T09:36:13.885 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:13.886 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:13.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:13.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 998, 2023-10-18T09:36:13.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1295244, 2023-10-18T09:36:13.886 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 998, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 318, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1295244, 2023-10-18T09:36:13.887 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:13.888 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:13.888 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:13.888 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:13.888 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.888 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:13.889 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-18T09:36:14.173 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-10-18T09:36:14.174 INFO:tasks.check_counter:Daemon mds.f mds.imported=9 2023-10-18T09:36:14.174 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=77 2023-10-18T09:36:14.174 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=422 2023-10-18T09:36:14.174 DEBUG:tasks.check_counter:Getting stats from i 2023-10-18T09:36:14.175 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-18T09:36:14.738 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:36:15.259 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:15 smithi170 ceph-mon[135150]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-18T09:36:15.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:15 smithi110 ceph-mon[134818]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-18T09:36:15.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:15 smithi061 ceph-mon[98573]: pgmap v888: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2023-10-18T09:36:15.994 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-18T09:36:15.995 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:15.995 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 5416, 2023-10-18T09:36:15.995 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 4385, 2023-10-18T09:36:15.995 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2348833, 2023-10-18T09:36:15.996 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 5234540, 2023-10-18T09:36:15.996 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 12, 2023-10-18T09:36:15.996 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2023-10-18T09:36:15.996 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.379719612, 2023-10-18T09:36:15.996 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.130152476, 2023-10-18T09:36:15.997 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.232679638, 2023-10-18T09:36:15.997 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.054458221, 2023-10-18T09:36:15.997 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:15.997 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4386, 2023-10-18T09:36:15.997 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.944616259, 2023-10-18T09:36:15.998 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000215370 2023-10-18T09:36:15.998 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:15.998 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:15.998 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6036, 2023-10-18T09:36:15.998 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001414943, 2023-10-18T09:36:15.999 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000234 2023-10-18T09:36:15.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:15.999 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 670512, 2023-10-18T09:36:15.999 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 202032 2023-10-18T09:36:15.999 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4415, 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 5494, 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1332161, 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 3324231, 2023-10-18T09:36:16.000 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2023-10-18T09:36:16.001 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2023-10-18T09:36:16.001 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.243116259, 2023-10-18T09:36:16.001 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.105561012, 2023-10-18T09:36:16.001 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.190764258, 2023-10-18T09:36:16.001 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.001619407, 2023-10-18T09:36:16.002 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:16.002 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5491, 2023-10-18T09:36:16.002 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.608999371, 2023-10-18T09:36:16.002 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000110908 2023-10-18T09:36:16.002 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4860, 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001674556, 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000344 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.003 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 4224, 2023-10-18T09:36:16.004 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 589280 2023-10-18T09:36:16.004 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.004 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:16.004 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 720, 2023-10-18T09:36:16.004 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 720, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 267953, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 425674, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.060067798, 2023-10-18T09:36:16.005 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.021088316, 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.031196725, 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.010047092, 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 718, 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.175015884, 2023-10-18T09:36:16.006 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000243754 2023-10-18T09:36:16.007 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.007 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:16.007 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 767, 2023-10-18T09:36:16.007 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000158143, 2023-10-18T09:36:16.007 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000206 2023-10-18T09:36:16.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.008 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T09:36:16.008 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T09:36:16.008 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.008 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:16.009 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:16.010 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2851, 2023-10-18T09:36:16.010 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.116040768, 2023-10-18T09:36:16.010 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000391455 2023-10-18T09:36:16.010 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.010 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.011 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:16.011 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:16.011 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:16.011 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 16, 2023-10-18T09:36:16.012 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000316424, 2023-10-18T09:36:16.012 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000019776 2023-10-18T09:36:16.012 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.012 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.012 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-10-18T09:36:16.013 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 1861, 2023-10-18T09:36:16.013 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 1817, 2023-10-18T09:36:16.013 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-10-18T09:36:16.013 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1817, 2023-10-18T09:36:16.013 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.979147748, 2023-10-18T09:36:16.014 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002189954 2023-10-18T09:36:16.014 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.014 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-10-18T09:36:16.014 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 44, 2023-10-18T09:36:16.014 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2023-10-18T09:36:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2023-10-18T09:36:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2023-10-18T09:36:16.015 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 467, 2023-10-18T09:36:16.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 7, 2023-10-18T09:36:16.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 315, 2023-10-18T09:36:16.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 145, 2023-10-18T09:36:16.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 451, 2023-10-18T09:36:16.016 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 130, 2023-10-18T09:36:16.017 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 72, 2023-10-18T09:36:16.017 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 72, 2023-10-18T09:36:16.017 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 15, 2023-10-18T09:36:16.018 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 4702, 2023-10-18T09:36:16.018 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 3553, 2023-10-18T09:36:16.018 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 43, 2023-10-18T09:36:16.018 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 32, 2023-10-18T09:36:16.018 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:16.019 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:16.019 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 394, 2023-10-18T09:36:16.019 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 2041, 2023-10-18T09:36:16.019 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 2, 2023-10-18T09:36:16.019 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 9, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 133, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 8, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 135, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 278, 2023-10-18T09:36:16.020 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:16.021 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 1037, 2023-10-18T09:36:16.021 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 1636383425, 2023-10-18T09:36:16.021 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2023-10-18T09:36:16.021 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 278, 2023-10-18T09:36:16.021 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 5, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 5, 2023-10-18T09:36:16.022 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 273, 2023-10-18T09:36:16.023 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:16.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 414, 2023-10-18T09:36:16.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 974, 2023-10-18T09:36:16.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:16.023 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:16.024 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:16.024 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 1408, 2023-10-18T09:36:16.024 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 682, 2023-10-18T09:36:16.024 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 56, 2023-10-18T09:36:16.024 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 1128 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 309, 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 356, 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 193, 2023-10-18T09:36:16.025 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 308, 2023-10-18T09:36:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 316, 2023-10-18T09:36:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2023-10-18T09:36:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 263, 2023-10-18T09:36:16.027 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 1, 2023-10-18T09:36:16.027 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 262, 2023-10-18T09:36:16.027 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-10-18T09:36:16.027 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:16.027 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:16.028 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:16.028 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-10-18T09:36:16.028 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-10-18T09:36:16.028 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:16.028 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 9, 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.029 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-10-18T09:36:16.030 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-10-18T09:36:16.030 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 2043, 2023-10-18T09:36:16.030 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-10-18T09:36:16.030 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-10-18T09:36:16.030 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 2043, 2023-10-18T09:36:16.031 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-10-18T09:36:16.031 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-10-18T09:36:16.031 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 3, 2023-10-18T09:36:16.031 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-10-18T09:36:16.031 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 3, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-10-18T09:36:16.032 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 7718069, 2023-10-18T09:36:16.033 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2023-10-18T09:36:16.033 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-10-18T09:36:16.033 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1256, 2023-10-18T09:36:16.033 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7.285959625, 2023-10-18T09:36:16.033 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005800923 2023-10-18T09:36:16.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.034 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2023-10-18T09:36:16.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.034 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-10-18T09:36:16.034 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 108, 2023-10-18T09:36:16.035 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 233, 2023-10-18T09:36:16.035 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 125, 2023-10-18T09:36:16.035 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 42, 2023-10-18T09:36:16.035 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 187, 2023-10-18T09:36:16.035 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 145, 2023-10-18T09:36:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 467, 2023-10-18T09:36:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 598, 2023-10-18T09:36:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 131, 2023-10-18T09:36:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 72, 2023-10-18T09:36:16.036 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 376, 2023-10-18T09:36:16.037 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 304, 2023-10-18T09:36:16.037 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 55112, 2023-10-18T09:36:16.037 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 198960 2023-10-18T09:36:16.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.037 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-10-18T09:36:16.038 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 2850, 2023-10-18T09:36:16.038 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:16.038 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 1861, 2023-10-18T09:36:16.038 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 539, 2023-10-18T09:36:16.039 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 26, 2023-10-18T09:36:16.039 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-10-18T09:36:16.039 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 64, 2023-10-18T09:36:16.039 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.062711541, 2023-10-18T09:36:16.039 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000979867 2023-10-18T09:36:16.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.040 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-10-18T09:36:16.040 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2023-10-18T09:36:16.040 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000743960, 2023-10-18T09:36:16.040 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000371980 2023-10-18T09:36:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.042 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-10-18T09:36:16.042 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 262, 2023-10-18T09:36:16.042 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.523776194, 2023-10-18T09:36:16.042 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001999145 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 813, 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.994911823, 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001223753 2023-10-18T09:36:16.043 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.044 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-10-18T09:36:16.045 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.045 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.045 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.045 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.045 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-10-18T09:36:16.046 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.046 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.046 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.046 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.046 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:16.047 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-10-18T09:36:16.048 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.049 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.049 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.049 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.049 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-10-18T09:36:16.049 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.050 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.050 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.050 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.050 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-10-18T09:36:16.050 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.051 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.051 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.051 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.051 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-10-18T09:36:16.052 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.052 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.052 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.052 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.052 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-10-18T09:36:16.053 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 132, 2023-10-18T09:36:16.053 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.070628966, 2023-10-18T09:36:16.053 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000535067 2023-10-18T09:36:16.053 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.053 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-10-18T09:36:16.054 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 18, 2023-10-18T09:36:16.054 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.023806651, 2023-10-18T09:36:16.054 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001322591 2023-10-18T09:36:16.054 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.054 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-10-18T09:36:16.055 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 263, 2023-10-18T09:36:16.055 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.921359268, 2023-10-18T09:36:16.055 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007305548 2023-10-18T09:36:16.055 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.055 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:16.056 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.056 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.056 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.056 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.056 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-10-18T09:36:16.057 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.058 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.058 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.058 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.058 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:16.058 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.059 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.059 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.059 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.059 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:16.059 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.060 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.061 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.061 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.061 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:16.061 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.061 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.062 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.063 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.063 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-10-18T09:36:16.063 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.063 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.063 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.064 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-10-18T09:36:16.064 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.064 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.064 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.066 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-10-18T09:36:16.066 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 263, 2023-10-18T09:36:16.066 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.381209345, 2023-10-18T09:36:16.066 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001449465 2023-10-18T09:36:16.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.068 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-10-18T09:36:16.068 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2023-10-18T09:36:16.068 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2023-10-18T09:36:16.069 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2023-10-18T09:36:16.069 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2023-10-18T09:36:16.069 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-10-18T09:36:16.069 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 179, 2023-10-18T09:36:16.070 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 179, 2023-10-18T09:36:16.070 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1038, 2023-10-18T09:36:16.070 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:16.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.071 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-10-18T09:36:16.071 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:16.071 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:16.071 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:16.072 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:16.072 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:16.072 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:16.072 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:16.072 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:16.073 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:16.073 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:16.073 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:16.073 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:16.073 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:16.074 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:16.074 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:16.074 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:16.074 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:16.074 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:16.075 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:16.075 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:16.075 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:16.075 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:16.075 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:16.076 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:16.076 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:16.076 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:16.076 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:16.076 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:16.077 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:16.077 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:16.077 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:16.077 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-10-18T09:36:16.077 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:16.078 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:16.078 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:16.078 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:16.078 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 312812, 2023-10-18T09:36:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 215, 2023-10-18T09:36:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:36:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2023-10-18T09:36:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-10-18T09:36:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-10-18T09:36:16.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:16.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:16.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:16.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:16.080 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:16.081 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2023-10-18T09:36:16.081 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:16.081 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:16.081 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:16.081 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-10-18T09:36:16.082 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 699190, 2023-10-18T09:36:16.082 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 21674, 2023-10-18T09:36:16.082 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:16.083 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-10-18T09:36:16.083 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:16.083 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-10-18T09:36:16.083 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.083 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 1748, 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 3581745, 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 27, 2023-10-18T09:36:16.084 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 1748, 2023-10-18T09:36:16.085 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-10-18T09:36:16.085 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1748, 2023-10-18T09:36:16.085 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 11.481737136, 2023-10-18T09:36:16.085 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006568499 2023-10-18T09:36:16.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1748, 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1903 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.086 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 1748, 2023-10-18T09:36:16.087 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 279, 2023-10-18T09:36:16.087 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 1469, 2023-10-18T09:36:16.087 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-10-18T09:36:16.087 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-10-18T09:36:16.087 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 1257, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 70, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-10-18T09:36:16.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-10-18T09:36:16.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-10-18T09:36:16.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2023-10-18T09:36:16.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 14, 2023-10-18T09:36:16.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-10-18T09:36:16.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:16.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:16.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 278, 2023-10-18T09:36:16.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2023-10-18T09:36:16.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:16.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-10-18T09:36:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-10-18T09:36:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-10-18T09:36:16.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-10-18T09:36:16.092 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-10-18T09:36:16.093 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-10-18T09:36:16.093 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-10-18T09:36:16.093 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-10-18T09:36:16.093 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-10-18T09:36:16.093 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-10-18T09:36:16.094 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-10-18T09:36:16.094 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-10-18T09:36:16.094 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-10-18T09:36:16.094 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 92, 2023-10-18T09:36:16.095 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 4, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-10-18T09:36:16.096 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 175, 2023-10-18T09:36:16.097 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2023-10-18T09:36:16.097 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 78 2023-10-18T09:36:16.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.097 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-10-18T09:36:16.097 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-10-18T09:36:16.098 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 2, 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-10-18T09:36:16.099 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 1, 2023-10-18T09:36:16.100 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 1, 2023-10-18T09:36:16.100 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:16.100 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.100 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:16.100 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 10547, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 3039077, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:16.101 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 10547, 2023-10-18T09:36:16.102 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:16.102 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:16.102 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 10547, 2023-10-18T09:36:16.102 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3039077, 2023-10-18T09:36:16.102 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.103 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:16.104 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:16.104 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:16.104 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:16.104 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:16.104 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1748, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3581854, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1748, 2023-10-18T09:36:16.105 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3581854, 2023-10-18T09:36:16.106 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:16.106 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.106 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.106 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:16.107 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 1748, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1748, 2023-10-18T09:36:16.108 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1748, 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1748, 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.109 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.110 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.110 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:16.110 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:16.110 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 1, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 109, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 1, 2023-10-18T09:36:16.111 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1, 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 109, 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.112 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.113 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.113 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.113 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:16.113 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x564426e2c6e0": { 2023-10-18T09:36:16.113 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2043, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 3523765, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:16.114 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2043, 2023-10-18T09:36:16.115 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:16.115 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:16.115 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1256, 2023-10-18T09:36:16.115 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3523765, 2023-10-18T09:36:16.115 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:16.116 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-18T09:36:16.395 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2023-10-18T09:36:16.396 INFO:tasks.check_counter:Daemon mds.i mds.imported=8 2023-10-18T09:36:16.396 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=309 2023-10-18T09:36:16.396 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=356 2023-10-18T09:36:16.396 DEBUG:tasks.check_counter:Getting stats from l 2023-10-18T09:36:16.397 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f7420d7f8e55ceea786dcbf536c338ce920c21eb shell --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-18T09:36:16.973 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-18T09:36:17.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:17 smithi110 ceph-mon[134818]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:17.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:17 smithi061 ceph-mon[98573]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:17.541 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:17 smithi170 ceph-mon[135150]: pgmap v889: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:18.310 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-18T09:36:18.311 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2023-10-18T09:36:18.311 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1492, 2023-10-18T09:36:18.311 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1471, 2023-10-18T09:36:18.311 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1019152, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 551408, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.165780857, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.050018136, 2023-10-18T09:36:18.312 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.049971118, 2023-10-18T09:36:18.313 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.053693671, 2023-10-18T09:36:18.313 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:18.313 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1470, 2023-10-18T09:36:18.313 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.631671438, 2023-10-18T09:36:18.313 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000429708 2023-10-18T09:36:18.314 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.314 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:18.314 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.314 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.314 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 637456, 2023-10-18T09:36:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 198112 2023-10-18T09:36:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.315 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1036, 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1282, 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 257344, 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 903726, 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2023-10-18T09:36:18.316 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2023-10-18T09:36:18.317 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.108218279, 2023-10-18T09:36:18.317 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.036293231, 2023-10-18T09:36:18.317 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.022643974, 2023-10-18T09:36:18.317 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.040525754, 2023-10-18T09:36:18.317 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:18.318 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1281, 2023-10-18T09:36:18.318 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.490655372, 2023-10-18T09:36:18.318 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000383025 2023-10-18T09:36:18.318 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.318 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:18.319 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.319 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.319 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.319 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.319 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 583328 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1375, 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1377, 2023-10-18T09:36:18.320 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1651727, 2023-10-18T09:36:18.321 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 427626, 2023-10-18T09:36:18.321 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2023-10-18T09:36:18.321 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2023-10-18T09:36:18.321 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.123078078, 2023-10-18T09:36:18.321 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.037865176, 2023-10-18T09:36:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.032264351, 2023-10-18T09:36:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.043176032, 2023-10-18T09:36:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2023-10-18T09:36:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1375, 2023-10-18T09:36:18.322 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.460171154, 2023-10-18T09:36:18.323 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000334669 2023-10-18T09:36:18.323 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.323 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2023-10-18T09:36:18.323 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.323 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.324 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2023-10-18T09:36:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2023-10-18T09:36:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2023-10-18T09:36:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2023-10-18T09:36:18.325 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:18.326 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:18.326 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5971, 2023-10-18T09:36:18.326 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.162891221, 2023-10-18T09:36:18.326 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000027280 2023-10-18T09:36:18.326 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.327 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.328 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.328 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.328 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.328 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2023-10-18T09:36:18.328 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.329 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.330 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2023-10-18T09:36:18.330 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2023-10-18T09:36:18.330 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2023-10-18T09:36:18.330 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2023-10-18T09:36:18.330 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2023-10-18T09:36:18.331 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2023-10-18T09:36:18.331 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2023-10-18T09:36:18.331 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 281, 2023-10-18T09:36:18.331 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2023-10-18T09:36:18.331 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 0, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 281, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 281, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2023-10-18T09:36:18.332 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 13, 2023-10-18T09:36:18.333 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2023-10-18T09:36:18.333 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2023-10-18T09:36:18.333 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2023-10-18T09:36:18.333 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2023-10-18T09:36:18.333 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2023-10-18T09:36:18.334 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2023-10-18T09:36:18.334 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2023-10-18T09:36:18.334 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2023-10-18T09:36:18.334 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2023-10-18T09:36:18.334 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2023-10-18T09:36:18.335 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2023-10-18T09:36:18.336 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2023-10-18T09:36:18.336 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2023-10-18T09:36:18.336 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2023-10-18T09:36:18.336 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2023-10-18T09:36:18.336 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2023-10-18T09:36:18.337 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2023-10-18T09:36:18.337 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2023-10-18T09:36:18.337 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2023-10-18T09:36:18.337 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2023-10-18T09:36:18.337 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2023-10-18T09:36:18.338 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2023-10-18T09:36:18.338 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2023-10-18T09:36:18.338 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2023-10-18T09:36:18.338 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2023-10-18T09:36:18.338 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2023-10-18T09:36:18.339 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-18T09:36:18.339 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-18T09:36:18.339 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2023-10-18T09:36:18.339 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2023-10-18T09:36:18.339 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2023-10-18T09:36:18.340 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2023-10-18T09:36:18.341 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2023-10-18T09:36:18.341 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2023-10-18T09:36:18.341 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2023-10-18T09:36:18.341 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2023-10-18T09:36:18.341 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2023-10-18T09:36:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2023-10-18T09:36:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2023-10-18T09:36:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2023-10-18T09:36:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2023-10-18T09:36:18.342 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2023-10-18T09:36:18.343 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2023-10-18T09:36:18.344 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2023-10-18T09:36:18.344 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2023-10-18T09:36:18.344 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2023-10-18T09:36:18.344 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2023-10-18T09:36:18.344 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2023-10-18T09:36:18.345 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 998, 2023-10-18T09:36:18.346 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2023-10-18T09:36:18.346 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2023-10-18T09:36:18.346 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2023-10-18T09:36:18.346 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2023-10-18T09:36:18.346 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2023-10-18T09:36:18.347 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2023-10-18T09:36:18.347 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 2, 2023-10-18T09:36:18.347 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2023-10-18T09:36:18.347 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2023-10-18T09:36:18.347 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 5489548, 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 5488974, 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.348 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.349 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 998 2023-10-18T09:36:18.349 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.349 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2023-10-18T09:36:18.349 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 262, 2023-10-18T09:36:18.350 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 272, 2023-10-18T09:36:18.350 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 10, 2023-10-18T09:36:18.350 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 21, 2023-10-18T09:36:18.350 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 21, 2023-10-18T09:36:18.350 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 0, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 281, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 281, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 0, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2023-10-18T09:36:18.351 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 49148, 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 207152 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2023-10-18T09:36:18.352 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2023-10-18T09:36:18.353 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2023-10-18T09:36:18.353 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2023-10-18T09:36:18.353 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2023-10-18T09:36:18.353 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2023-10-18T09:36:18.353 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.354 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.354 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.354 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.354 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2023-10-18T09:36:18.354 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.355 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.355 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.355 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.355 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2023-10-18T09:36:18.355 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.356 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.356 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.356 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.356 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2023-10-18T09:36:18.356 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.357 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.358 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.358 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.358 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2023-10-18T09:36:18.358 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.358 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.359 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.360 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.360 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2023-10-18T09:36:18.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.360 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.360 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.361 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.361 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2023-10-18T09:36:18.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.361 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.361 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.362 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.362 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2023-10-18T09:36:18.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.362 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.362 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.363 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.364 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2023-10-18T09:36:18.364 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.364 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.364 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.364 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.365 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2023-10-18T09:36:18.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.365 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.365 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.365 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.366 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2023-10-18T09:36:18.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.367 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.367 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.367 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.367 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2023-10-18T09:36:18.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.368 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.368 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.368 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.368 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2023-10-18T09:36:18.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.369 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.369 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.369 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.369 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2023-10-18T09:36:18.370 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.370 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.370 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.370 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.370 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2023-10-18T09:36:18.371 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.372 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.372 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.372 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2023-10-18T09:36:18.372 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.373 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.373 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.373 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.373 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2023-10-18T09:36:18.373 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.374 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.375 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.375 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.375 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2023-10-18T09:36:18.375 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.375 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.376 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.377 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.377 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2023-10-18T09:36:18.377 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.377 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.377 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.378 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.378 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2023-10-18T09:36:18.378 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.378 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.378 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.379 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.379 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2023-10-18T09:36:18.379 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.379 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.379 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.380 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.380 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2023-10-18T09:36:18.380 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.380 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.380 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.381 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.381 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2023-10-18T09:36:18.381 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2023-10-18T09:36:18.381 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2023-10-18T09:36:18.381 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.382 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.382 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.382 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.382 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2023-10-18T09:36:18.383 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2023-10-18T09:36:18.383 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 1, 2023-10-18T09:36:18.383 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 0, 2023-10-18T09:36:18.383 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2023-10-18T09:36:18.383 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2023-10-18T09:36:18.384 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2023-10-18T09:36:18.384 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2023-10-18T09:36:18.384 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 2953, 2023-10-18T09:36:18.384 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2023-10-18T09:36:18.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.385 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2023-10-18T09:36:18.385 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2023-10-18T09:36:18.385 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2023-10-18T09:36:18.385 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2023-10-18T09:36:18.385 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2023-10-18T09:36:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2023-10-18T09:36:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2023-10-18T09:36:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-18T09:36:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2023-10-18T09:36:18.386 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-18T09:36:18.387 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2023-10-18T09:36:18.387 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2023-10-18T09:36:18.387 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2023-10-18T09:36:18.387 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-18T09:36:18.387 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-18T09:36:18.388 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2023-10-18T09:36:18.389 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-18T09:36:18.389 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2023-10-18T09:36:18.389 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2023-10-18T09:36:18.389 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2023-10-18T09:36:18.389 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2023-10-18T09:36:18.390 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2023-10-18T09:36:18.390 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-18T09:36:18.390 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2023-10-18T09:36:18.390 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2023-10-18T09:36:18.390 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2023-10-18T09:36:18.391 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2023-10-18T09:36:18.392 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 200289, 2023-10-18T09:36:18.392 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 307, 2023-10-18T09:36:18.392 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2023-10-18T09:36:18.392 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2023-10-18T09:36:18.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2023-10-18T09:36:18.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2023-10-18T09:36:18.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2023-10-18T09:36:18.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2023-10-18T09:36:18.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2023-10-18T09:36:18.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2023-10-18T09:36:18.394 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 8888, 2023-10-18T09:36:18.394 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 170, 2023-10-18T09:36:18.394 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2023-10-18T09:36:18.394 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2023-10-18T09:36:18.394 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 817275, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 7387, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2023-10-18T09:36:18.395 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2023-10-18T09:36:18.396 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2023-10-18T09:36:18.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.396 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2023-10-18T09:36:18.396 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2023-10-18T09:36:18.396 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 3541, 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 3541, 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2023-10-18T09:36:18.397 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3541, 2023-10-18T09:36:18.398 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 23.812457994, 2023-10-18T09:36:18.398 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.006724783 2023-10-18T09:36:18.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.398 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2023-10-18T09:36:18.398 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2023-10-18T09:36:18.399 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3541, 2023-10-18T09:36:18.399 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3542 2023-10-18T09:36:18.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.399 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 3541, 2023-10-18T09:36:18.399 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 3541, 2023-10-18T09:36:18.400 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2023-10-18T09:36:18.400 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2023-10-18T09:36:18.400 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2023-10-18T09:36:18.400 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 2163, 2023-10-18T09:36:18.400 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1376, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2023-10-18T09:36:18.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2023-10-18T09:36:18.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2023-10-18T09:36:18.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2023-10-18T09:36:18.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2023-10-18T09:36:18.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2023-10-18T09:36:18.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2023-10-18T09:36:18.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2023-10-18T09:36:18.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2023-10-18T09:36:18.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2023-10-18T09:36:18.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2023-10-18T09:36:18.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2023-10-18T09:36:18.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2023-10-18T09:36:18.405 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2023-10-18T09:36:18.405 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2023-10-18T09:36:18.405 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2023-10-18T09:36:18.405 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2023-10-18T09:36:18.405 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2023-10-18T09:36:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2023-10-18T09:36:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2023-10-18T09:36:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2023-10-18T09:36:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2023-10-18T09:36:18.406 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2023-10-18T09:36:18.407 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2023-10-18T09:36:18.408 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2023-10-18T09:36:18.408 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 92, 2023-10-18T09:36:18.408 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2023-10-18T09:36:18.408 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 4, 2023-10-18T09:36:18.408 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 5, 2023-10-18T09:36:18.409 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 7, 2023-10-18T09:36:18.409 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 2, 2023-10-18T09:36:18.409 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2023-10-18T09:36:18.409 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2023-10-18T09:36:18.409 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2023-10-18T09:36:18.410 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2023-10-18T09:36:18.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.410 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2023-10-18T09:36:18.410 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2023-10-18T09:36:18.410 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2023-10-18T09:36:18.411 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2023-10-18T09:36:18.411 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2023-10-18T09:36:18.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.411 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2023-10-18T09:36:18.411 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2023-10-18T09:36:18.412 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2023-10-18T09:36:18.413 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2023-10-18T09:36:18.413 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2023-10-18T09:36:18.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.413 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-18T09:36:18.413 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:18.414 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:18.414 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:18.414 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3899, 2023-10-18T09:36:18.414 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 2556032, 2023-10-18T09:36:18.414 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3899, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3899, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2556032, 2023-10-18T09:36:18.415 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:18.416 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.416 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.416 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.416 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.416 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.417 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2023-10-18T09:36:18.417 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:18.417 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2023-10-18T09:36:18.417 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:18.417 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3541, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 1295249, 2023-10-18T09:36:18.418 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3541, 2023-10-18T09:36:18.419 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1295249, 2023-10-18T09:36:18.419 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:18.419 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.419 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.419 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.420 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.420 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.420 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2023-10-18T09:36:18.420 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:18.420 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2023-10-18T09:36:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:18.421 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 3541, 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 3541, 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 3541, 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 3541, 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:18.422 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.423 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.423 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.423 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.423 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.423 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:18.424 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:18.425 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:18.425 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:18.425 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:18.425 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-10-18T09:36:18.425 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-10-18T09:36:18.426 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:18.426 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.426 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.426 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.426 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x563c6cb16960": { 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2023-10-18T09:36:18.427 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2023-10-18T09:36:18.428 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2023-10-18T09:36:18.429 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2023-10-18T09:36:18.429 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2023-10-18T09:36:18.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2023-10-18T09:36:18.429 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2023-10-18T09:36:18.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2023-10-18T09:36:18.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-18T09:36:18.430 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-18T09:36:18.734 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-10-18T09:36:18.735 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-10-18T09:36:18.735 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-10-18T09:36:18.735 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-10-18T09:36:18.736 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-18T09:36:18.746 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-18T09:36:18.746 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:36:18.747 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:36:18.819 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:36:18.819 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:36:18.867 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:36:19.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 -- 172.21.15.61:0/763324982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8070103b40 msgr2=0x7f8070103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 --2- 172.21.15.61:0/763324982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8070103b40 0x7f8070103fa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f80640099c0 tx=0x7f8064009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.121 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 -- 172.21.15.61:0/763324982 shutdown_connections 2023-10-18T09:36:19.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 --2- 172.21.15.61:0/763324982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80701044e0 0x7f807010ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 --2- 172.21.15.61:0/763324982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070102950 0x7f8070102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 --2- 172.21.15.61:0/763324982 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8070103b40 0x7f8070103fa0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 -- 172.21.15.61:0/763324982 >> 172.21.15.61:0/763324982 conn(0x7f80700fdec0 msgr2=0x7f8070100300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:19.122 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 -- 172.21.15.61:0/763324982 shutdown_connections 2023-10-18T09:36:19.123 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.118+0000 7f80769ff700 1 -- 172.21.15.61:0/763324982 wait complete. 2023-10-18T09:36:19.124 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.120+0000 7f80769ff700 1 Processor -- start 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.120+0000 7f80769ff700 1 -- start start 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.120+0000 7f80769ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f80769ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070103b40 0x7f8070198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f80769ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80701044e0 0x7f807019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f80769ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a03f0 con 0x7f80701044e0 2023-10-18T09:36:19.125 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f80769ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a0530 con 0x7f8070102950 2023-10-18T09:36:19.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f80769ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a06c0 con 0x7f8070103b40 2023-10-18T09:36:19.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56156/0 (socket says 172.21.15.61:56156) 2023-10-18T09:36:19.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 -- 172.21.15.61:0/350426478 learned_addr learned my addr 172.21.15.61:0/350426478 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:19.126 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806f7fe700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070103b40 0x7f8070198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f8074f9c700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80701044e0 0x7f807019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 -- 172.21.15.61:0/350426478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070103b40 msgr2=0x7f8070198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070103b40 0x7f8070198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 -- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80701044e0 msgr2=0x7f807019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.127 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80701044e0 0x7f807019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.121+0000 7f806ffff700 1 -- 172.21.15.61:0/350426478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80640095f0 con 0x7f8070102950 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f806ffff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f806000d980 tx=0x7f806000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060004020 con 0x7f8070102950 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f806000f690 con 0x7f8070102950 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060003a60 con 0x7f8070102950 2023-10-18T09:36:19.128 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80701a09a0 con 0x7f8070102950 2023-10-18T09:36:19.129 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.122+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701a0eb0 con 0x7f8070102950 2023-10-18T09:36:19.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.123+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8070109e20 con 0x7f8070102950 2023-10-18T09:36:19.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.125+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f806000fb00 con 0x7f8070102950 2023-10-18T09:36:19.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.126+0000 7f806d7fa700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 0x7f80580a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.130 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.126+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f80600ccbe0 con 0x7f8070102950 2023-10-18T09:36:19.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.127+0000 7f806f7fe700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 0x7f80580a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.131 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.127+0000 7f806f7fe700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 0x7f80580a39c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f806400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:19.134 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.132+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8060095e90 con 0x7f8070102950 2023-10-18T09:36:19.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:19 smithi061 ceph-mon[98573]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-10-18T09:36:19.350 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.347+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8070067530 con 0x7f8070102950 2023-10-18T09:36:19.351 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.349+0000 7f806d7fa700 1 -- 172.21.15.61:0/350426478 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7f80600994b0 con 0x7f8070102950 2023-10-18T09:36:19.352 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:36:19.352 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:36:19.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 msgr2=0x7f80580a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 0x7f80580a39c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8064006010 tx=0x7f806400b540 comp rx=0 tx=0).stop 2023-10-18T09:36:19.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 msgr2=0x7f80701988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.354 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f806000d980 tx=0x7f806000dc90 comp rx=0 tx=0).stop 2023-10-18T09:36:19.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 shutdown_connections 2023-10-18T09:36:19.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f80580a1500 0x7f80580a39c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f80701044e0 0x7f807019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8070102950 0x7f80701988c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.355 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 --2- 172.21.15.61:0/350426478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8070103b40 0x7f8070198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 >> 172.21.15.61:0/350426478 conn(0x7f80700fdec0 msgr2=0x7f8070106f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:19.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.351+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 shutdown_connections 2023-10-18T09:36:19.356 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.352+0000 7f80769ff700 1 -- 172.21.15.61:0/350426478 wait complete. 2023-10-18T09:36:19.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-18T09:12:11.367908+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-18T09:16:24.457067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-18T09:16:25.828879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-18T09:16:29.402923+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-18T09:36:19.375 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-18T09:36:19.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:19 smithi110 ceph-mon[134818]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-10-18T09:36:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 -- 172.21.15.61:0/3668173455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 msgr2=0x7fd268072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.586 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/3668173455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268072c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd264009990 tx=0x7fd264009ca0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 -- 172.21.15.61:0/3668173455 shutdown_connections 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/3668173455 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/3668173455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268072c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/3668173455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd2680736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 -- 172.21.15.61:0/3668173455 >> 172.21.15.61:0/3668173455 conn(0x7fd2680fbcf0 msgr2=0x7fd2680fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:19.587 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.583+0000 7fd26f7ff700 1 -- 172.21.15.61:0/3668173455 shutdown_connections 2023-10-18T09:36:19.588 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.584+0000 7fd26f7ff700 1 -- 172.21.15.61:0/3668173455 wait complete. 2023-10-18T09:36:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.584+0000 7fd26f7ff700 1 Processor -- start 2023-10-18T09:36:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 -- start start 2023-10-18T09:36:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.590 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681a0430 con 0x7fd268108660 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681a0570 con 0x7fd268073250 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26f7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2681a0700 con 0x7fd268074830 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26d59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26d59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56160/0 (socket says 172.21.15.61:56160) 2023-10-18T09:36:19.591 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26d59b700 1 -- 172.21.15.61:0/1488144646 learned_addr learned my addr 172.21.15.61:0/1488144646 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26cd9a700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26dd9c700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26d59b700 1 -- 172.21.15.61:0/1488144646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 msgr2=0x7fd268198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.585+0000 7fd26d59b700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.592 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26d59b700 1 -- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 msgr2=0x7fd26819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26d59b700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26d59b700 1 -- 172.21.15.61:0/1488144646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640095f0 con 0x7fd268073250 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26dd9c700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26d59b700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098e0 tx=0x7fd264005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26401d070 con 0x7fd268073250 2023-10-18T09:36:19.593 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd264022470 con 0x7fd268073250 2023-10-18T09:36:19.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264017650 con 0x7fd268073250 2023-10-18T09:36:19.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2681a0980 con 0x7fd268073250 2023-10-18T09:36:19.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.586+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2681a0d50 con 0x7fd268073250 2023-10-18T09:36:19.594 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.587+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd268107420 con 0x7fd268073250 2023-10-18T09:36:19.595 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.589+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fd264022870 con 0x7fd268073250 2023-10-18T09:36:19.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.590+0000 7fd25e7fc700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 0x7fd2540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:19.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.591+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd2640ce7e0 con 0x7fd268073250 2023-10-18T09:36:19.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.591+0000 7fd26cd9a700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 0x7fd2540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:19.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.591+0000 7fd26cd9a700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 0x7fd2540a39c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd268074550 tx=0x7fd258009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:19.596 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.592+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd264097b20 con 0x7fd268073250 2023-10-18T09:36:19.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:19 smithi170 ceph-mon[135150]: pgmap v890: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2023-10-18T09:36:19.808 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.806+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd26810e1e0 con 0x7fd268073250 2023-10-18T09:36:19.809 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.807+0000 7fd25e7fc700 1 -- 172.21.15.61:0/1488144646 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd264027020 con 0x7fd268073250 2023-10-18T09:36:19.810 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-10-18T09:36:19.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.809+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 msgr2=0x7fd2540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.812 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.809+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 0x7fd2540a39c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd268074550 tx=0x7fd258009260 comp rx=0 tx=0).stop 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.809+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 msgr2=0x7fd268198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.809+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd2640098e0 tx=0x7fd264005740 comp rx=0 tx=0).stop 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 shutdown_connections 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fd2540a1500 0x7fd2540a39c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fd268108660 0x7fd26819fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.813 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd268073250 0x7fd268198900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 --2- 172.21.15.61:0/1488144646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd268074830 0x7fd268198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:19.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 >> 172.21.15.61:0/1488144646 conn(0x7fd2680fbcf0 msgr2=0x7fd268104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:19.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 shutdown_connections 2023-10-18T09:36:19.814 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:19.810+0000 7fd26f7ff700 1 -- 172.21.15.61:0/1488144646 wait complete. 2023-10-18T09:36:19.831 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-18T09:36:20.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3693371482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8101990 msgr2=0x7f76c8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3693371482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8101990 0x7f76c8101df0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f76b40099c0 tx=0x7f76b4009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3693371482 shutdown_connections 2023-10-18T09:36:20.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3693371482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 0x7f76c8108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.040 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3693371482 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c81007a0 0x7f76c8100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3693371482 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c8101990 0x7f76c8101df0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3693371482 >> 172.21.15.61:0/3693371482 conn(0x7f76c80fbcf0 msgr2=0x7f76c80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:20.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3693371482 shutdown_connections 2023-10-18T09:36:20.041 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.037+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3693371482 wait complete. 2023-10-18T09:36:20.043 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.038+0000 7f76cf1ff700 1 Processor -- start 2023-10-18T09:36:20.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.038+0000 7f76cf1ff700 1 -- start start 2023-10-18T09:36:20.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 0x7f76c8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 0x7f76c819fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.044 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a0460 con 0x7f76c8101990 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a05a0 con 0x7f76c81007a0 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cf1ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c81a0730 con 0x7f76c8102330 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56174/0 (socket says 172.21.15.61:56174) 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76c7fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 0x7f76c8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.045 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 -- 172.21.15.61:0/3382917039 learned_addr learned my addr 172.21.15.61:0/3382917039 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:20.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76cd79c700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 0x7f76c819fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 -- 172.21.15.61:0/3382917039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 msgr2=0x7f76c819fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 0x7f76c819fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 -- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 msgr2=0x7f76c8198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.046 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 0x7f76c8198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 -- 172.21.15.61:0/3382917039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40095f0 con 0x7f76c81007a0 2023-10-18T09:36:20.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76c7fff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 0x7f76c8198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:36:20.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.039+0000 7f76ccf9b700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e970 tx=0x7f76bc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.040+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc009790 con 0x7f76c81007a0 2023-10-18T09:36:20.047 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.040+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76bc00eec0 con 0x7f76c81007a0 2023-10-18T09:36:20.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.040+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c81a0a10 con 0x7f76c81007a0 2023-10-18T09:36:20.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.040+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003cf0 con 0x7f76c81007a0 2023-10-18T09:36:20.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.040+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c81a0f20 con 0x7f76c81007a0 2023-10-18T09:36:20.048 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.041+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c80675a0 con 0x7f76c81007a0 2023-10-18T09:36:20.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.043+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f76bc00ccd0 con 0x7f76c81007a0 2023-10-18T09:36:20.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.044+0000 7f76c5ffb700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 0x7f76b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.049 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.044+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76bc014070 con 0x7f76c81007a0 2023-10-18T09:36:20.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.045+0000 7f76c7fff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 0x7f76b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.050 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.045+0000 7f76c7fff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 0x7f76b80a3940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003730 tx=0x7f76b401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.052 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.050+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f76bc0965d0 con 0x7f76c81007a0 2023-10-18T09:36:20.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.263+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76c810c220 con 0x7f76c81007a0 2023-10-18T09:36:20.266 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.264+0000 7f76c5ffb700 1 -- 172.21.15.61:0/3382917039 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76bc099bf0 con 0x7f76c81007a0 2023-10-18T09:36:20.267 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:36:20.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 msgr2=0x7f76b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 0x7f76b80a3940 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f76b4003730 tx=0x7f76b401a040 comp rx=0 tx=0).stop 2023-10-18T09:36:20.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 msgr2=0x7f76c81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.269 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00e970 tx=0x7f76bc00ec80 comp rx=0 tx=0).stop 2023-10-18T09:36:20.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 shutdown_connections 2023-10-18T09:36:20.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f76b80a1480 0x7f76b80a3940 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76c8102330 0x7f76c819fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f76c81007a0 0x7f76c81989d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.270 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 --2- 172.21.15.61:0/3382917039 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f76c8101990 0x7f76c8198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 >> 172.21.15.61:0/3382917039 conn(0x7f76c80fbcf0 msgr2=0x7f76c8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:20.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 shutdown_connections 2023-10-18T09:36:20.271 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.266+0000 7f76cf1ff700 1 -- 172.21.15.61:0/3382917039 wait complete. 2023-10-18T09:36:20.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-18T09:36:20.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:20 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/350426478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:20.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:20 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1488144646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:36:20.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 -- 172.21.15.61:0/1878425578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 msgr2=0x7f1b4c108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 --2- 172.21.15.61:0/1878425578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c108be0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400b0d0 tx=0x7f1b4400b3e0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 -- 172.21.15.61:0/1878425578 shutdown_connections 2023-10-18T09:36:20.498 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 --2- 172.21.15.61:0/1878425578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c108be0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 --2- 172.21.15.61:0/1878425578 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c1007a0 0x7f1b4c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 --2- 172.21.15.61:0/1878425578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c101990 0x7f1b4c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 -- 172.21.15.61:0/1878425578 >> 172.21.15.61:0/1878425578 conn(0x7f1b4c0fbcf0 msgr2=0x7f1b4c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:20.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 -- 172.21.15.61:0/1878425578 shutdown_connections 2023-10-18T09:36:20.499 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.495+0000 7f1b547ff700 1 -- 172.21.15.61:0/1878425578 wait complete. 2023-10-18T09:36:20.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.496+0000 7f1b547ff700 1 Processor -- start 2023-10-18T09:36:20.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.496+0000 7f1b547ff700 1 -- start start 2023-10-18T09:36:20.501 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c1007a0 0x7f1b4c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1a0570 con 0x7f1b4c101990 2023-10-18T09:36:20.502 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1a06b0 con 0x7f1b4c102330 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b547ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1a0840 con 0x7f1b4c1007a0 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b51d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b51d9a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45206/0 (socket says 172.21.15.61:45206) 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b51d9a700 1 -- 172.21.15.61:0/3329565137 learned_addr learned my addr 172.21.15.61:0/3329565137 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:20.503 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b5259b700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c1007a0 0x7f1b4c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 -- 172.21.15.61:0/3329565137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c1007a0 msgr2=0x7f1b4c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c1007a0 0x7f1b4c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 -- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 msgr2=0x7f1b4c198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b52d9c700 1 -- 172.21.15.61:0/3329565137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009d00 con 0x7f1b4c102330 2023-10-18T09:36:20.504 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.497+0000 7f1b51d9a700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:36:20.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b52d9c700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c19ffa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007b60 tx=0x7f1b44008e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400e040 con 0x7f1b4c102330 2023-10-18T09:36:20.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b44005070 con 0x7f1b4c102330 2023-10-18T09:36:20.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4401d6d0 con 0x7f1b4c102330 2023-10-18T09:36:20.505 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c1a0ac0 con 0x7f1b4c102330 2023-10-18T09:36:20.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.498+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c1a0ee0 con 0x7f1b4c102330 2023-10-18T09:36:20.506 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.499+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c0675a0 con 0x7f1b4c102330 2023-10-18T09:36:20.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:20 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/350426478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:20.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:20 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1488144646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:36:20.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.502+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f1b440051e0 con 0x7f1b4c102330 2023-10-18T09:36:20.507 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.503+0000 7f1b437fe700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 0x7f1b380a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.503+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1b440cde20 con 0x7f1b4c102330 2023-10-18T09:36:20.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.503+0000 7f1b5259b700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 0x7f1b380a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.508 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.504+0000 7f1b5259b700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 0x7f1b380a39c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1017f0 tx=0x7f1b48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.510 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.508+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1b44097250 con 0x7f1b4c102330 2023-10-18T09:36:20.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:20 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/350426478' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:20.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:20 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1488144646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-18T09:36:20.722 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.719+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b4c1a1260 con 0x7f1b4c102330 2023-10-18T09:36:20.723 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.719+0000 7f1b437fe700 1 -- 172.21.15.61:0/3329565137 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b4409a870 con 0x7f1b4c102330 2023-10-18T09:36:20.723 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 64 2023-10-18T09:36:20.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 msgr2=0x7f1b380a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 0x7f1b380a39c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c1017f0 tx=0x7f1b48009260 comp rx=0 tx=0).stop 2023-10-18T09:36:20.725 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 msgr2=0x7f1b4c19ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c19ffa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007b60 tx=0x7f1b44008e40 comp rx=0 tx=0).stop 2023-10-18T09:36:20.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 shutdown_connections 2023-10-18T09:36:20.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f1b380a1500 0x7f1b380a39c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1b4c102330 0x7f1b4c19ffa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.726 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b4c1007a0 0x7f1b4c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 --2- 172.21.15.61:0/3329565137 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f1b4c101990 0x7f1b4c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.721+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 >> 172.21.15.61:0/3329565137 conn(0x7f1b4c0fbcf0 msgr2=0x7f1b4c072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:20.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.722+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 shutdown_connections 2023-10-18T09:36:20.727 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.722+0000 7f1b547ff700 1 -- 172.21.15.61:0/3329565137 wait complete. 2023-10-18T09:36:20.744 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-18T09:36:20.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 -- 172.21.15.61:0/2872798352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc100880 msgr2=0x7f8ffc100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 --2- 172.21.15.61:0/2872798352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc100880 0x7f8ffc100ce0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00099c0 tx=0x7f8ff0009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 -- 172.21.15.61:0/2872798352 shutdown_connections 2023-10-18T09:36:20.956 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 --2- 172.21.15.61:0/2872798352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 0x7f8ffc107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 --2- 172.21.15.61:0/2872798352 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 --2- 172.21.15.61:0/2872798352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc100880 0x7f8ffc100ce0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 -- 172.21.15.61:0/2872798352 >> 172.21.15.61:0/2872798352 conn(0x7f8ffc0fac00 msgr2=0x7f8ffc0fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:20.957 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 -- 172.21.15.61:0/2872798352 shutdown_connections 2023-10-18T09:36:20.958 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.952+0000 7f90027ff700 1 -- 172.21.15.61:0/2872798352 wait complete. 2023-10-18T09:36:20.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.953+0000 7f90027ff700 1 Processor -- start 2023-10-18T09:36:20.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.953+0000 7f90027ff700 1 -- start start 2023-10-18T09:36:20.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.960 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc100880 0x7f8ffc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 0x7f8ffc19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a0410 con 0x7f8ffc100880 2023-10-18T09:36:20.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a0550 con 0x7f8ffc0ff690 2023-10-18T09:36:20.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f90027ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1a06e0 con 0x7f8ffc101220 2023-10-18T09:36:20.961 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56210/0 (socket says 172.21.15.61:56210) 2023-10-18T09:36:20.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 -- 172.21.15.61:0/209389356 learned_addr learned my addr 172.21.15.61:0/209389356 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:20.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffb7fe700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc100880 0x7f8ffc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f9000d9c700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 0x7f8ffc19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.962 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 -- 172.21.15.61:0/209389356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 msgr2=0x7f8ffc19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 0x7f8ffc19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 -- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc100880 msgr2=0x7f8ffc198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:20.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc100880 0x7f8ffc198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:20.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.954+0000 7f8ffbfff700 1 -- 172.21.15.61:0/209389356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00095f0 con 0x7f8ffc0ff690 2023-10-18T09:36:20.963 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f8ffbfff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e970 tx=0x7f8fec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec009790 con 0x7f8ffc0ff690 2023-10-18T09:36:20.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fec00c7e0 con 0x7f8ffc0ff690 2023-10-18T09:36:20.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1a09c0 con 0x7f8ffc0ff690 2023-10-18T09:36:20.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003c70 con 0x7f8ffc0ff690 2023-10-18T09:36:20.964 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.955+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1a0ed0 con 0x7f8ffc0ff690 2023-10-18T09:36:20.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.956+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc0675a0 con 0x7f8ffc0ff690 2023-10-18T09:36:20.965 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.958+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f8fec00cc50 con 0x7f8ffc0ff690 2023-10-18T09:36:20.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.959+0000 7f8ff97fa700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 0x7f8fe40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:20.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.960+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8fec014070 con 0x7f8ffc0ff690 2023-10-18T09:36:20.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.960+0000 7f8ffb7fe700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 0x7f8fe40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:20.966 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.960+0000 7f8ffb7fe700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 0x7f8fe40a3940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0006010 tx=0x7f8ff000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:20.967 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:20.962+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8fec096660 con 0x7f8ffc0ff690 2023-10-18T09:36:21.180 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.177+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8ffc10c1e0 con 0x7f8ffc0ff690 2023-10-18T09:36:21.181 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.177+0000 7f8ff97fa700 1 -- 172.21.15.61:0/209389356 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8fec099c80 con 0x7f8ffc0ff690 2023-10-18T09:36:21.181 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 32 2023-10-18T09:36:21.183 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 msgr2=0x7f8fe40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 0x7f8fe40a3940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0006010 tx=0x7f8ff000b540 comp rx=0 tx=0).stop 2023-10-18T09:36:21.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 msgr2=0x7f8ffc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.184 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00e970 tx=0x7f8fec00ec80 comp rx=0 tx=0).stop 2023-10-18T09:36:21.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 shutdown_connections 2023-10-18T09:36:21.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f8fe40a1480 0x7f8fe40a3940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ffc101220 0x7f8ffc19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.185 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ffc0ff690 0x7f8ffc1988e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 --2- 172.21.15.61:0/209389356 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f8ffc100880 0x7f8ffc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 >> 172.21.15.61:0/209389356 conn(0x7f8ffc0fac00 msgr2=0x7f8ffc108420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:21.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 shutdown_connections 2023-10-18T09:36:21.186 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.180+0000 7f90027ff700 1 -- 172.21.15.61:0/209389356 wait complete. 2023-10-18T09:36:21.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-18T09:36:21.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 -- 172.21.15.61:0/3093956075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c41085f0 msgr2=0x7f38c410aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/3093956075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c41085f0 0x7f38c410aa60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00b0a0 tx=0x7f38bc00b3b0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 -- 172.21.15.61:0/3093956075 shutdown_connections 2023-10-18T09:36:21.412 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/3093956075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c41085f0 0x7f38c410aa60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/3093956075 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c40747d0 0x7f38c4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/3093956075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 -- 172.21.15.61:0/3093956075 >> 172.21.15.61:0/3093956075 conn(0x7f38c40fbc90 msgr2=0x7f38c40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:21.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 -- 172.21.15.61:0/3093956075 shutdown_connections 2023-10-18T09:36:21.413 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.408+0000 7f38ca9ff700 1 -- 172.21.15.61:0/3093956075 wait complete. 2023-10-18T09:36:21.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 Processor -- start 2023-10-18T09:36:21.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 -- start start 2023-10-18T09:36:21.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:21.416 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c40747d0 0x7f38c4198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:21.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c41085f0 0x7f38c419fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:21.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c41a04a0 con 0x7f38c41085f0 2023-10-18T09:36:21.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c41a05e0 con 0x7f38c40747d0 2023-10-18T09:36:21.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.409+0000 7f38ca9ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c41a0770 con 0x7f38c40731f0 2023-10-18T09:36:21.417 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:21.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c40747d0 0x7f38c4198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:21.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:45972/0 (socket says 172.21.15.61:45972) 2023-10-18T09:36:21.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 -- 172.21.15.61:0/1231744136 learned_addr learned my addr 172.21.15.61:0/1231744136 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:21.418 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c8f9c700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c41085f0 0x7f38c419fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:21.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 -- 172.21.15.61:0/1231744136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c40747d0 msgr2=0x7f38c4198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c40747d0 0x7f38c4198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 -- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c41085f0 msgr2=0x7f38c419fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c41085f0 0x7f38c419fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.419 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 -- 172.21.15.61:0/1231744136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009520 con 0x7f38c40731f0 2023-10-18T09:36:21.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c3fff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f38b400eb30 tx=0x7f38b400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:21.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.410+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4009880 con 0x7f38c40731f0 2023-10-18T09:36:21.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.411+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b40044e0 con 0x7f38c40731f0 2023-10-18T09:36:21.420 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.411+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc009d00 con 0x7f38c40731f0 2023-10-18T09:36:21.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.411+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4003d80 con 0x7f38c40731f0 2023-10-18T09:36:21.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.411+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c41a0e20 con 0x7f38c40731f0 2023-10-18T09:36:21.421 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.412+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c4107400 con 0x7f38c40731f0 2023-10-18T09:36:21.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.414+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f38b4003f20 con 0x7f38c40731f0 2023-10-18T09:36:21.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.415+0000 7f38c17fa700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 0x7f38ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:21.422 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.415+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f38b4014070 con 0x7f38c40731f0 2023-10-18T09:36:21.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.415+0000 7f38c37fe700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 0x7f38ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:21.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.416+0000 7f38c37fe700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 0x7f38ac0a39c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009c60 tx=0x7f38b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:21.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.419+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f38b4096820 con 0x7f38c40731f0 2023-10-18T09:36:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:21 smithi110 ceph-mon[134818]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:21 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3382917039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:21 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3329565137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:21 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/209389356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:21 smithi061 ceph-mon[98573]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:21.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:21 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3382917039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:21 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3329565137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:21 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/209389356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:21 smithi170 ceph-mon[135150]: pgmap v891: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-10-18T09:36:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:21 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3382917039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:21 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3329565137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:21 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/209389356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-18T09:36:21.653 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.650+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38c4067530 con 0x7f38c40731f0 2023-10-18T09:36:21.654 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.650+0000 7f38c17fa700 1 -- 172.21.15.61:0/1231744136 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) v1 ==== 81+0+591 (secure 0 0 0) 0x7f38b4099e40 con 0x7f38c40731f0 2023-10-18T09:36:21.654 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/3728584080 2023-10-19T09:09:20.739745+0000 2023-10-18T09:36:21.654 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1938489455 2023-10-19T09:09:20.739745+0000 2023-10-18T09:36:21.654 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:36:21.655 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6801/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:36:21.655 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2303013428 2023-10-19T09:08:53.425206+0000 2023-10-18T09:36:21.655 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/1676472102 2023-10-19T09:08:53.425206+0000 2023-10-18T09:36:21.655 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/1518173664 2023-10-19T09:09:20.739745+0000 2023-10-18T09:36:21.655 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:6800/2589487616 2023-10-19T09:09:20.739745+0000 2023-10-18T09:36:21.656 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/2840133191 2023-10-19T09:08:53.425206+0000 2023-10-18T09:36:21.656 INFO:teuthology.orchestra.run.smithi061.stdout:172.21.15.61:0/623274229 2023-10-19T09:08:53.425206+0000 2023-10-18T09:36:21.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 msgr2=0x7f38ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 0x7f38ac0a39c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009c60 tx=0x7f38b8009260 comp rx=0 tx=0).stop 2023-10-18T09:36:21.657 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 msgr2=0x7f38c4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:21.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f38b400eb30 tx=0x7f38b400ee40 comp rx=0 tx=0).stop 2023-10-18T09:36:21.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 shutdown_connections 2023-10-18T09:36:21.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f38ac0a1500 0x7f38ac0a39c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f38c41085f0 0x7f38c419fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.658 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c40731f0 0x7f38c4198970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 --2- 172.21.15.61:0/1231744136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f38c40747d0 0x7f38c4198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:21.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 >> 172.21.15.61:0/1231744136 conn(0x7f38c40fbc90 msgr2=0x7f38c4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:21.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 shutdown_connections 2023-10-18T09:36:21.659 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:21.653+0000 7f38ca9ff700 1 -- 172.21.15.61:0/1231744136 wait complete. 2023-10-18T09:36:21.660 INFO:teuthology.orchestra.run.smithi061.stderr:listed 10 entries 2023-10-18T09:36:21.676 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-18T09:36:21.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:21.677 DEBUG:teuthology.orchestra.run.smithi061:> sudo umount /tmp/cephfs 2023-10-18T09:36:21.893 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:36:21.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:21.894 DEBUG:teuthology.orchestra.run.smithi061:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-10-18T09:36:21.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:21.998 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-10-18T09:36:22.117 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-10-18T09:36:22.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:22.118 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-10-18T09:36:22.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:22.197 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link delete ceph-brx 2023-10-18T09:36:22.312 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-18T09:36:22.312 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-10-18T09:36:22.312 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-18T09:36:22.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348348k,nr_inodes=4087087,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=31,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275624k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e33befa65d2c4303106dad032df09fdc7bd5c2ac09762877fe4d1c0dccbcb320/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/e33befa65d2c4303106dad032df09fdc7bd5c2ac09762877fe4d1c0dccbcb320/diff,workdir=/var/lib/containers/storage/overlay/e33befa65d2c4303106dad032df09fdc7bd5c2ac09762877fe4d1c0dccbcb320/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a17f708cbc5d115fe53af034ce97d2b1956e4a554559e15260f365a4aab39e3e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/a17f708cbc5d115fe53af034ce97d2b1956e4a554559e15260f365a4aab39e3e/diff,workdir=/var/lib/containers/storage/overlay/a17f708cbc5d115fe53af034ce97d2b1956e4a554559e15260f365a4aab39e3e/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275624k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/8593a1638b83b829bd73b9a1c5da1eaa6a0932e74cc8562ac5ca26ea9405fa12/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/8593a1638b83b829bd73b9a1c5da1eaa6a0932e74cc8562ac5ca26ea9405fa12/diff,workdir=/var/lib/containers/storage/overlay/8593a1638b83b829bd73b9a1c5da1eaa6a0932e74cc8562ac5ca26ea9405fa12/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/566e43660e99231e261829e6127a91a101ee13be2b3d690af95dae04a88462d6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/566e43660e99231e261829e6127a91a101ee13be2b3d690af95dae04a88462d6/diff,workdir=/var/lib/containers/storage/overlay/566e43660e99231e261829e6127a91a101ee13be2b3d690af95dae04a88462d6/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/2b6739b09efc3fe0985d2ed74e8703f5c87283881611279c9e0cafaf53b405b0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/2b6739b09efc3fe0985d2ed74e8703f5c87283881611279c9e0cafaf53b405b0/diff,workdir=/var/lib/containers/storage/overlay/2b6739b09efc3fe0985d2ed74e8703f5c87283881611279c9e0cafaf53b405b0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/21f3cfb88dc721328cc64c2023126d83609639df7162b8e263970d6ecd24d75a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/21f3cfb88dc721328cc64c2023126d83609639df7162b8e263970d6ecd24d75a/diff,workdir=/var/lib/containers/storage/overlay/21f3cfb88dc721328cc64c2023126d83609639df7162b8e263970d6ecd24d75a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/86a567c9012d8fa141815ae13369639b7d60d2307675a099b3df2956a8463b16/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/86a567c9012d8fa141815ae13369639b7d60d2307675a099b3df2956a8463b16/diff,workdir=/var/lib/containers/storage/overlay/86a567c9012d8fa141815ae13369639b7d60d2307675a099b3df2956a8463b16/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/35011c6cae15a9284b32d1457cb3d59f844134fd225100a281b010985e21eecd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/35011c6cae15a9284b32d1457cb3d59f844134fd225100a281b010985e21eecd/diff,workdir=/var/lib/containers/storage/overlay/35011c6cae15a9284b32d1457cb3d59f844134fd225100a281b010985e21eecd/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0610bb67148c5c8564eab5d009889424111c6ade0fc1cd67d8be796ef8581ca1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/0610bb67148c5c8564eab5d009889424111c6ade0fc1cd67d8be796ef8581ca1/diff,workdir=/var/lib/containers/storage/overlay/0610bb67148c5c8564eab5d009889424111c6ade0fc1cd67d8be796ef8581ca1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/04d5220655a032b165e26267146fed967509be99b305afd81e9876aa4ad52ad3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/OTAY3NPSJX6TF5RHJS5JDETFMW:/var/lib/containers/storage/overlay/l/FELMKSTYPRQFR4UHSEPY2Z6XRN,upperdir=/var/lib/containers/storage/overlay/04d5220655a032b165e26267146fed967509be99b305afd81e9876aa4ad52ad3/diff,workdir=/var/lib/containers/storage/overlay/04d5220655a032b165e26267146fed967509be99b305afd81e9876aa4ad52ad3/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-18T09:36:22.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:22.358 DEBUG:teuthology.orchestra.run.smithi061:> ip netns list 2023-10-18T09:36:22.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-18T09:36:22.444 DEBUG:teuthology.orchestra.run.smithi061:> sudo ip link delete ceph-brx 2023-10-18T09:36:22.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:22 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1231744136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:36:22.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:22 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1231744136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:36:22.668 INFO:teuthology.orchestra.run.smithi061.stderr:Cannot find device "ceph-brx" 2023-10-18T09:36:22.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:36:22.672 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-18T09:36:22.683 INFO:tasks.cephadm:Teardown begin 2023-10-18T09:36:22.683 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-18T09:36:22.684 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:36:22.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:22 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1231744136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-18T09:36:22.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 -- 172.21.15.61:0/186089347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 msgr2=0x7fbb38101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:22.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 --2- 172.21.15.61:0/186089347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38101d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbb280099c0 tx=0x7fbb28009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 -- 172.21.15.61:0/186089347 shutdown_connections 2023-10-18T09:36:22.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 --2- 172.21.15.61:0/186089347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 --2- 172.21.15.61:0/186089347 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 --2- 172.21.15.61:0/186089347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38101d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 -- 172.21.15.61:0/186089347 >> 172.21.15.61:0/186089347 conn(0x7fbb380fbc90 msgr2=0x7fbb380fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:22.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 -- 172.21.15.61:0/186089347 shutdown_connections 2023-10-18T09:36:22.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.916+0000 7fbb401ff700 1 -- 172.21.15.61:0/186089347 wait complete. 2023-10-18T09:36:22.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.917+0000 7fbb401ff700 1 Processor -- start 2023-10-18T09:36:22.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.917+0000 7fbb401ff700 1 -- start start 2023-10-18T09:36:22.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:22.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:22.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:22.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380738c0 con 0x7fbb38100740 2023-10-18T09:36:22.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38073a30 con 0x7fbb38101930 2023-10-18T09:36:22.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb401ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb381a06f0 con 0x7fbb381022d0 2023-10-18T09:36:22.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3df9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:22.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:22.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3df9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45252/0 (socket says 172.21.15.61:45252) 2023-10-18T09:36:22.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3df9b700 1 -- 172.21.15.61:0/3094406081 learned_addr learned my addr 172.21.15.61:0/3094406081 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:22.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3e79c700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:22.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 -- 172.21.15.61:0/3094406081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 msgr2=0x7fbb38073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:22.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 -- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 msgr2=0x7fbb38074550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:22.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38074550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:22.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3d79a700 1 -- 172.21.15.61:0/3094406081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280095f0 con 0x7fbb38101930 2023-10-18T09:36:22.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.918+0000 7fbb3e79c700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-18T09:36:22.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb3d79a700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38072be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009fd0 tx=0x7fbb28005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:22.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2801d070 con 0x7fbb38101930 2023-10-18T09:36:22.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb280059a0 con 0x7fbb38101930 2023-10-18T09:36:22.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb381a0920 con 0x7fbb38101930 2023-10-18T09:36:22.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb280173f0 con 0x7fbb38101930 2023-10-18T09:36:22.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.919+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb381a0dc0 con 0x7fbb38101930 2023-10-18T09:36:22.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.920+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb38107c10 con 0x7fbb38101930 2023-10-18T09:36:22.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.922+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbb28022670 con 0x7fbb38101930 2023-10-18T09:36:22.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.923+0000 7fbb2effd700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 0x7fbb240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:22.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.923+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbb280ce830 con 0x7fbb38101930 2023-10-18T09:36:22.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.924+0000 7fbb3df9b700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 0x7fbb240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:22.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.924+0000 7fbb3df9b700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 0x7fbb240a39c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005950 tx=0x7fbb34009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:22.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:22.928+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb28097b70 con 0x7fbb38101930 2023-10-18T09:36:23.192 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.188+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb3810c070 con 0x7fbb38101930 2023-10-18T09:36:23.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.189+0000 7fbb2effd700 1 -- 172.21.15.61:0/3094406081 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+11389 (secure 0 0 0) 0x7fbb2809b190 con 0x7fbb38101930 2023-10-18T09:36:23.193 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:36:23.193 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:34:55.372778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:36:23.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 msgr2=0x7fbb240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 0x7fbb240a39c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb34005950 tx=0x7fbb34009260 comp rx=0 tx=0).stop 2023-10-18T09:36:23.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 msgr2=0x7fbb38072be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.196 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38072be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009fd0 tx=0x7fbb28005f10 comp rx=0 tx=0).stop 2023-10-18T09:36:23.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 shutdown_connections 2023-10-18T09:36:23.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbb240a1500 0x7fbb240a39c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb381022d0 0x7fbb38073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.197 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbb38100740 0x7fbb38074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 --2- 172.21.15.61:0/3094406081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb38101930 0x7fbb38072be0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 >> 172.21.15.61:0/3094406081 conn(0x7fbb380fbc90 msgr2=0x7fbb38104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:23.198 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 shutdown_connections 2023-10-18T09:36:23.199 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.192+0000 7fbb401ff700 1 -- 172.21.15.61:0/3094406081 wait complete. 2023-10-18T09:36:23.200 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 46 2023-10-18T09:36:23.217 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-18T09:36:23.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/221731848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf64100800 msgr2=0x7fbf64100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/221731848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf64100800 0x7fbf64100c60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf540099c0 tx=0x7fbf54009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.423 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/221731848 shutdown_connections 2023-10-18T09:36:23.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/221731848 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf641011a0 0x7fbf64107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/221731848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf640ff610 0x7fbf640ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/221731848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf64100800 0x7fbf64100c60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/221731848 >> 172.21.15.61:0/221731848 conn(0x7fbf640faba0 msgr2=0x7fbf640fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:23.424 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/221731848 shutdown_connections 2023-10-18T09:36:23.425 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.419+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/221731848 wait complete. 2023-10-18T09:36:23.426 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.420+0000 7fbf6c7ff700 1 Processor -- start 2023-10-18T09:36:23.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 -- start start 2023-10-18T09:36:23.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf64100800 0x7fbf64196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf641011a0 0x7fbf6419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.427 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419e1b0 con 0x7fbf64100800 2023-10-18T09:36:23.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419e2f0 con 0x7fbf640ff610 2023-10-18T09:36:23.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6c7ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6419e480 con 0x7fbf641011a0 2023-10-18T09:36:23.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56270/0 (socket says 172.21.15.61:56270) 2023-10-18T09:36:23.428 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 -- 172.21.15.61:0/666471304 learned_addr learned my addr 172.21.15.61:0/666471304 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf69d9a700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf64100800 0x7fbf64196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6ad9c700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf641011a0 0x7fbf6419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 -- 172.21.15.61:0/666471304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf641011a0 msgr2=0x7fbf6419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf641011a0 0x7fbf6419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 -- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf64100800 msgr2=0x7fbf64196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.429 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.421+0000 7fbf6a59b700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf64100800 0x7fbf64196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf6a59b700 1 -- 172.21.15.61:0/666471304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf540095f0 con 0x7fbf640ff610 2023-10-18T09:36:23.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf6a59b700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e970 tx=0x7fbf6000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:23.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60009790 con 0x7fbf640ff610 2023-10-18T09:36:23.430 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf6000eec0 con 0x7fbf640ff610 2023-10-18T09:36:23.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64101c20 con 0x7fbf640ff610 2023-10-18T09:36:23.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60003cf0 con 0x7fbf640ff610 2023-10-18T09:36:23.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.422+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf64102130 con 0x7fbf640ff610 2023-10-18T09:36:23.431 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.423+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf64067530 con 0x7fbf640ff610 2023-10-18T09:36:23.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.425+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fbf6000ccd0 con 0x7fbf640ff610 2023-10-18T09:36:23.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.427+0000 7fbf5b7fe700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 0x7fbf500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.427+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbf60014070 con 0x7fbf640ff610 2023-10-18T09:36:23.432 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.427+0000 7fbf69d9a700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 0x7fbf500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.427+0000 7fbf69d9a700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 0x7fbf500a3940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbf54006010 tx=0x7fbf5400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:23.433 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.430+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf600969b0 con 0x7fbf640ff610 2023-10-18T09:36:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:23 smithi061 ceph-mon[98573]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:23.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:23 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3094406081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:23 smithi170 ceph-mon[135150]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:23.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:23 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3094406081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:23.698 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.695+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf6410ae70 con 0x7fbf640ff610 2023-10-18T09:36:23.699 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.696+0000 7fbf5b7fe700 1 -- 172.21.15.61:0/666471304 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+11389 (secure 0 0 0) 0x7fbf60099fd0 con 0x7fbf640ff610 2023-10-18T09:36:23.699 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:36:23.699 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6835/2809482931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6834","nonce":2809482931},{"type":"v1","addr":"172.21.15.61:6835","nonce":2809482931}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14616,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.61:6837/1807421967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6836","nonce":1807421967},{"type":"v1","addr":"172.21.15.61:6837","nonce":1807421967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-18T09:16:26.396655+0000","modified":"2023-10-18T09:34:55.372778+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24692,"mds_1":24676,"mds_2":24653,"mds_3":24646,"mds_4":24629},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14688":{"gid":14688,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6841/2114349313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6840","nonce":2114349313},{"type":"v1","addr":"172.21.15.61:6841","nonce":2114349313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24589":{"gid":24589,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6833/2686391817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2686391817},{"type":"v1","addr":"172.21.15.170:6833","nonce":2686391817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6833/3590204423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":3590204423},{"type":"v1","addr":"172.21.15.110:6833","nonce":3590204423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24629":{"gid":24629,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.110:6835/145516333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":145516333},{"type":"v1","addr":"172.21.15.110:6835","nonce":145516333}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24646":{"gid":24646,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":15,"addr":"172.21.15.170:6835/1203136193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1203136193},{"type":"v1","addr":"172.21.15.170:6835","nonce":1203136193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24653":{"gid":24653,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.110:6837/2750873982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2750873982},{"type":"v1","addr":"172.21.15.110:6837","nonce":2750873982}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24658":{"gid":24658,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6837/3025282224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3025282224},{"type":"v1","addr":"172.21.15.170:6837","nonce":3025282224}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24668":{"gid":24668,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.61:6839/2550944546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6838","nonce":2550944546},{"type":"v1","addr":"172.21.15.61:6839","nonce":2550944546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24676":{"gid":24676,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.170:6839/2453939791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":2453939791},{"type":"v1","addr":"172.21.15.170:6839","nonce":2453939791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24692":{"gid":24692,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.110:6839/1483417844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":1483417844},{"type":"v1","addr":"172.21.15.110:6839","nonce":1483417844}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-18T09:36:23.701 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 msgr2=0x7fbf500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 0x7fbf500a3940 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbf54006010 tx=0x7fbf5400b540 comp rx=0 tx=0).stop 2023-10-18T09:36:23.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 msgr2=0x7fbf641966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000e970 tx=0x7fbf6000ec80 comp rx=0 tx=0).stop 2023-10-18T09:36:23.702 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 shutdown_connections 2023-10-18T09:36:23.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fbf500a1480 0x7fbf500a3940 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbf641011a0 0x7fbf6419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf640ff610 0x7fbf641966d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 --2- 172.21.15.61:0/666471304 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fbf64100800 0x7fbf64196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.703 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 >> 172.21.15.61:0/666471304 conn(0x7fbf640faba0 msgr2=0x7fbf64067830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:23.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 shutdown_connections 2023-10-18T09:36:23.704 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.698+0000 7fbf6c7ff700 1 -- 172.21.15.61:0/666471304 wait complete. 2023-10-18T09:36:23.704 INFO:teuthology.orchestra.run.smithi061.stderr:dumped fsmap epoch 46 2023-10-18T09:36:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:23 smithi110 ceph-mon[134818]: pgmap v892: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-10-18T09:36:23.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:23 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3094406081' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:23.722 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-18T09:36:23.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 -- 172.21.15.61:0/1247708647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f8101950 msgr2=0x7fe2f8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/1247708647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f8101950 0x7fe2f8101db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0099c0 tx=0x7fe2ec009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 -- 172.21.15.61:0/1247708647 shutdown_connections 2023-10-18T09:36:23.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/1247708647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f81022f0 0x7fe2f8108ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/1247708647 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8100760 0x7fe2f8100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/1247708647 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f8101950 0x7fe2f8101db0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 -- 172.21.15.61:0/1247708647 >> 172.21.15.61:0/1247708647 conn(0x7fe2f80fbcf0 msgr2=0x7fe2f80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:23.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 -- 172.21.15.61:0/1247708647 shutdown_connections 2023-10-18T09:36:23.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.927+0000 7fe2fdbff700 1 -- 172.21.15.61:0/1247708647 wait complete. 2023-10-18T09:36:23.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 Processor -- start 2023-10-18T09:36:23.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 -- start start 2023-10-18T09:36:23.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8100760 0x7fe2f81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 0x7fe2f8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.936 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.928+0000 7fe2fdbff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f81a03d0 con 0x7fe2f8101950 2023-10-18T09:36:23.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2fdbff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f81a0510 con 0x7fe2f81022f0 2023-10-18T09:36:23.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2fdbff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f81a06a0 con 0x7fe2f8100760 2023-10-18T09:36:23.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.937 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f6ffd700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 0x7fe2f8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56282/0 (socket says 172.21.15.61:56282) 2023-10-18T09:36:23.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 -- 172.21.15.61:0/2599287724 learned_addr learned my addr 172.21.15.61:0/2599287724 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:23.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f77fe700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8100760 0x7fe2f81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 -- 172.21.15.61:0/2599287724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8100760 msgr2=0x7fe2f81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.938 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8100760 0x7fe2f81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 -- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 msgr2=0x7fe2f8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 0x7fe2f8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 -- 172.21.15.61:0/2599287724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0095f0 con 0x7fe2f81022f0 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f6ffd700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 0x7fe2f8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.929+0000 7fe2f7fff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000ef30 tx=0x7fe2f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:23.939 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.930+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0018070 con 0x7fe2f81022f0 2023-10-18T09:36:23.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.930+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2f000f040 con 0x7fe2f81022f0 2023-10-18T09:36:23.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.930+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f81a0920 con 0x7fe2f81022f0 2023-10-18T09:36:23.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.930+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000cd00 con 0x7fe2f81022f0 2023-10-18T09:36:23.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.930+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f81a0de0 con 0x7fe2f81022f0 2023-10-18T09:36:23.940 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.931+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f8067530 con 0x7fe2f81022f0 2023-10-18T09:36:23.941 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.933+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7fe2f0003a00 con 0x7fe2f81022f0 2023-10-18T09:36:23.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.934+0000 7fe2f4ff9700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 0x7fe2e00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:23.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.934+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe2f00ccbc0 con 0x7fe2f81022f0 2023-10-18T09:36:23.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.934+0000 7fe2f77fe700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 0x7fe2e00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:23.942 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.935+0000 7fe2f77fe700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 0x7fe2e00a3940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe2f81017b0 tx=0x7fe2e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:23.943 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:23.939+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe2f0095f60 con 0x7fe2f81022f0 2023-10-18T09:36:24.161 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.157+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2f810bfd0 con 0x7fe2f81022f0 2023-10-18T09:36:24.163 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.159+0000 7fe2f4ff9700 1 -- 172.21.15.61:0/2599287724 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24488 (secure 0 0 0) 0x7fe2f0099580 con 0x7fe2f81022f0 2023-10-18T09:36:24.163 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-10-18T09:36:24.163 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":92,"fsid":"c578e0ac-6d95-11ee-8db6-212e2dc638e7","created":"2023-10-18T09:08:20.265998+0000","modified":"2023-10-18T09:18:22.923624+0000","last_up_change":"2023-10-18T09:15:36.900988+0000","last_in_change":"2023-10-18T09:15:19.249856+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-18T09:12:11.367908+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-18T09:16:24.457067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-18T09:16:25.828879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-18T09:16:29.402923+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1a451299-d658-4830-9f80-9ba0776f9122","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6803","nonce":3245715235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6805","nonce":3245715235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6809","nonce":3245715235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":3245715235},{"type":"v1","addr":"172.21.15.61:6807","nonce":3245715235}]},"public_addr":"172.21.15.61:6803/3245715235","cluster_addr":"172.21.15.61:6805/3245715235","heartbeat_back_addr":"172.21.15.61:6809/3245715235","heartbeat_front_addr":"172.21.15.61:6807/3245715235","state":["exists","up"]},{"osd":1,"uuid":"5535c5e8-804e-4d87-915d-00862ef1b386","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6801","nonce":4194494003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6803","nonce":4194494003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6807","nonce":4194494003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":4194494003},{"type":"v1","addr":"172.21.15.110:6805","nonce":4194494003}]},"public_addr":"172.21.15.110:6801/4194494003","cluster_addr":"172.21.15.110:6803/4194494003","heartbeat_back_addr":"172.21.15.110:6807/4194494003","heartbeat_front_addr":"172.21.15.110:6805/4194494003","state":["exists","up"]},{"osd":2,"uuid":"222ac61b-a988-44c1-b86e-b31756c1f539","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6801","nonce":696261473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6803","nonce":696261473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6807","nonce":696261473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":696261473},{"type":"v1","addr":"172.21.15.170:6805","nonce":696261473}]},"public_addr":"172.21.15.170:6801/696261473","cluster_addr":"172.21.15.170:6803/696261473","heartbeat_back_addr":"172.21.15.170:6807/696261473","heartbeat_front_addr":"172.21.15.170:6805/696261473","state":["exists","up"]},{"osd":3,"uuid":"c79af743-8b1a-477c-a7ce-f4845ca1a61a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6811","nonce":671964640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6813","nonce":671964640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6817","nonce":671964640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":671964640},{"type":"v1","addr":"172.21.15.61:6815","nonce":671964640}]},"public_addr":"172.21.15.61:6811/671964640","cluster_addr":"172.21.15.61:6813/671964640","heartbeat_back_addr":"172.21.15.61:6817/671964640","heartbeat_front_addr":"172.21.15.61:6815/671964640","state":["exists","up"]},{"osd":4,"uuid":"e542f6db-5212-4e76-b5d8-05029c585fd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6809","nonce":113917828}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6811","nonce":113917828}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6815","nonce":113917828}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":113917828},{"type":"v1","addr":"172.21.15.110:6813","nonce":113917828}]},"public_addr":"172.21.15.110:6809/113917828","cluster_addr":"172.21.15.110:6811/113917828","heartbeat_back_addr":"172.21.15.110:6815/113917828","heartbeat_front_addr":"172.21.15.110:6813/113917828","state":["exists","up"]},{"osd":5,"uuid":"2069c32b-e740-4ea4-b647-35be7d41f3df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6809","nonce":1107914770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6811","nonce":1107914770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6815","nonce":1107914770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":1107914770},{"type":"v1","addr":"172.21.15.170:6813","nonce":1107914770}]},"public_addr":"172.21.15.170:6809/1107914770","cluster_addr":"172.21.15.170:6811/1107914770","heartbeat_back_addr":"172.21.15.170:6815/1107914770","heartbeat_front_addr":"172.21.15.170:6813/1107914770","state":["exists","up"]},{"osd":6,"uuid":"a152124f-a0de-47d0-97b3-b65467c83915","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6818","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6819","nonce":2592494554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6820","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6821","nonce":2592494554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6824","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6825","nonce":2592494554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6822","nonce":2592494554},{"type":"v1","addr":"172.21.15.61:6823","nonce":2592494554}]},"public_addr":"172.21.15.61:6819/2592494554","cluster_addr":"172.21.15.61:6821/2592494554","heartbeat_back_addr":"172.21.15.61:6825/2592494554","heartbeat_front_addr":"172.21.15.61:6823/2592494554","state":["exists","up"]},{"osd":7,"uuid":"4e523e57-7424-443a-99ba-a6becb55b91a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6817","nonce":3683648719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6819","nonce":3683648719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6823","nonce":3683648719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3683648719},{"type":"v1","addr":"172.21.15.110:6821","nonce":3683648719}]},"public_addr":"172.21.15.110:6817/3683648719","cluster_addr":"172.21.15.110:6819/3683648719","heartbeat_back_addr":"172.21.15.110:6823/3683648719","heartbeat_front_addr":"172.21.15.110:6821/3683648719","state":["exists","up"]},{"osd":8,"uuid":"ff8f0e0e-869f-4e76-a3b5-7f6d43633dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6817","nonce":3321062874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6819","nonce":3321062874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6823","nonce":3321062874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3321062874},{"type":"v1","addr":"172.21.15.170:6821","nonce":3321062874}]},"public_addr":"172.21.15.170:6817/3321062874","cluster_addr":"172.21.15.170:6819/3321062874","heartbeat_back_addr":"172.21.15.170:6823/3321062874","heartbeat_front_addr":"172.21.15.170:6821/3321062874","state":["exists","up"]},{"osd":9,"uuid":"f66eb5b2-0e99-4f40-ad44-4d81f5a6c910","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6826","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6827","nonce":65349699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6828","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6829","nonce":65349699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6832","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6833","nonce":65349699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6830","nonce":65349699},{"type":"v1","addr":"172.21.15.61:6831","nonce":65349699}]},"public_addr":"172.21.15.61:6827/65349699","cluster_addr":"172.21.15.61:6829/65349699","heartbeat_back_addr":"172.21.15.61:6833/65349699","heartbeat_front_addr":"172.21.15.61:6831/65349699","state":["exists","up"]},{"osd":10,"uuid":"0cbca8f8-088f-43d3-bd0e-dcb212e8756e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6825","nonce":508893070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6827","nonce":508893070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6831","nonce":508893070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":508893070},{"type":"v1","addr":"172.21.15.110:6829","nonce":508893070}]},"public_addr":"172.21.15.110:6825/508893070","cluster_addr":"172.21.15.110:6827/508893070","heartbeat_back_addr":"172.21.15.110:6831/508893070","heartbeat_front_addr":"172.21.15.110:6829/508893070","state":["exists","up"]},{"osd":11,"uuid":"4f2b592b-6b55-4c04-b153-da70a63b88f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6825","nonce":4083010574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6827","nonce":4083010574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6831","nonce":4083010574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":4083010574},{"type":"v1","addr":"172.21.15.170:6829","nonce":4083010574}]},"public_addr":"172.21.15.170:6825/4083010574","cluster_addr":"172.21.15.170:6827/4083010574","heartbeat_back_addr":"172.21.15.170:6831/4083010574","heartbeat_front_addr":"172.21.15.170:6829/4083010574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:24.345694+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:11:46.332299+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:07.574607+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:35.041002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:12:56.459680+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:17.427400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:13:43.740392+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:05.085734+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:26.212768+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:14:52.031344+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:14.153632+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-18T09:15:35.023841+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.61:0/3728584080":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/1938489455":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6801/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/2303013428":"2023-10-19T09:08:53.425206+0000","172.21.15.61:6800/1676472102":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/1518173664":"2023-10-19T09:09:20.739745+0000","172.21.15.61:6800/2589487616":"2023-10-19T09:09:20.739745+0000","172.21.15.61:0/2840133191":"2023-10-19T09:08:53.425206+0000","172.21.15.61:0/623274229":"2023-10-19T09:08:53.425206+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-18T09:36:24.165 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 msgr2=0x7fe2e00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 0x7fe2e00a3940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe2f81017b0 tx=0x7fe2e8009260 comp rx=0 tx=0).stop 2023-10-18T09:36:24.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 msgr2=0x7fe2f819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000ef30 tx=0x7fe2f000c3a0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 shutdown_connections 2023-10-18T09:36:24.166 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7fe2e00a1480 0x7fe2e00a3940 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2f81022f0 0x7fe2f819fe00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8100760 0x7fe2f81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 --2- 172.21.15.61:0/2599287724 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7fe2f8101950 0x7fe2f8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 >> 172.21.15.61:0/2599287724 conn(0x7fe2f80fbcf0 msgr2=0x7fe2f8107140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:24.167 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 shutdown_connections 2023-10-18T09:36:24.168 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.162+0000 7fe2fdbff700 1 -- 172.21.15.61:0/2599287724 wait complete. 2023-10-18T09:36:24.185 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-18T09:36:24.394 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.390+0000 7f71841ff700 1 -- 172.21.15.61:0/1689844625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 msgr2=0x7f717c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.390+0000 7f71841ff700 1 --2- 172.21.15.61:0/1689844625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c073670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f716c009990 tx=0x7f716c009ca0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.390+0000 7f71841ff700 1 -- 172.21.15.61:0/1689844625 shutdown_connections 2023-10-18T09:36:24.395 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.390+0000 7f71841ff700 1 --2- 172.21.15.61:0/1689844625 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c1085f0 0x7f717c10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.391+0000 7f71841ff700 1 --2- 172.21.15.61:0/1689844625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c0747d0 0x7f717c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.396 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.391+0000 7f71841ff700 1 --2- 172.21.15.61:0/1689844625 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c073670 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.391+0000 7f71841ff700 1 -- 172.21.15.61:0/1689844625 >> 172.21.15.61:0/1689844625 conn(0x7f717c0fbc90 msgr2=0x7f717c0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:24.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.391+0000 7f71841ff700 1 -- 172.21.15.61:0/1689844625 shutdown_connections 2023-10-18T09:36:24.397 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.391+0000 7f71841ff700 1 -- 172.21.15.61:0/1689844625 wait complete. 2023-10-18T09:36:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.392+0000 7f71841ff700 1 Processor -- start 2023-10-18T09:36:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.392+0000 7f71841ff700 1 -- start start 2023-10-18T09:36:24.402 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c0747d0 0x7f717c198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c1085f0 0x7f717c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c1a03a0 con 0x7f717c0747d0 2023-10-18T09:36:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c1a04e0 con 0x7f717c0731f0 2023-10-18T09:36:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f71841ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c1a0670 con 0x7f717c1085f0 2023-10-18T09:36:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.404 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.61:56308/0 (socket says 172.21.15.61:56308) 2023-10-18T09:36:24.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 -- 172.21.15.61:0/3767106349 learned_addr learned my addr 172.21.15.61:0/3767106349 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:24.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f718179a700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c0747d0 0x7f717c198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.405 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f718279c700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c1085f0 0x7f717c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 -- 172.21.15.61:0/3767106349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c1085f0 msgr2=0x7f717c19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.406 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c1085f0 0x7f717c19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 -- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c0747d0 msgr2=0x7f717c198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c0747d0 0x7f717c198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.393+0000 7f7181f9b700 1 -- 172.21.15.61:0/3767106349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0095f0 con 0x7f717c0731f0 2023-10-18T09:36:24.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f7181f9b700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9a0 tx=0x7f717800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:24.407 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178009790 con 0x7f717c0731f0 2023-10-18T09:36:24.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f717800cbd0 con 0x7f717c0731f0 2023-10-18T09:36:24.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c1a0950 con 0x7f717c0731f0 2023-10-18T09:36:24.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178018680 con 0x7f717c0731f0 2023-10-18T09:36:24.408 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.394+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c1a0e60 con 0x7f717c0731f0 2023-10-18T09:36:24.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.395+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c107400 con 0x7f717c0731f0 2023-10-18T09:36:24.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.397+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f717800cd40 con 0x7f717c0731f0 2023-10-18T09:36:24.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.398+0000 7f7172ffd700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 0x7f71680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.398+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7178014070 con 0x7f717c0731f0 2023-10-18T09:36:24.409 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.398+0000 7f718179a700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 0x7f71680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.410 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.399+0000 7f718179a700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 0x7f71680a3940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f716c00b2e0 tx=0x7f716c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:24.411 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.403+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7178052100 con 0x7f717c0731f0 2023-10-18T09:36:24.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/666471304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/2599287724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/666471304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/2599287724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.664 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.661+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f717c067980 con 0x7f717c0731f0 2023-10-18T09:36:24.674 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.670+0000 7f7172ffd700 1 -- 172.21.15.61:0/3767106349 <== mon.2 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7178096600 con 0x7f717c0731f0 2023-10-18T09:36:24.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 msgr2=0x7f71680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 0x7f71680a3940 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f716c00b2e0 tx=0x7f716c005fd0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 msgr2=0x7f717c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f717800e9a0 tx=0x7f717800ecb0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 shutdown_connections 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71680a1480 0x7f71680a3940 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f717c1085f0 0x7f717c19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f717c0731f0 0x7f717c1988c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 --2- 172.21.15.61:0/3767106349 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f717c0747d0 0x7f717c198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 >> 172.21.15.61:0/3767106349 conn(0x7f717c0fbc90 msgr2=0x7f717c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:24.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 shutdown_connections 2023-10-18T09:36:24.684 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.675+0000 7f71841ff700 1 -- 172.21.15.61:0/3767106349 wait complete. 2023-10-18T09:36:24.684 INFO:teuthology.orchestra.run.smithi061.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-18T09:36:24.700 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-18T09:36:24.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/666471304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/2599287724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-18T09:36:24.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 -- 172.21.15.61:0/2115433718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4101990 msgr2=0x7f71a4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 --2- 172.21.15.61:0/2115433718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4101990 0x7f71a4101df0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7194009920 tx=0x7f7194009c30 comp rx=0 tx=0).stop 2023-10-18T09:36:24.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 -- 172.21.15.61:0/2115433718 shutdown_connections 2023-10-18T09:36:24.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 --2- 172.21.15.61:0/2115433718 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4102330 0x7f71a4108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 --2- 172.21.15.61:0/2115433718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 --2- 172.21.15.61:0/2115433718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4101990 0x7f71a4101df0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 -- 172.21.15.61:0/2115433718 >> 172.21.15.61:0/2115433718 conn(0x7f71a40fbcf0 msgr2=0x7f71a40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:24.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 -- 172.21.15.61:0/2115433718 shutdown_connections 2023-10-18T09:36:24.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.920+0000 7f71abfff700 1 -- 172.21.15.61:0/2115433718 wait complete. 2023-10-18T09:36:24.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.921+0000 7f71abfff700 1 Processor -- start 2023-10-18T09:36:24.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.921+0000 7f71abfff700 1 -- start start 2023-10-18T09:36:24.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4101990 0x7f71a4198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4102330 0x7f71a419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a04f0 con 0x7f71a4101990 2023-10-18T09:36:24.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a0630 con 0x7f71a4102330 2023-10-18T09:36:24.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71abfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a41a07c0 con 0x7f71a41007a0 2023-10-18T09:36:24.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a959a700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4101990 0x7f71a4198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46120/0 (socket says 172.21.15.61:46120) 2023-10-18T09:36:24.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 -- 172.21.15.61:0/3875802315 learned_addr learned my addr 172.21.15.61:0/3875802315 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:24.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71aa59c700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4102330 0x7f71a419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 -- 172.21.15.61:0/3875802315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4102330 msgr2=0x7f71a419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4102330 0x7f71a419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 -- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4101990 msgr2=0x7f71a4198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4101990 0x7f71a4198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.922+0000 7f71a9d9b700 1 -- 172.21.15.61:0/3875802315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194009520 con 0x7f71a41007a0 2023-10-18T09:36:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f71a9d9b700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f71a000e970 tx=0x7f71a000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0009790 con 0x7f71a41007a0 2023-10-18T09:36:24.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a41a0aa0 con 0x7f71a41007a0 2023-10-18T09:36:24.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71a000eec0 con 0x7f71a41007a0 2023-10-18T09:36:24.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0003cb0 con 0x7f71a41007a0 2023-10-18T09:36:24.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.923+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a41a1000 con 0x7f71a41007a0 2023-10-18T09:36:24.932 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.924+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a40675a0 con 0x7f71a41007a0 2023-10-18T09:36:24.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.926+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f71a000cbd0 con 0x7f71a41007a0 2023-10-18T09:36:24.933 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.927+0000 7f719affd700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 0x7f71900a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:24.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.928+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9508+0+0 (secure 0 0 0) 0x7f71a0014070 con 0x7f71a41007a0 2023-10-18T09:36:24.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.928+0000 7f71a959a700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 0x7f71900a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:24.934 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.928+0000 7f71a959a700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 0x7f71900a39c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f719400fa40 tx=0x7f7194019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:24.935 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:24.932+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f71a0096570 con 0x7f71a41007a0 2023-10-18T09:36:25.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a[98549]: 2023-10-18T09:36:24.663+0000 7f9f3fb48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:36:25.004 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.004 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.004 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-mds[150761]: -- 172.21.15.61:0/1979320884 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5565d61451e0 con 0x5565d611d000 2023-10-18T09:36:25.004 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: starting mds.g at 2023-10-18T09:36:25.005 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.005 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.005 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-mds[151149]: -- 172.21.15.61:0/1534585111 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611ca39b1e0 con 0x5611ca372800 2023-10-18T09:36:25.005 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:36:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: starting mds.a at 2023-10-18T09:36:25.142 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.143 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.143 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mds[173303]: -- 172.21.15.170:0/3604440905 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55dc21e0d1e0 con 0x55dc21de2800 2023-10-18T09:36:25.143 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: starting mds.c at 2023-10-18T09:36:25.143 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.144 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.144 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mds[172963]: -- 172.21.15.170:0/543192916 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4c03351e0 con 0x55d4c030ac00 2023-10-18T09:36:25.144 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: starting mds.f at 2023-10-18T09:36:25.144 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.145 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.145 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mds[173641]: -- 172.21.15.170:0/113237139 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a6d92291e0 con 0x55a6d91ff000 2023-10-18T09:36:25.145 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: starting mds.i at 2023-10-18T09:36:25.145 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.146 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.146 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-mds[172622]: -- 172.21.15.170:0/2473602126 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563de70491e0 con 0x563de701e800 2023-10-18T09:36:25.146 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:36:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: starting mds.l at 2023-10-18T09:36:25.193 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.190+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71a41a1270 con 0x7f71a41007a0 2023-10-18T09:36:25.212 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.212 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.212 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mds[171828]: -- 172.21.15.110:0/3319986002 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560935b911e0 con 0x560935b66c00 2023-10-18T09:36:25.213 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: starting mds.b at 2023-10-18T09:36:25.213 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.213 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.214 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mds[171135]: -- 172.21.15.110:0/1105577158 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d9862a11e0 con 0x55d986276800 2023-10-18T09:36:25.214 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: starting mds.e at 2023-10-18T09:36:25.215 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.215 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.215 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mds[171478]: -- 172.21.15.110:0/2952235175 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556e128591e0 con 0x556e1282ec00 2023-10-18T09:36:25.215 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: starting mds.h at 2023-10-18T09:36:25.216 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: ignoring --setuser ceph since I am not root 2023-10-18T09:36:25.216 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: ignoring --setgroup ceph since I am not root 2023-10-18T09:36:25.216 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-mds[170796]: -- 172.21.15.110:0/202466101 <== mon.2 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56160e50f1e0 con 0x56160e4e4c00 2023-10-18T09:36:25.216 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:36:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: starting mds.k at 2023-10-18T09:36:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:36:25.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3767106349' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T09:36:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:36:25.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T09:36:25.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T09:36:25.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-18T09:36:25.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3875802315' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:25 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:36:25.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3767106349' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T09:36:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:36:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T09:36:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T09:36:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-18T09:36:25.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3875802315' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:25 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.672+0000 7f719affd700 1 -- 172.21.15.61:0/3875802315 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71a0099b90 con 0x7f71a41007a0 2023-10-18T09:36:25.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 msgr2=0x7f71900a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:25.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 0x7f71900a39c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f719400fa40 tx=0x7f7194019040 comp rx=0 tx=0).stop 2023-10-18T09:36:25.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 msgr2=0x7f71a4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:25.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f71a000e970 tx=0x7f71a000ec80 comp rx=0 tx=0).stop 2023-10-18T09:36:25.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 shutdown_connections 2023-10-18T09:36:25.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f71900a1500 0x7f71900a39c0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f719400fa40 tx=0x7f7194019040 comp rx=0 tx=0).stop 2023-10-18T09:36:25.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4102330 0x7f71a419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a41007a0 0x7f71a4198a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.675+0000 7f71abfff700 1 --2- 172.21.15.61:0/3875802315 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f71a4101990 0x7f71a4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.676+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 >> 172.21.15.61:0/3875802315 conn(0x7f71a40fbcf0 msgr2=0x7f71a4107200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:25.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.676+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 shutdown_connections 2023-10-18T09:36:25.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.676+0000 7f71abfff700 1 -- 172.21.15.61:0/3875802315 wait complete. 2023-10-18T09:36:25.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-18T09:36:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: pgmap v893: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 10 op/s 2023-10-18T09:36:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3767106349' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-18T09:36:25.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-18T09:36:25.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-18T09:36:25.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-18T09:36:25.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: osdmap e93: 12 total, 12 up, 12 in 2023-10-18T09:36:25.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-10-18T09:36:25.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3875802315' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:25 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-18T09:36:25.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 -- 172.21.15.61:0/3197881506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c1007a0 msgr2=0x7f775c100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:25.915 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 --2- 172.21.15.61:0/3197881506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c1007a0 0x7f775c100bc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7758009920 tx=0x7f7758009c30 comp rx=0 tx=0).stop 2023-10-18T09:36:25.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 -- 172.21.15.61:0/3197881506 shutdown_connections 2023-10-18T09:36:25.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 --2- 172.21.15.61:0/3197881506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c102330 0x7f775c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 --2- 172.21.15.61:0/3197881506 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c1007a0 0x7f775c100bc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 --2- 172.21.15.61:0/3197881506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c101990 0x7f775c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.916 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 -- 172.21.15.61:0/3197881506 >> 172.21.15.61:0/3197881506 conn(0x7f775c0fbcf0 msgr2=0x7f775c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:25.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 -- 172.21.15.61:0/3197881506 shutdown_connections 2023-10-18T09:36:25.917 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.911+0000 7f7763fff700 1 -- 172.21.15.61:0/3197881506 wait complete. 2023-10-18T09:36:25.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.912+0000 7f7763fff700 1 Processor -- start 2023-10-18T09:36:25.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.912+0000 7f7763fff700 1 -- start start 2023-10-18T09:36:25.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c1007a0 0x7f775c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:25.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c101990 0x7f775c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:25.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:25.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a0560 con 0x7f775c102330 2023-10-18T09:36:25.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a06a0 con 0x7f775c101990 2023-10-18T09:36:25.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7763fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c1a0830 con 0x7f775c1007a0 2023-10-18T09:36:25.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:25.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45376/0 (socket says 172.21.15.61:45376) 2023-10-18T09:36:25.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 -- 172.21.15.61:0/3040519272 learned_addr learned my addr 172.21.15.61:0/3040519272 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:25.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f7761d9b700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c1007a0 0x7f775c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:25.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776159a700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c101990 0x7f775c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:25.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 -- 172.21.15.61:0/3040519272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c1007a0 msgr2=0x7f775c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:25.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c1007a0 0x7f775c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 -- 172.21.15.61:0/3040519272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c101990 msgr2=0x7f775c198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:25.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c101990 0x7f775c198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:25.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.913+0000 7f776259c700 1 -- 172.21.15.61:0/3040519272 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0095f0 con 0x7f775c102330 2023-10-18T09:36:25.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f776259c700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f775400ef30 tx=0x7f775400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:25.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754018070 con 0x7f775c102330 2023-10-18T09:36:25.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f775400f040 con 0x7f775c102330 2023-10-18T09:36:25.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758009520 con 0x7f775c102330 2023-10-18T09:36:25.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400cd00 con 0x7f775c102330 2023-10-18T09:36:25.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.914+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c1a0de0 con 0x7f775c102330 2023-10-18T09:36:25.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.915+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775c0675a0 con 0x7f775c102330 2023-10-18T09:36:25.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.919+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7754003a00 con 0x7f775c102330 2023-10-18T09:36:25.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.921+0000 7f7752ffd700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 0x7f77480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:25.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.921+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9508+0+0 (secure 0 0 0) 0x7f77540cd760 con 0x7f775c102330 2023-10-18T09:36:25.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.921+0000 7f7761d9b700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 0x7f77480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:25.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.922+0000 7f7761d9b700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 0x7f77480a3940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f77580098f0 tx=0x7f7758019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:25.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:25.923+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77540968f0 con 0x7f775c102330 2023-10-18T09:36:26.137 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.134+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f775c10c000 con 0x7f775c102330 2023-10-18T09:36:26.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T09:36:26.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T09:36:26.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: Cluster is now healthy 2023-10-18T09:36:26.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:36:26.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6836/3908580594,v1:172.21.15.170:6838/3908580594] up:boot 2023-10-18T09:36:26.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6837/1615307998,v1:172.21.15.170:6839/1615307998] up:boot 2023-10-18T09:36:26.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6832/735482100,v1:172.21.15.170:6834/735482100] up:boot 2023-10-18T09:36:26.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.170:6833/2289743489,v1:172.21.15.170:6835/2289743489] up:boot 2023-10-18T09:36:26.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6837/3424383485,v1:172.21.15.110:6839/3424383485] up:boot 2023-10-18T09:36:26.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6836/2604599545,v1:172.21.15.110:6838/2604599545] up:boot 2023-10-18T09:36:26.505 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6833/1410781575,v1:172.21.15.110:6835/1410781575] up:boot 2023-10-18T09:36:26.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.110:6832/366894393,v1:172.21.15.110:6834/366894393] up:boot 2023-10-18T09:36:26.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6838/3027923255,v1:172.21.15.61:6839/3027923255] up:boot 2023-10-18T09:36:26.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: mds.? [v2:172.21.15.61:6840/3658675362,v1:172.21.15.61:6841/3658675362] up:boot 2023-10-18T09:36:26.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: fsmap 12 up:standby 2023-10-18T09:36:26.506 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:36:26.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:36:26.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:36:26.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:36:26.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:36:26.507 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:36:26.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:36:26.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:36:26.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:36:26.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:36:26.508 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:26 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:26.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T09:36:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T09:36:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: Cluster is now healthy 2023-10-18T09:36:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:36:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6836/3908580594,v1:172.21.15.170:6838/3908580594] up:boot 2023-10-18T09:36:26.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6837/1615307998,v1:172.21.15.170:6839/1615307998] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6832/735482100,v1:172.21.15.170:6834/735482100] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.170:6833/2289743489,v1:172.21.15.170:6835/2289743489] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6837/3424383485,v1:172.21.15.110:6839/3424383485] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6836/2604599545,v1:172.21.15.110:6838/2604599545] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6833/1410781575,v1:172.21.15.110:6835/1410781575] up:boot 2023-10-18T09:36:26.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.110:6832/366894393,v1:172.21.15.110:6834/366894393] up:boot 2023-10-18T09:36:26.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6838/3027923255,v1:172.21.15.61:6839/3027923255] up:boot 2023-10-18T09:36:26.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: mds.? [v2:172.21.15.61:6840/3658675362,v1:172.21.15.61:6841/3658675362] up:boot 2023-10-18T09:36:26.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: fsmap 12 up:standby 2023-10-18T09:36:26.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:36:26.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:36:26.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:36:26.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:36:26.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:36:26.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:36:26.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:36:26.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:36:26.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:36:26.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:36:26.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:26 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:26.673 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.670+0000 7f7752ffd700 1 -- 172.21.15.61:0/3040519272 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7754099f10 con 0x7f775c102330 2023-10-18T09:36:26.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 msgr2=0x7f77480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:26.678 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 0x7f77480a3940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f77580098f0 tx=0x7f7758019040 comp rx=0 tx=0).stop 2023-10-18T09:36:26.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 msgr2=0x7f775c19ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:26.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f775400ef30 tx=0x7f775400c3a0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 shutdown_connections 2023-10-18T09:36:26.679 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f77480a1480 0x7f77480a3940 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f775c102330 0x7f775c19ff90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f775c1007a0 0x7f775c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 --2- 172.21.15.61:0/3040519272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f775c101990 0x7f775c198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.674+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 >> 172.21.15.61:0/3040519272 conn(0x7f775c0fbcf0 msgr2=0x7f775c107180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:26.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.676+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 shutdown_connections 2023-10-18T09:36:26.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.677+0000 7f7763fff700 1 -- 172.21.15.61:0/3040519272 wait complete. 2023-10-18T09:36:26.682 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_metadata' removed 2023-10-18T09:36:26.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-18T09:36:26.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-18T09:36:26.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-18T09:36:26.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: Cluster is now healthy 2023-10-18T09:36:26.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-18T09:36:26.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6836/3908580594,v1:172.21.15.170:6838/3908580594] up:boot 2023-10-18T09:36:26.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6837/1615307998,v1:172.21.15.170:6839/1615307998] up:boot 2023-10-18T09:36:26.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6832/735482100,v1:172.21.15.170:6834/735482100] up:boot 2023-10-18T09:36:26.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.170:6833/2289743489,v1:172.21.15.170:6835/2289743489] up:boot 2023-10-18T09:36:26.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6837/3424383485,v1:172.21.15.110:6839/3424383485] up:boot 2023-10-18T09:36:26.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6836/2604599545,v1:172.21.15.110:6838/2604599545] up:boot 2023-10-18T09:36:26.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6833/1410781575,v1:172.21.15.110:6835/1410781575] up:boot 2023-10-18T09:36:26.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.110:6832/366894393,v1:172.21.15.110:6834/366894393] up:boot 2023-10-18T09:36:26.714 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6838/3027923255,v1:172.21.15.61:6839/3027923255] up:boot 2023-10-18T09:36:26.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: mds.? [v2:172.21.15.61:6840/3658675362,v1:172.21.15.61:6841/3658675362] up:boot 2023-10-18T09:36:26.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: fsmap 12 up:standby 2023-10-18T09:36:26.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-18T09:36:26.715 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-18T09:36:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-18T09:36:26.717 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-18T09:36:26.718 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-18T09:36:26.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-18T09:36:26.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-18T09:36:26.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-18T09:36:26.719 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-18T09:36:26.720 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='mgr.14150 172.21.15.61:0/1270738753' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-18T09:36:26.720 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:26 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:26.918 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.914+0000 7f6d303ff700 1 -- 172.21.15.61:0/362848323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 msgr2=0x7f6d28100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:26.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.914+0000 7f6d303ff700 1 --2- 172.21.15.61:0/362848323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28100bc0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d240099c0 tx=0x7f6d24009cd0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 -- 172.21.15.61:0/362848323 shutdown_connections 2023-10-18T09:36:26.919 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 --2- 172.21.15.61:0/362848323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 0x7f6d28108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 --2- 172.21.15.61:0/362848323 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28100bc0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 --2- 172.21.15.61:0/362848323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 0x7f6d28101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 -- 172.21.15.61:0/362848323 >> 172.21.15.61:0/362848323 conn(0x7f6d280fbcf0 msgr2=0x7f6d280fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:26.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 -- 172.21.15.61:0/362848323 shutdown_connections 2023-10-18T09:36:26.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.915+0000 7f6d303ff700 1 -- 172.21.15.61:0/362848323 wait complete. 2023-10-18T09:36:26.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 Processor -- start 2023-10-18T09:36:26.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 -- start start 2023-10-18T09:36:26.923 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:26.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 0x7f6d28072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:26.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 0x7f6d28073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:26.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280738c0 con 0x7f6d281007a0 2023-10-18T09:36:26.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28073a30 con 0x7f6d28102330 2023-10-18T09:36:26.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d303ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281a07b0 con 0x7f6d28101990 2023-10-18T09:36:26.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.916+0000 7f6d2e19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:26.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.61:3300/0 says I am v2:172.21.15.61:45388/0 (socket says 172.21.15.61:45388) 2023-10-18T09:36:26.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 -- 172.21.15.61:0/1452135656 learned_addr learned my addr 172.21.15.61:0/1452135656 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:26.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2d99a700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 0x7f6d28072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:26.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e99c700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 0x7f6d28073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:26.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 -- 172.21.15.61:0/1452135656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 msgr2=0x7f6d28072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:26.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 0x7f6d28072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 -- 172.21.15.61:0/1452135656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 msgr2=0x7f6d28073120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:26.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 0x7f6d28073120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:26.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 -- 172.21.15.61:0/1452135656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240095f0 con 0x7f6d281007a0 2023-10-18T09:36:26.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d2e19b700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003730 tx=0x7f6d2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:26.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2401d070 con 0x7f6d281007a0 2023-10-18T09:36:26.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d24005050 con 0x7f6d281007a0 2023-10-18T09:36:26.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d281a09e0 con 0x7f6d281007a0 2023-10-18T09:36:26.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.917+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24004a50 con 0x7f6d281007a0 2023-10-18T09:36:26.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.918+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d281a0e00 con 0x7f6d281007a0 2023-10-18T09:36:26.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.919+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d28107bc0 con 0x7f6d281007a0 2023-10-18T09:36:26.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.921+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f6d240051c0 con 0x7f6d281007a0 2023-10-18T09:36:26.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.922+0000 7f6d1f7fe700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 0x7f6d140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:26.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.922+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8983+0+0 (secure 0 0 0) 0x7f6d240ce120 con 0x7f6d281007a0 2023-10-18T09:36:26.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.922+0000 7f6d2d99a700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 0x7f6d140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:26.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.923+0000 7f6d2d99a700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 0x7f6d140a39c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009c60 tx=0x7f6d18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:26.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:26.924+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6d240974c0 con 0x7f6d281007a0 2023-10-18T09:36:27.141 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.138+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d280675a0 con 0x7f6d281007a0 2023-10-18T09:36:27.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:27 smithi061 ceph-mon[98573]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-18T09:36:27.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:27 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:27.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:27 smithi061 ceph-mon[98573]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T09:36:27.504 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:27 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:27.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:27 smithi170 ceph-mon[135150]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-18T09:36:27.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:27 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:27.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:27 smithi170 ceph-mon[135150]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T09:36:27.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:27 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:27.675 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.672+0000 7f6d1f7fe700 1 -- 172.21.15.61:0/1452135656 <== mon.0 v2:172.21.15.61:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6d2409aae0 con 0x7f6d281007a0 2023-10-18T09:36:27.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 msgr2=0x7f6d140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:27.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 0x7f6d140a39c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009c60 tx=0x7f6d18009260 comp rx=0 tx=0).stop 2023-10-18T09:36:27.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 msgr2=0x7f6d28074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:27.680 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003730 tx=0x7f6d2400f690 comp rx=0 tx=0).stop 2023-10-18T09:36:27.681 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 shutdown_connections 2023-10-18T09:36:27.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f6d140a1500 0x7f6d140a39c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d28102330 0x7f6d28073120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.682 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f6d281007a0 0x7f6d28074550 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 --2- 172.21.15.61:0/1452135656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d28101990 0x7f6d28072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 >> 172.21.15.61:0/1452135656 conn(0x7f6d280fbcf0 msgr2=0x7f6d28104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:27.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 shutdown_connections 2023-10-18T09:36:27.683 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.677+0000 7f6d303ff700 1 -- 172.21.15.61:0/1452135656 wait complete. 2023-10-18T09:36:27.684 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_data' removed 2023-10-18T09:36:27.706 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-18T09:36:27.712 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:27 smithi110 ceph-mon[134818]: pgmap v895: 129 pgs: 129 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2023-10-18T09:36:27.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:27 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/3040519272' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:27.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:27 smithi110 ceph-mon[134818]: osdmap e94: 12 total, 12 up, 12 in 2023-10-18T09:36:27.713 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:27 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 -- 172.21.15.61:0/2515890747 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78102330 msgr2=0x7f7b78108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:27.920 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/2515890747 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78102330 0x7f7b78108be0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000b0d0 tx=0x7f7b7000b3e0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 -- 172.21.15.61:0/2515890747 shutdown_connections 2023-10-18T09:36:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/2515890747 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78102330 0x7f7b78108be0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/2515890747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 0x7f7b78100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.921 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/2515890747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78101990 0x7f7b78101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.916+0000 7f7b7cdff700 1 -- 172.21.15.61:0/2515890747 >> 172.21.15.61:0/2515890747 conn(0x7f7b780fbcf0 msgr2=0x7f7b780fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.917+0000 7f7b7cdff700 1 -- 172.21.15.61:0/2515890747 shutdown_connections 2023-10-18T09:36:27.922 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.917+0000 7f7b7cdff700 1 -- 172.21.15.61:0/2515890747 wait complete. 2023-10-18T09:36:27.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.917+0000 7f7b7cdff700 1 Processor -- start 2023-10-18T09:36:27.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 -- start start 2023-10-18T09:36:27.924 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 0x7f7b78196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78101990 0x7f7b78196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 -- --> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7819e360 con 0x7f7b78101990 2023-10-18T09:36:27.925 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7819e4a0 con 0x7f7b781007a0 2023-10-18T09:36:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7cdff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7819e630 con 0x7f7b78102330 2023-10-18T09:36:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b76d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b75d9b700 1 --2- >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78101990 0x7f7b78196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b76d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.61:46176/0 (socket says 172.21.15.61:46176) 2023-10-18T09:36:27.926 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b76d9d700 1 -- 172.21.15.61:0/4119990777 learned_addr learned my addr 172.21.15.61:0/4119990777 (peer_addr_for_me v2:172.21.15.61:0/0) 2023-10-18T09:36:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.918+0000 7f7b7659c700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 0x7f7b78196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 -- 172.21.15.61:0/4119990777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 msgr2=0x7f7b78196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 0x7f7b78196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 -- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78101990 msgr2=0x7f7b78196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:27.927 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78101990 0x7f7b78196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 -- 172.21.15.61:0/4119990777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009d00 con 0x7f7b78102330 2023-10-18T09:36:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b76d9d700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b70007b60 tx=0x7f7b70008e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000e040 con 0x7f7b78102330 2023-10-18T09:36:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.919+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7819e8b0 con 0x7f7b78102330 2023-10-18T09:36:27.928 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.920+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7819ed80 con 0x7f7b78102330 2023-10-18T09:36:27.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.920+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7001d930 con 0x7f7b78102330 2023-10-18T09:36:27.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.920+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7001ba10 con 0x7f7b78102330 2023-10-18T09:36:27.929 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.920+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78107b70 con 0x7f7b78102330 2023-10-18T09:36:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.923+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) v1 ==== 136003+0+0 (secure 0 0 0) 0x7f7b7001daa0 con 0x7f7b78102330 2023-10-18T09:36:27.930 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.924+0000 7f7b677fe700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 0x7f7b600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-18T09:36:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.924+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f7b700ce350 con 0x7f7b78102330 2023-10-18T09:36:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.924+0000 7f7b7659c700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 0x7f7b600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-18T09:36:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.925+0000 7f7b7659c700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 0x7f7b600a3940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b781017f0 tx=0x7f7b6800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-18T09:36:27.931 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:27.926+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7b70097920 con 0x7f7b78102330 2023-10-18T09:36:28.144 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.140+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b780675a0 con 0x7f7b78102330 2023-10-18T09:36:28.689 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.686+0000 7f7b677fe700 1 -- 172.21.15.61:0/4119990777 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) v1 ==== 178+0+0 (secure 0 0 0) 0x7f7b7009af40 con 0x7f7b78102330 2023-10-18T09:36:28.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 msgr2=0x7f7b600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:28.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 0x7f7b600a3940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b781017f0 tx=0x7f7b6800b410 comp rx=0 tx=0).stop 2023-10-18T09:36:28.694 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 msgr2=0x7f7b7819dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-18T09:36:28.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7b70007b60 tx=0x7f7b70008e90 comp rx=0 tx=0).stop 2023-10-18T09:36:28.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 shutdown_connections 2023-10-18T09:36:28.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:6800/3155127227,v1:172.21.15.61:6801/3155127227] conn(0x7f7b600a1480 0x7f7b600a3940 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7b781017f0 tx=0x7f7b6800b410 comp rx=0 tx=0).stop 2023-10-18T09:36:28.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7b78102330 0x7f7b7819dd90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:28.695 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7b781007a0 0x7f7b78196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:28.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 --2- 172.21.15.61:0/4119990777 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f7b78101990 0x7f7b78196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-18T09:36:28.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.690+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 >> 172.21.15.61:0/4119990777 conn(0x7f7b780fbcf0 msgr2=0x7f7b78104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-18T09:36:28.696 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.693+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 shutdown_connections 2023-10-18T09:36:28.697 INFO:teuthology.orchestra.run.smithi061.stderr:2023-10-18T09:36:28.693+0000 7f7b7cdff700 1 -- 172.21.15.61:0/4119990777 wait complete. 2023-10-18T09:36:28.699 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'cephfs_data_ec' removed 2023-10-18T09:36:28.724 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:36:28.827 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:36:28.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:36:28.908 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-18T09:36:28.908 DEBUG:teuthology.orchestra.run.smithi061:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T09:36:28.962 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:28 smithi110 ceph-mon[134818]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:36:28.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:28.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:28 smithi110 ceph-mon[134818]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T09:36:28.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:28 smithi110 ceph-mon[134818]: from='client.? 172.21.15.61:0/4119990777' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:28.963 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:28 smithi110 ceph-mon[134818]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:28.976 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T09:36:28.994 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-18T09:36:29.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:28 smithi061 ceph-mon[98573]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:36:29.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:29.003 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:28 smithi061 ceph-mon[98573]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T09:36:29.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:28 smithi061 ceph-mon[98573]: from='client.? 172.21.15.61:0/4119990777' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:29.004 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:28 smithi061 ceph-mon[98573]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:29.011 INFO:tasks.cephadm:Stopping all daemons... 2023-10-18T09:36:29.012 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-18T09:36:29.012 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a 2023-10-18T09:36:29.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:28 smithi170 ceph-mon[135150]: pgmap v897: 97 pgs: 97 active+clean; 1.5 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-18T09:36:29.142 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/1452135656' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-18T09:36:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:28 smithi170 ceph-mon[135150]: osdmap e95: 12 total, 12 up, 12 in 2023-10-18T09:36:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:28 smithi170 ceph-mon[135150]: from='client.? 172.21.15.61:0/4119990777' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:29.143 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:28 smithi170 ceph-mon[135150]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-18T09:36:29.503 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:29 smithi061 systemd[1]: Stopping Ceph mon.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:29.949 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a[98549]: 2023-10-18T09:36:29.579+0000 7f9f45353700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:29.949 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a[98549]: 2023-10-18T09:36:29.579+0000 7f9f45353700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-18T09:36:30.253 INFO:journalctl@ceph.mon.a.smithi061.stdout:Oct 18 09:36:30 smithi061 bash[178486]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-a 2023-10-18T09:36:30.492 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.a.service' 2023-10-18T09:36:30.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:30.604 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-18T09:36:30.604 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-18T09:36:30.605 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.b 2023-10-18T09:36:30.965 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:30 smithi110 systemd[1]: Stopping Ceph mon.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:31.462 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-b[134794]: 2023-10-18T09:36:31.159+0000 7f0a181d7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:31.463 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-b[134794]: 2023-10-18T09:36:31.159+0000 7f0a181d7700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-18T09:36:31.844 INFO:journalctl@ceph.mon.b.smithi110.stdout:Oct 18 09:36:31 smithi110 bash[177453]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-b 2023-10-18T09:36:31.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.b.service' 2023-10-18T09:36:31.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:31.896 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-18T09:36:31.897 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-18T09:36:31.897 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.c 2023-10-18T09:36:32.392 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:31 smithi170 systemd[1]: Stopping Ceph mon.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:32.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-c[135105]: 2023-10-18T09:36:32.608+0000 7f4ea6016700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:32.892 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-c[135105]: 2023-10-18T09:36:32.608+0000 7f4ea6016700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-18T09:36:34.108 INFO:journalctl@ceph.mon.c.smithi170.stdout:Oct 18 09:36:34 smithi170 bash[179614]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mon-c 2023-10-18T09:36:34.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mon.c.service' 2023-10-18T09:36:34.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:34.895 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-18T09:36:34.895 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-18T09:36:34.896 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x 2023-10-18T09:36:35.253 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:36:34 smithi061 systemd[1]: Stopping Ceph mgr.x for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:35.707 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:36:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:36:35.405+0000 7fb190d27700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:35.707 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:36:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x[98818]: 2023-10-18T09:36:35.405+0000 7fb190d27700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-18T09:36:36.003 INFO:journalctl@ceph.mgr.x.smithi061.stdout:Oct 18 09:36:35 smithi061 bash[178633]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-x 2023-10-18T09:36:36.359 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.x.service' 2023-10-18T09:36:36.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:36.450 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-18T09:36:36.451 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-18T09:36:36.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.y 2023-10-18T09:36:36.892 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:36:36 smithi110 systemd[1]: Stopping Ceph mgr.y for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:37.159 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:36:37 smithi110 bash[177609]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-y 2023-10-18T09:36:37.341 INFO:journalctl@ceph.mgr.y.smithi110.stdout:Oct 18 09:36:37 smithi110 systemd[1]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-18T09:36:37.345 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.y.service' 2023-10-18T09:36:37.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:37.383 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-18T09:36:37.384 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-18T09:36:37.384 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.z 2023-10-18T09:36:37.830 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:36:37 smithi170 systemd[1]: Stopping Ceph mgr.z for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:38.091 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:36:38 smithi170 bash[179770]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mgr-z 2023-10-18T09:36:38.274 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Oct 18 09:36:38 smithi170 systemd[1]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-18T09:36:38.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mgr.z.service' 2023-10-18T09:36:38.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:38.315 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-18T09:36:38.316 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-18T09:36:38.316 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.0 2023-10-18T09:36:38.753 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:36:38 smithi061 systemd[1]: Stopping Ceph osd.0 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:40.253 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:36:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0[117726]: 2023-10-18T09:36:39.948+0000 7fc5b1b41700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:40.253 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:36:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0[117726]: 2023-10-18T09:36:39.948+0000 7fc5b1b41700 -1 osd.0 96 *** Got signal Terminated *** 2023-10-18T09:36:40.254 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:36:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0[117726]: 2023-10-18T09:36:39.948+0000 7fc5b1b41700 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:36:46.503 INFO:journalctl@ceph.osd.0.smithi061.stdout:Oct 18 09:36:46 smithi061 bash[178779]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-0 2023-10-18T09:36:48.300 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.0.service' 2023-10-18T09:36:48.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:48.394 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-18T09:36:48.395 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-18T09:36:48.395 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.1 2023-10-18T09:36:48.712 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:36:48 smithi110 systemd[1]: Stopping Ceph osd.1 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:49.212 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:36:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:36:48.762+0000 7f2908f41700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:49.212 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:36:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:36:48.762+0000 7f2908f41700 -1 osd.1 96 *** Got signal Terminated *** 2023-10-18T09:36:49.213 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:36:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1[139897]: 2023-10-18T09:36:48.762+0000 7f2908f41700 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:36:54.962 INFO:journalctl@ceph.osd.1.smithi110.stdout:Oct 18 09:36:54 smithi110 bash[177757]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-1 2023-10-18T09:36:56.199 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.1.service' 2023-10-18T09:36:56.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:36:56.239 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-18T09:36:56.240 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-18T09:36:56.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.2 2023-10-18T09:36:56.612 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:36:56 smithi170 systemd[1]: Stopping Ceph osd.2 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:36:56.892 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:36:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:36:56.609+0000 7feca9597700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:36:56.892 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:36:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:36:56.609+0000 7feca9597700 -1 osd.2 96 *** Got signal Terminated *** 2023-10-18T09:36:56.892 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:36:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2[140915]: 2023-10-18T09:36:56.609+0000 7feca9597700 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:02.892 INFO:journalctl@ceph.osd.2.smithi170.stdout:Oct 18 09:37:02 smithi170 bash[179919]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-2 2023-10-18T09:37:04.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.2.service' 2023-10-18T09:37:04.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:04.178 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-18T09:37:04.178 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-18T09:37:04.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.3 2023-10-18T09:37:04.503 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:04 smithi061 systemd[1]: Stopping Ceph osd.3 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:05.003 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:04 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:37:04.709+0000 7ff4d05a7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:05.003 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:04 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:37:04.709+0000 7ff4d05a7700 -1 osd.3 96 *** Got signal Terminated *** 2023-10-18T09:37:05.003 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:04 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:37:04.709+0000 7ff4d05a7700 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:07.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:07.612+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:08.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:08.082+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:08.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:08.618+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:09.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:09.053+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:09.503 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:09 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3[125720]: 2023-10-18T09:37:09.227+0000 7ff4cab99700 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.509318+0000 front 2023-10-18T09:36:44.508938+0000 (oldest deadline 2023-10-18T09:37:09.109755+0000) 2023-10-18T09:37:10.003 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:09 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:09.533+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:10.063 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:09.662+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:10.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:10.060+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:10.790 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:10 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:10.516+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:11.079 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:10.642+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:11.253 INFO:journalctl@ceph.osd.3.smithi061.stdout:Oct 18 09:37:10 smithi061 bash[179060]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-3 2023-10-18T09:37:11.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:11.076+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:11.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:10 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:10.982+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:11.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:11 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:11.473+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:12.090 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:11.673+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:12.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:11 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:11.939+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:12.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:12.088+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:12.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:12 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:12.463+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:12.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:12.637+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:12.967 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.3.service' 2023-10-18T09:37:13.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:13.060 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-18T09:37:13.060 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-18T09:37:13.060 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.4 2023-10-18T09:37:13.212 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:13 smithi110 systemd[1]: Stopping Ceph osd.4 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:13.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:12 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:12.945+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:13.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:13.112+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:13.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:13 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:13.426+0000 7f1326384700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:13.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:13 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:13.426+0000 7f1326384700 -1 osd.4 96 *** Got signal Terminated *** 2023-10-18T09:37:13.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:13 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:13.426+0000 7f1326384700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:13.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:13 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:13.471+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:13.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:13.604+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:14.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:13 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:13.952+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:14.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:14.119+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:14.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:14 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:14.476+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:14.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:14.613+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:15.390 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:14 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:14.970+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:15.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:15 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:15.119+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:15.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:15 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:15.388+0000 7f131e972700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:51.211867+0000 front 2023-10-18T09:36:51.211906+0000 (oldest deadline 2023-10-18T09:37:15.312013+0000) 2023-10-18T09:37:15.735 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:15 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:15.476+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:15.736 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:15 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:15.476+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:15.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:15 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:15.577+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:15.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:15 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:15.723+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:16.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:15 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:15.733+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:16.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:15 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:15.987+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:16.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:15 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:15.811+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:16.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:16 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:16.080+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:16.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:16 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:16.422+0000 7f131e972700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:51.211867+0000 front 2023-10-18T09:36:51.211906+0000 (oldest deadline 2023-10-18T09:37:15.312013+0000) 2023-10-18T09:37:16.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:16.713+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:16.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:16.492+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:16.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:16 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:16.492+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:16.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:16 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:16.677+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:16.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:16 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:16.582+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:17.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:16 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:16.797+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:17.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:16 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:16.985+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:17.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:17 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:17.088+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:17.827 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:17 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4[147828]: 2023-10-18T09:37:17.464+0000 7f131e972700 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:51.211867+0000 front 2023-10-18T09:36:51.211906+0000 (oldest deadline 2023-10-18T09:37:15.312013+0000) 2023-10-18T09:37:17.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:17 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:17.681+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:17.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:17 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:17.564+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:18.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:17.678+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:18.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:17.502+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:18.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:17.502+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:18.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:17 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:17.825+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:18.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:18 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:18.003+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:18.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:18 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:18.003+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:18.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:18 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:18.058+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:18.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:18.477+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:18.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:18.477+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:18.754 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:18 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:18.692+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:18.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:18 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:18.649+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:18.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:18 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:18.564+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:19.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:18 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:18.866+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:19.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:19 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:19.019+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:19.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:19 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:19.019+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:19.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:19 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:19.039+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:19.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:19 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:19.039+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:19.712 INFO:journalctl@ceph.osd.4.smithi110.stdout:Oct 18 09:37:19 smithi110 bash[178026]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-4 2023-10-18T09:37:19.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:19 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:19.674+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:19.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:19 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:19.586+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:20.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:19.658+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:20.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:19.508+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:20.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:19 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:19.508+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:20.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:19 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:19.844+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:20.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:19 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:19.998+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:20.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:19 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:19.998+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:20.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:20.015+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:20.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:20.015+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:20.662 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.4.service' 2023-10-18T09:37:20.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:20.705 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-18T09:37:20.705 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-18T09:37:20.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.5 2023-10-18T09:37:20.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:20 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:20.617+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:20.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:20 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:20.467+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:20.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:20 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:20.467+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:20.892 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:20 smithi170 systemd[1]: Stopping Ceph osd.5 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:20.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:20.672+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:20.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:20.544+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:20.960 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:20 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:20.796+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:21.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:20 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:20.958+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:21.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:20 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:20.958+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:21.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:20.995+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:21.393 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:20 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:20.995+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:21.393 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:21 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:21.070+0000 7faa48f84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:21.393 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:21 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:21.070+0000 7faa48f84700 -1 osd.5 96 *** Got signal Terminated *** 2023-10-18T09:37:21.393 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:21 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:21.070+0000 7faa48f84700 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:21.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:21 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:21.684+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:21.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:21 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:21.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:22.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:21 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:21.661+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:22.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:21 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:21.509+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:22.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:21 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:21.509+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:22.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:21 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:21.823+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:22.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:21 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:21.926+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:22.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:21 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:21.926+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:22.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:22 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:22.038+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:22.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:22 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:22.038+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:22.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:22 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:22.721+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:22.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:22 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:22.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:23.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:22 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:22.637+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:23.003 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:22 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:22.516+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:23.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:22 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:22.516+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:23.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:22 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:22.804+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:23.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:22 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:22.891+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:23.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:22 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:22.891+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:23.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:23.011+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:23.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:23.011+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:23.733 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:23.455+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:23.734 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:23.455+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:23.988 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:23.730+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:24.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:23 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:23.657+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:24.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:23 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:23.657+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:24.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:23 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:23.516+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:24.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:23 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:23.516+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:24.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:23 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:23.794+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:24.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:23 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:23.794+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:24.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:23 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:23.897+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:24.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:23 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:23.897+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:24.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:23 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:23.897+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:24.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:23.986+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:24.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:23 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:23.986+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:24.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:24.709+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:24.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:24.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:24.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:24.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:25.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:24.649+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:25.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:24.649+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:25.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:24.548+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:25.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:24 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:24.548+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:25.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:24.757+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:25.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:24.757+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:25.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:24.848+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:25.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:24.848+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:25.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:24 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:24.848+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:25.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:24.944+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:25.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:24 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:24.944+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:25.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:25.730+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:25.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:25.730+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:25.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:25.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:25.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:25.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:26.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:25.608+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:26.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:25.608+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:26.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:25.553+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:26.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:25.553+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:26.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:25.553+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:26.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:25 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:25.738+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:26.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:25 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:25.738+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:26.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:25 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:25.840+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:26.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:25 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:25.840+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:26.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:25 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:25.840+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:26.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:25.896+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:41.902505+0000 front 2023-10-18T09:36:41.902518+0000 (oldest deadline 2023-10-18T09:37:07.202380+0000) 2023-10-18T09:37:26.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:25.896+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.604402+0000 front 2023-10-18T09:36:53.604491+0000 (oldest deadline 2023-10-18T09:37:18.904459+0000) 2023-10-18T09:37:26.393 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:25 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5[149572]: 2023-10-18T09:37:25.896+0000 7faa43d77700 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.405412+0000 front 2023-10-18T09:36:59.405442+0000 (oldest deadline 2023-10-18T09:37:25.305551+0000) 2023-10-18T09:37:26.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:26 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:26.731+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:26.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:26 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:26.731+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:26.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:26 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:26.534+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:26.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:26 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:26.534+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:27.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:26 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:26.642+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:27.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:26 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:26.642+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:27.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:26 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:26.571+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:27.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:26 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:26.571+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:27.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:26 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:26.571+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:27.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:26 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:26.773+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:27.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:26 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:26.773+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:27.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:26 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:26.832+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:27.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:26 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:26.832+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:27.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:26 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:26.832+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:27.392 INFO:journalctl@ceph.osd.5.smithi170.stdout:Oct 18 09:37:26 smithi170 bash[180192]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-5 2023-10-18T09:37:27.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:27.702+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:27.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:27.702+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:27.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:27.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:27.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:27 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:27.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:28.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:27 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:27.684+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:28.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:27 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:27.684+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:28.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:27 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:27.547+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:28.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:27 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:27.547+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:28.004 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:27 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:27.547+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:28.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:27 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:27.737+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:28.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:27 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:27.737+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:28.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:27 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:27.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:28.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:27 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:27.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:28.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:27 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:27.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:28.305 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.5.service' 2023-10-18T09:37:28.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:28.347 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-18T09:37:28.347 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-18T09:37:28.347 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.6 2023-10-18T09:37:28.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:28 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:28.517+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:28.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:28 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:28.517+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:28.703 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 systemd[1]: Stopping Ceph osd.6 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:28.703 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:28.504+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:28.703 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:28.504+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:28.704 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:28.504+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:29.003 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:28.700+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:29.004 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:28.700+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:29.004 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:28.888+0000 7f064fe4f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:29.004 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:28.888+0000 7f064fe4f700 -1 osd.6 96 *** Got signal Terminated *** 2023-10-18T09:37:29.005 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:28 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:28.888+0000 7f064fe4f700 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:29.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:28 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:28.720+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:29.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:28 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:28.720+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:29.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:28 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:28.819+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:29.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:28 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:28.819+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:29.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:28 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:28.819+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:29.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:28 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:28.744+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:29.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:28 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:28.744+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:29.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:29.695+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:29.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:29.695+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:29.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:29.477+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:29.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:29.477+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:29.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:29 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:29.477+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:29.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:29.737+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:29.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:29.737+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:29.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:29.497+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:29.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:29 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:29.497+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:30.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:29 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:29.735+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:30.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:29 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:29.735+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:30.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:29 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:29.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:30.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:29 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:29.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:30.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:29 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:29.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:30.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:30.737+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:30.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:30.737+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:30.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:30.488+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:30.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:30.488+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:30.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:30.488+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:30.772 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:30.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:30.772 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:30.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:31.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:30.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:31.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:30 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:30.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:31.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:30 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:30.720+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:31.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:30 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:30.720+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:31.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:30 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:30.836+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:31.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:30 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:30.836+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:31.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:30 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:30.836+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:31.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:31.692+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:31.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:31.692+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:31.754 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:31.692+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.551751+0000 front 2023-10-18T09:37:09.551922+0000 (oldest deadline 2023-10-18T09:37:31.152360+0000) 2023-10-18T09:37:31.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:31.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:31.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:31.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:31.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:31 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:31.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:31.799 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:31.517+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:31.800 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:31.517+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:32.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:31.797+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:32.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:31 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:31.797+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:32.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:31.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:32.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:31.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:32.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:31.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:32.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:31.719+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:32.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:31.719+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:32.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:31 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:31.719+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:32.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:32.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:32.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:32.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:32.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:32.475+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:32.754 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:32.658+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:32.754 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:32.658+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:32.755 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:32 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:32.658+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.551751+0000 front 2023-10-18T09:37:09.551922+0000 (oldest deadline 2023-10-18T09:37:31.152360+0000) 2023-10-18T09:37:32.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:32.766+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:32.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:32.766+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:32.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:32.766+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:32.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:32.523+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:32.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:32.523+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:32.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:32 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:32.523+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:32.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:32.812+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:32.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:32.812+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:32.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:32.812+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:32.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:32.812+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:32.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:32.708+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:32.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:32.708+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:32.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:32 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:32.708+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:33.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:33.683+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.848093+0000 front 2023-10-18T09:36:49.847710+0000 (oldest deadline 2023-10-18T09:37:15.147871+0000) 2023-10-18T09:37:33.753 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:33.683+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.349611+0000 front 2023-10-18T09:37:00.349793+0000 (oldest deadline 2023-10-18T09:37:22.649786+0000) 2023-10-18T09:37:33.754 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6[134255]: 2023-10-18T09:37:33.683+0000 7f0649c40700 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.551751+0000 front 2023-10-18T09:37:09.551922+0000 (oldest deadline 2023-10-18T09:37:31.152360+0000) 2023-10-18T09:37:33.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:33.490+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:33.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:33.490+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:33.755 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:33 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:33.490+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:33.814 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:33.482+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:33.815 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:33.482+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:33.815 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:33.482+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:33.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:33.670+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:33.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:33.670+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:33.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:33.670+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:33.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:33.783+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:33.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:33.783+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:33.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:33.783+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:33.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:33 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:33.783+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:34.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:33.812+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:34.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:33.812+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:34.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:33 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:33.812+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:34.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:34.450+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:34.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:34.450+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:34.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:34 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:34.450+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:34.765 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:34.513+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:34.765 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:34.513+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:34.765 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:34.513+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:34.766 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:34.762+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:34.766 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:34.762+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:34.766 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:34 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:34.762+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:34.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:34.655+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:34.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:34.655+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:34.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:34.655+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:34.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:34.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:34.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:34.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:34.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:34.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:34.964 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:34.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:35.253 INFO:journalctl@ceph.osd.6.smithi061.stdout:Oct 18 09:37:34 smithi061 bash[179339]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-6 2023-10-18T09:37:35.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:35.418+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:35.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:35.418+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:35.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:35.418+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:35.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:35 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:35.418+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:35.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:35.713+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:35.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:35.713+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:35.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:35.713+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:35.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:35.478+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:35.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:35.478+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:35.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:35 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:35.478+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:35.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:35.639+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:35.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:35.639+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:35.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:35.639+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:35.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:35.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:35.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:35.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:35.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:35.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:35.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:35 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:35.774+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:36.688 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:36.409+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:36.689 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:36.409+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:36.689 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:36.409+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:36.689 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:36 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:36.409+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:36.743 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:36.473+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:36.744 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:36.473+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:36.744 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:36.473+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:36.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:36.666+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:36.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:36.666+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:36.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:36.666+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:36.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:36.806+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:36.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:36.806+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:36.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:36.806+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:36.964 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:36 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:36.806+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:37.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.6.service' 2023-10-18T09:37:37.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:37.114 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-18T09:37:37.114 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-18T09:37:37.115 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.7 2023-10-18T09:37:37.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:36.740+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:37.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:36.740+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:37.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:36 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:36.740+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:37.462 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 systemd[1]: Stopping Ceph osd.7 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:37.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:37 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:37.456+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:37.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:37 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:37.456+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:37.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:37 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:37.456+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:37.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:37 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:37.456+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:37.778 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:37.424+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:37.779 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:37.424+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:37.779 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:37.424+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:37.849 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.482+0000 7fd58538e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:37.850 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.482+0000 7fd58538e700 -1 osd.7 96 *** Got signal Terminated *** 2023-10-18T09:37:37.850 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.482+0000 7fd58538e700 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:37.850 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.664+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:37.850 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.664+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:37.851 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:37.664+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:38.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:37.776+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:38.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:37.776+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:38.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:37 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:37.776+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:38.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:37.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:38.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:37.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:38.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:37.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:38.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:37.847+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:38.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:38 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:38.437+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:38.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:38 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:38.437+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:38.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:38 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:38.437+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:38.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:38 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:38.437+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:38.820 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:38.465+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:38.821 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:38.465+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:38.821 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:38.465+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:38.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:38.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:38.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:38.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:38.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:38.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:38.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:38.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:38.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:38.675+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:38.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:38.675+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:38.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:38 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:38.675+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:39.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:38.818+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:39.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:38.818+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:39.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:38 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:38.818+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:39.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:39.426+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:39.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:39.426+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:39.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:39.426+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:39.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:39 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:39.426+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:39.839 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:39.486+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:39.839 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:39.486+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:39.840 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:39.486+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:39.840 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:39.486+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:39.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:39.668+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:39.962 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:39.668+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:39.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:39.668+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:39.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:39.842+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:39.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:39.842+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:39.964 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:39.842+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:39.964 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:39 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:39.842+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:40.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:39.837+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:40.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:39.837+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:40.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:39 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:39.837+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:40.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:40 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:40.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:40.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:40 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:40.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:40.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:40 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:40.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:40.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:40 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:40.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:40.819 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:40.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:40.819 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:40.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:40.820 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:40.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:40.820 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:40.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:40.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:40.865+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:40.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:40.865+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:40.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:40.865+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:40.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:40.865+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:40.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:40.707+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:40.963 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:40.707+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:40.964 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:40 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:40.707+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:41.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:40.817+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:41.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:40.817+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:41.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:40 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:40.817+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:41.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:41 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:41.380+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:41.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:41 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:41.380+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:41.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:41 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:41.380+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:41.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:41 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:41.380+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:41.812 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:41.541+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:41.812 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:41.541+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:41.812 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:41.541+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:41.812 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:41.541+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:42.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:41.810+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:42.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:41.810+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:42.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:41 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:41.810+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:42.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:41.746+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.495177+0000 front 2023-10-18T09:36:53.495207+0000 (oldest deadline 2023-10-18T09:37:15.195425+0000) 2023-10-18T09:37:42.212 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:41.746+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:58.696579+0000 front 2023-10-18T09:36:58.696728+0000 (oldest deadline 2023-10-18T09:37:23.396373+0000) 2023-10-18T09:37:42.213 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7[156551]: 2023-10-18T09:37:41.746+0000 7fd57f980700 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.398065+0000 front 2023-10-18T09:37:07.398122+0000 (oldest deadline 2023-10-18T09:37:30.898228+0000) 2023-10-18T09:37:42.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:41.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:42.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:41.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:42.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:41.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:42.214 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:41.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:42.214 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:41.853+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:42.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:42 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:42.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:42.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:42 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:42.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:42.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:42 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:42.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:42.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:42 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:42.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:42.830 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:42.510+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:42.831 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:42.510+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:42.831 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:42.510+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:42.831 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:42.510+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:43.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:42.828+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:43.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:42.828+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:43.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:42 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:42.828+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:43.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:42 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:42.867+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:43.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:42 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:42.867+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:43.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:42 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:42.867+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:43.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:42 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:42.867+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:43.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:42 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:42.867+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:43.712 INFO:journalctl@ceph.osd.7.smithi110.stdout:Oct 18 09:37:43 smithi110 bash[178288]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-7 2023-10-18T09:37:43.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:43 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:43.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:43.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:43 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:43.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:43.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:43 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:43.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:43.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:43 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:43.346+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:43.789 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:43.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:43.789 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:43.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:43.790 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:43.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:43.790 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:43.468+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:44.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:43.787+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:44.142 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:43.787+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:44.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:43.787+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:44.143 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:43 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:43.787+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:44.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:43 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:43.855+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:44.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:43 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:43.855+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:44.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:43 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:43.855+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:44.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:43 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:43.855+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:44.214 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:43 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:43.855+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:44.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.7.service' 2023-10-18T09:37:44.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:44.751 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-18T09:37:44.752 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-18T09:37:44.752 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.8 2023-10-18T09:37:44.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:44.367+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:44.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:44.367+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:44.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:44.367+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:44.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:44.367+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:44.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:44 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:44.367+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:44.783 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:44.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:44.784 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:44.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:44.784 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:44.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:44.784 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:44.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:44.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:44 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:44.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:44.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:44 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:44.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:44.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:44 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:44.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:44.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:44 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:44.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:44.963 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:44 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:44.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:45.119 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:44 smithi170 systemd[1]: Stopping Ceph osd.8 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:45.120 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:44.827+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:45.120 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:44.827+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:45.120 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:44.827+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:45.120 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:44 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:44.827+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:45.120 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.117+0000 7f0c46fb8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:45.121 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.117+0000 7f0c46fb8700 -1 osd.8 96 *** Got signal Terminated *** 2023-10-18T09:37:45.392 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.117+0000 7f0c46fb8700 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:45.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:45.396+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:45.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:45.396+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:45.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:45.396+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:45.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:45.396+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:45.755 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:45 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:45.396+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:45.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.780+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:45.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.780+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:45.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.780+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:45.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:45.780+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:45.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:45.532+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:45.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:45.532+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:45.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:45.532+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:45.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:45 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:45.532+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:46.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:45.890+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:46.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:45.890+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:46.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:45.890+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:46.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:45.890+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:46.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:45.890+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:46.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:46.365+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:46.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:46.365+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:46.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:46.365+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:46.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:46.365+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:46.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:46 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:46.365+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:46.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:46.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:46.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:46.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:46.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:46.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:46.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:46.769+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:46.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:46.526+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:46.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:46.526+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:46.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:46.526+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:46.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:46 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:46.526+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:47.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:47.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:47.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:47.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:47.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:47.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:46 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:46.888+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:47.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:47.339+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:47.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:47.339+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:47.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:47.339+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:47.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:47.339+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:47.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:47 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:47.339+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:47.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:47.514+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:47.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:47.514+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:47.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:47.514+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:47.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:47.514+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:47.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:47.514+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:47.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:47.760+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:47.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:47.760+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:47.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:47.760+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:47.895 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:47.760+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:47.895 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:47 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:47.760+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:23.127822+0000 front 2023-10-18T09:37:23.127856+0000 (oldest deadline 2023-10-18T09:37:47.227902+0000) 2023-10-18T09:37:48.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:48.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:48.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:48.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:48.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:48.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:47 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:47.910+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:48.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:48.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:48.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:48.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:48.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:48.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:48.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:48.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:48.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:48 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:48.379+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:48.892 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:48.728+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:48.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:48.728+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:48.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:48.728+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:48.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:48.728+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:48.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:48.728+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:23.127822+0000 front 2023-10-18T09:37:23.127856+0000 (oldest deadline 2023-10-18T09:37:47.227902+0000) 2023-10-18T09:37:48.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:48.488+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:48.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:48.488+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:48.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:48.488+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:48.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:48.488+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:48.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:48 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:48.488+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:49.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:49.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:49.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:49.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:49.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:49.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:48 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:48.951+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:49.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:49.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:49.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:49.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:49.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:49.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:49 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:49.414+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:49.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:49.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:49.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:49.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:49.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:49.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:49.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:49.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:49.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:49.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:49.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:49.734+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:53.323339+0000 front 2023-10-18T09:36:53.323241+0000 (oldest deadline 2023-10-18T09:37:15.623183+0000) 2023-10-18T09:37:49.893 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:49.734+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.324182+0000 front 2023-10-18T09:37:00.324210+0000 (oldest deadline 2023-10-18T09:37:25.024370+0000) 2023-10-18T09:37:49.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:49.734+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:05.024878+0000 front 2023-10-18T09:37:05.024900+0000 (oldest deadline 2023-10-18T09:37:32.625471+0000) 2023-10-18T09:37:49.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:49.734+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:18.427268+0000 front 2023-10-18T09:37:18.427246+0000 (oldest deadline 2023-10-18T09:37:43.127388+0000) 2023-10-18T09:37:49.894 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8[158334]: 2023-10-18T09:37:49.734+0000 7f0c41dab700 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:23.127822+0000 front 2023-10-18T09:37:23.127856+0000 (oldest deadline 2023-10-18T09:37:47.227902+0000) 2023-10-18T09:37:50.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:50.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:50.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:50.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:50.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:50.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:49 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:49.965+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:50.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:50.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:50.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:50.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:50.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:50.755 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:50 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:50.403+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:50.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:50 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:50.483+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:50.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:50 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:50.483+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:50.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:50 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:50.483+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:50.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:50 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:50.483+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:50.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:50 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:50.483+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:51.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:51.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:51.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:51.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:51.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:51.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:50 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:50.955+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:51.392 INFO:journalctl@ceph.osd.8.smithi170.stdout:Oct 18 09:37:51 smithi170 bash[180457]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-8 2023-10-18T09:37:51.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:51.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:51.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:51.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:51.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:51.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:51 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:51.355+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:51.816 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:51.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:51.817 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:51.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:51.817 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:51.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:51.817 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:51.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:51.818 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:51 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:51.516+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:52.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:52.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:52.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:52.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:52.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:52.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:51 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:51.991+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:52.553 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.8.service' 2023-10-18T09:37:52.633 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:37:52.633 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-18T09:37:52.634 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-18T09:37:52.634 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.9 2023-10-18T09:37:52.703 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:52.704 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:52.704 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:52.704 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:52.705 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:52.705 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:52.326+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:52.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:52 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:52.548+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:52.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:52 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:52.548+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:52.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:52 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:52.548+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:52.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:52 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:52.548+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:52.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:52 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:52.548+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:53.003 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:52 smithi061 systemd[1]: Stopping Ceph osd.9 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:37:53.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:53.462 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:53.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:53.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:53.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:53.463 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:52 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:52.974+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:53.503 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.153+0000 7f6d7fd8b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:37:53.504 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.153+0000 7f6d7fd8b700 -1 osd.9 96 *** Got signal Terminated *** 2023-10-18T09:37:53.504 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.153+0000 7f6d7fd8b700 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:37:53.504 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:53.504 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:53.505 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:53.505 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:53.505 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:53.505 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:53 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:53.374+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:53.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:53.539+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:53.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:53.539+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:53.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:53.539+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:53.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:53.539+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:53.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:53.539+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:54.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:54.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:54.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:54.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:54.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:54.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:53 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:53.940+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:54.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:54.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:54.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:54.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:54.755 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:54.755 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:54 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:54.389+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:54.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:54 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:54.504+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:54.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:54 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:54.504+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:54.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:54 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:54.504+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:54.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:54 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:54.504+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:54.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:54 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:54.504+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:55.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:55.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:55.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:55.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:55.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:55.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:54 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:54.917+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:55.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:55.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:55.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:55.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:55.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:55.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:55 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:55.383+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:55.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:55 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:55.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:55.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:55 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:55.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:55.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:55 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:55.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:55.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:55 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:55.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:55.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:55 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:55.476+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:56.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:56.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:56.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:56.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:56.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:56.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:55 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:55.914+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:56.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:56.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:56.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:56.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:56.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:56.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:56 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:56.423+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:56.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:56.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:56.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:56.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:56.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:56.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:56 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:56.499+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:37:57.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:57.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:57.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:57.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:57.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:57.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:56 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:56.894+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:57.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:44.044766+0000 front 2023-10-18T09:36:44.044595+0000 (oldest deadline 2023-10-18T09:37:09.344881+0000) 2023-10-18T09:37:57.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:49.345362+0000 front 2023-10-18T09:36:49.345458+0000 (oldest deadline 2023-10-18T09:37:14.546614+0000) 2023-10-18T09:37:57.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:59.248279+0000 front 2023-10-18T09:36:59.248308+0000 (oldest deadline 2023-10-18T09:37:25.148490+0000) 2023-10-18T09:37:57.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:08.649504+0000 front 2023-10-18T09:37:08.649524+0000 (oldest deadline 2023-10-18T09:37:35.049954+0000) 2023-10-18T09:37:57.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.851886+0000 front 2023-10-18T09:37:17.852054+0000 (oldest deadline 2023-10-18T09:37:43.751966+0000) 2023-10-18T09:37:57.754 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:57 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9[142749]: 2023-10-18T09:37:57.472+0000 7f6d7c381700 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.253847+0000 front 2023-10-18T09:37:24.253662+0000 (oldest deadline 2023-10-18T09:37:48.953918+0000) 2023-10-18T09:37:57.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:57.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:57.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:57.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:57.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:57.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:57.509+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:37:58.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:58.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:58.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:58.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:58.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:58.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.876+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:58.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:57 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:57.877+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:37:58.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:58.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:58.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:58.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:58.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:58.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:58 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:58.500+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:37:59.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:37:59.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:37:59.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:37:59.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:37:59.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:37:59.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:37:59.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:58 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:58.843+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:37:59.753 INFO:journalctl@ceph.osd.9.smithi061.stdout:Oct 18 09:37:59 smithi061 bash[179610]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-9 2023-10-18T09:37:59.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:37:59.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:37:59.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:37:59.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:37:59.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:37:59.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:37:59 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:37:59.545+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:00.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:00.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:00.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:00.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:00.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:00.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:00.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:37:59 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:37:59.820+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:00.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:00.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:00.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:00.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:00.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:00.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:00 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:00.515+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:01.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:01.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:01.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:01.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:01.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:01.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:01.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:00 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:00.856+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:01.418 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.9.service' 2023-10-18T09:38:01.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:01.510 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-18T09:38:01.510 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-18T09:38:01.510 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.10 2023-10-18T09:38:01.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:01.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:01.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:01.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:01.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:01.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:01.494+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:01.840 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 systemd[1]: Stopping Ceph osd.10 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:02.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:02.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:02.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:02.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:02.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:02.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:02.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.837+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:02.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.869+0000 7f0f5a177700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:02.214 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.869+0000 7f0f5a177700 -1 osd.10 96 *** Got signal Terminated *** 2023-10-18T09:38:02.214 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:01 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:01.869+0000 7f0f5a177700 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:38:02.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:02.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:02.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:02.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:02.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:02.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:02 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:02.525+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:03.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:03.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:03.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:03.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:03.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:03.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:03.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:02 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:02.810+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:03.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:03.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:03.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:03.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:03.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:03.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:03.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:03 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:03.538+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:04.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:04.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:04.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:04.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:04.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:04.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:04.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:03 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:03.833+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:04.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:04.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:04.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:04.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:04.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:04.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:04.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:04 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:04.551+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:05.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:05.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:05.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:05.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:05.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:05.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:05.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:04 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:04.883+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:05.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:05.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:05.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:05.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:05.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:05.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:05.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:05 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:05.536+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:06.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.851126+0000 front 2023-10-18T09:36:40.851159+0000 (oldest deadline 2023-10-18T09:37:10.251584+0000) 2023-10-18T09:38:06.212 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6804 osd.1 since back 2023-10-18T09:36:52.552602+0000 front 2023-10-18T09:36:52.552627+0000 (oldest deadline 2023-10-18T09:37:17.852825+0000) 2023-10-18T09:38:06.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:36:57.853441+0000 front 2023-10-18T09:36:57.853423+0000 (oldest deadline 2023-10-18T09:37:23.654167+0000) 2023-10-18T09:38:06.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:07.055942+0000 front 2023-10-18T09:37:07.055808+0000 (oldest deadline 2023-10-18T09:37:32.355855+0000) 2023-10-18T09:38:06.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:13.456739+0000 front 2023-10-18T09:37:13.456707+0000 (oldest deadline 2023-10-18T09:37:41.657831+0000) 2023-10-18T09:38:06.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:25.758992+0000 front 2023-10-18T09:37:25.759058+0000 (oldest deadline 2023-10-18T09:37:46.858894+0000) 2023-10-18T09:38:06.213 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:05 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10[164697]: 2023-10-18T09:38:05.869+0000 7f0f54769700 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:33.160948+0000 front 2023-10-18T09:37:33.160890+0000 (oldest deadline 2023-10-18T09:37:57.860840+0000) 2023-10-18T09:38:06.767 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:06.767 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:06.767 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:06.768 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:06.768 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:06.768 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:06.768 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:06 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:06.487+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:07.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:07.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:07.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:07.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:07.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:07.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:07.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:07 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:07.495+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:07.962 INFO:journalctl@ceph.osd.10.smithi110.stdout:Oct 18 09:38:07 smithi110 bash[178555]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-10 2023-10-18T09:38:08.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:08.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:08.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:08.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:08.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:08.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:08.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:08 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:08.484+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:09.006 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.10.service' 2023-10-18T09:38:09.045 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:09.046 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-18T09:38:09.046 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-18T09:38:09.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.11 2023-10-18T09:38:09.392 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 systemd[1]: Stopping Ceph osd.11 for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:09.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.412+0000 7fa62f8d9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:09.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.412+0000 7fa62f8d9700 -1 osd.11 96 *** Got signal Terminated *** 2023-10-18T09:38:09.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.412+0000 7fa62f8d9700 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-18T09:38:09.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:09.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:09.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:09.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:09.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:09.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:09.894 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:09 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:09.442+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:10.892 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:10.893 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:10 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:10.410+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:11.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:11.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:11.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:11.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:11.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:11.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:11.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:11 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:11.371+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:12.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:12.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:12.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:12.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:12.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:12.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:12.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:12 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:12.366+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:13.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:13.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:13.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:13.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:13.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:13.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:13.644 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:13.644 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:13 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:13.339+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2023-10-18T09:37:47.321842+0000 front 2023-10-18T09:37:47.321885+0000 (oldest deadline 2023-10-18T09:38:13.222004+0000) 2023-10-18T09:38:14.642 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6806 osd.0 since back 2023-10-18T09:36:40.910134+0000 front 2023-10-18T09:36:40.910070+0000 (oldest deadline 2023-10-18T09:37:06.810050+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2023-10-18T09:37:00.312558+0000 front 2023-10-18T09:37:00.312586+0000 (oldest deadline 2023-10-18T09:37:23.212698+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6814 osd.3 since back 2023-10-18T09:37:09.614120+0000 front 2023-10-18T09:37:09.614166+0000 (oldest deadline 2023-10-18T09:37:31.914104+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6812 osd.4 since back 2023-10-18T09:37:17.716044+0000 front 2023-10-18T09:37:17.716071+0000 (oldest deadline 2023-10-18T09:37:39.416048+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2023-10-18T09:37:24.717250+0000 front 2023-10-18T09:37:24.717271+0000 (oldest deadline 2023-10-18T09:37:47.017429+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.61:6822 osd.6 since back 2023-10-18T09:37:31.118987+0000 front 2023-10-18T09:37:31.118874+0000 (oldest deadline 2023-10-18T09:37:55.818767+0000) 2023-10-18T09:38:14.643 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.110:6820 osd.7 since back 2023-10-18T09:37:38.620472+0000 front 2023-10-18T09:37:38.620538+0000 (oldest deadline 2023-10-18T09:38:03.320431+0000) 2023-10-18T09:38:14.644 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:14 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11[167095]: 2023-10-18T09:38:14.370+0000 7fa627ec7700 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2023-10-18T09:37:47.321842+0000 front 2023-10-18T09:37:47.321885+0000 (oldest deadline 2023-10-18T09:38:13.222004+0000) 2023-10-18T09:38:15.191 INFO:journalctl@ceph.osd.11.smithi170.stdout:Oct 18 09:38:15 smithi170 bash[180777]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-osd-11 2023-10-18T09:38:16.554 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@osd.11.service' 2023-10-18T09:38:16.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:16.594 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-18T09:38:16.595 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-18T09:38:16.595 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.a 2023-10-18T09:38:17.003 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:38:16 smithi061 systemd[1]: Stopping Ceph mds.a for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:17.503 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:38:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: 2023-10-18T09:38:17.108+0000 7f79a2cde700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:17.503 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:38:17 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a[151125]: 2023-10-18T09:38:17.108+0000 7f79a2cde700 -1 mds.a *** got signal Terminated *** 2023-10-18T09:38:21.253 INFO:journalctl@ceph.mds.a.smithi061.stdout:Oct 18 09:38:21 smithi061 bash[179883]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-a 2023-10-18T09:38:21.520 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.a.service' 2023-10-18T09:38:21.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:21.610 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-18T09:38:21.611 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-18T09:38:21.611 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.d 2023-10-18T09:38:22.003 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:38:21 smithi061 systemd[1]: Stopping Ceph mds.d for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:22.503 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:38:22 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-d[150126]: 2023-10-18T09:38:22.174+0000 7f7cd301f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:22.504 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:38:22 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-d[150126]: 2023-10-18T09:38:22.174+0000 7f7cd301f700 -1 mds.d *** got signal Terminated *** 2023-10-18T09:38:25.003 INFO:journalctl@ceph.mds.d.smithi061.stdout:Oct 18 09:38:24 smithi061 bash[180036]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-d 2023-10-18T09:38:25.227 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.d.service' 2023-10-18T09:38:25.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:25.316 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-18T09:38:25.316 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-18T09:38:25.316 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.g 2023-10-18T09:38:25.503 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:38:25 smithi061 systemd[1]: Stopping Ceph mds.g for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:26.253 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:38:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: 2023-10-18T09:38:25.823+0000 7f595733e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:26.253 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:38:25 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g[150737]: 2023-10-18T09:38:25.823+0000 7f595733e700 -1 mds.g *** got signal Terminated *** 2023-10-18T09:38:29.503 INFO:journalctl@ceph.mds.g.smithi061.stdout:Oct 18 09:38:29 smithi061 bash[180184]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-g 2023-10-18T09:38:29.607 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.g.service' 2023-10-18T09:38:29.693 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:29.693 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-18T09:38:29.694 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-18T09:38:29.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.j 2023-10-18T09:38:30.003 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:38:29 smithi061 systemd[1]: Stopping Ceph mds.j for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:30.503 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:38:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-j[149739]: 2023-10-18T09:38:30.205+0000 7f982b281700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:30.503 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:38:30 smithi061 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-j[149739]: 2023-10-18T09:38:30.205+0000 7f982b281700 -1 mds.j *** got signal Terminated *** 2023-10-18T09:38:34.161 INFO:journalctl@ceph.mds.j.smithi061.stdout:Oct 18 09:38:33 smithi061 bash[180331]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-j 2023-10-18T09:38:34.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.j.service' 2023-10-18T09:38:34.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:34.292 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-18T09:38:34.293 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-18T09:38:34.293 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.b 2023-10-18T09:38:34.677 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:38:34 smithi110 systemd[1]: Stopping Ceph mds.b for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:34.962 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:38:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: 2023-10-18T09:38:34.674+0000 7fd048f84700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:34.962 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:38:34 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b[171804]: 2023-10-18T09:38:34.674+0000 7fd048f84700 -1 mds.b *** got signal Terminated *** 2023-10-18T09:38:37.212 INFO:journalctl@ceph.mds.b.smithi110.stdout:Oct 18 09:38:36 smithi110 bash[178824]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-b 2023-10-18T09:38:37.297 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.b.service' 2023-10-18T09:38:37.333 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:37.334 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-18T09:38:37.334 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-18T09:38:37.335 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.e 2023-10-18T09:38:37.712 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:38:37 smithi110 systemd[1]: Stopping Ceph mds.e for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:38.212 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:38:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: 2023-10-18T09:38:37.717+0000 7fdabc167700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:38.212 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:38:37 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e[171111]: 2023-10-18T09:38:37.717+0000 7fdabc167700 -1 mds.e *** got signal Terminated *** 2023-10-18T09:38:41.081 INFO:journalctl@ceph.mds.e.smithi110.stdout:Oct 18 09:38:41 smithi110 bash[178971]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-e 2023-10-18T09:38:41.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.e.service' 2023-10-18T09:38:41.302 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:41.303 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-18T09:38:41.303 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-18T09:38:41.303 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.h 2023-10-18T09:38:41.692 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:38:41 smithi110 systemd[1]: Stopping Ceph mds.h for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:41.962 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:38:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: 2023-10-18T09:38:41.690+0000 7f472fee4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:41.962 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:38:41 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h[171454]: 2023-10-18T09:38:41.690+0000 7f472fee4700 -1 mds.h *** got signal Terminated *** 2023-10-18T09:38:45.372 INFO:journalctl@ceph.mds.h.smithi110.stdout:Oct 18 09:38:45 smithi110 bash[179108]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-h 2023-10-18T09:38:45.378 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.h.service' 2023-10-18T09:38:45.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:45.412 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-18T09:38:45.413 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-18T09:38:45.413 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.k 2023-10-18T09:38:45.712 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:38:45 smithi110 systemd[1]: Stopping Ceph mds.k for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:46.212 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:38:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: 2023-10-18T09:38:45.780+0000 7fa23d325700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:46.212 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:38:45 smithi110 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k[170772]: 2023-10-18T09:38:45.780+0000 7fa23d325700 -1 mds.k *** got signal Terminated *** 2023-10-18T09:38:49.270 INFO:journalctl@ceph.mds.k.smithi110.stdout:Oct 18 09:38:49 smithi110 bash[179253]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-k 2023-10-18T09:38:49.379 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.k.service' 2023-10-18T09:38:49.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:49.414 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-18T09:38:49.415 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-18T09:38:49.415 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.c 2023-10-18T09:38:49.785 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:38:49 smithi170 systemd[1]: Stopping Ceph mds.c for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:50.142 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:38:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: 2023-10-18T09:38:49.781+0000 7f3f8c326700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:50.142 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:38:49 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c[173279]: 2023-10-18T09:38:49.781+0000 7f3f8c326700 -1 mds.c *** got signal Terminated *** 2023-10-18T09:38:53.340 INFO:journalctl@ceph.mds.c.smithi170.stdout:Oct 18 09:38:52 smithi170 bash[181049]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-c 2023-10-18T09:38:53.347 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.c.service' 2023-10-18T09:38:53.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:53.385 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-18T09:38:53.385 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-18T09:38:53.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.f 2023-10-18T09:38:53.642 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:38:53 smithi170 systemd[1]: Stopping Ceph mds.f for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:54.142 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:38:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: 2023-10-18T09:38:53.763+0000 7f4317710700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:54.142 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:38:53 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f[172939]: 2023-10-18T09:38:53.763+0000 7f4317710700 -1 mds.f *** got signal Terminated *** 2023-10-18T09:38:57.282 INFO:journalctl@ceph.mds.f.smithi170.stdout:Oct 18 09:38:56 smithi170 bash[181195]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-f 2023-10-18T09:38:57.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.f.service' 2023-10-18T09:38:57.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:38:57.322 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-18T09:38:57.323 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-18T09:38:57.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.i 2023-10-18T09:38:57.642 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:38:57 smithi170 systemd[1]: Stopping Ceph mds.i for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:38:58.142 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:38:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: 2023-10-18T09:38:57.703+0000 7fa02f9bc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:38:58.142 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:38:57 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i[173617]: 2023-10-18T09:38:57.703+0000 7fa02f9bc700 -1 mds.i *** got signal Terminated *** 2023-10-18T09:39:01.070 INFO:journalctl@ceph.mds.i.smithi170.stdout:Oct 18 09:39:01 smithi170 bash[181340]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-i 2023-10-18T09:39:01.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.i.service' 2023-10-18T09:39:01.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:39:01.292 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-18T09:39:01.293 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-18T09:39:01.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.l 2023-10-18T09:39:01.642 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:39:01 smithi170 systemd[1]: Stopping Ceph mds.l for c578e0ac-6d95-11ee-8db6-212e2dc638e7... 2023-10-18T09:39:02.142 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:39:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: 2023-10-18T09:39:01.678+0000 7f4491eff700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-18T09:39:02.142 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:39:01 smithi170 ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l[172598]: 2023-10-18T09:39:01.678+0000 7f4491eff700 -1 mds.l *** got signal Terminated *** 2023-10-18T09:39:05.330 INFO:journalctl@ceph.mds.l.smithi170.stdout:Oct 18 09:39:05 smithi170 bash[181478]: ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7-mds-l 2023-10-18T09:39:05.335 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c578e0ac-6d95-11ee-8db6-212e2dc638e7@mds.l.service' 2023-10-18T09:39:05.368 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-18T09:39:05.369 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-18T09:39:05.369 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T09:39:05.777 INFO:teuthology.orchestra.run.smithi061.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:39:09.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T09:39:09.557 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:39:11.158 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-18T09:39:11.473 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:39:13.042 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:39:13.124 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:39:13.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-18T09:39:13.199 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-18T09:39:13.208 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi061/crash 2023-10-18T09:39:13.211 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T09:39:13.317 INFO:teuthology.orchestra.run.smithi061.stderr:tar: /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T09:39:13.318 INFO:teuthology.orchestra.run.smithi061.stderr:tar: Error is not recoverable: exiting now 2023-10-18T09:39:13.321 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi110/crash 2023-10-18T09:39:13.322 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T09:39:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T09:39:13.375 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2023-10-18T09:39:13.377 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi170/crash 2023-10-18T09:39:13.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash -- . 2023-10-18T09:39:13.424 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-18T09:39:13.425 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2023-10-18T09:39:13.425 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-18T09:39:13.426 DEBUG:teuthology.orchestra.run.smithi061:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-18T09:39:13.506 INFO:teuthology.orchestra.run.smithi061.stderr:grep: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph.log: No such file or directory 2023-10-18T09:39:13.511 INFO:tasks.cephadm:Compressing logs... 2023-10-18T09:39:13.511 DEBUG:teuthology.orchestra.run.smithi061:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T09:39:13.516 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T09:39:13.518 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-18T09:39:13.565 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-18T09:39:13.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-18T09:39:13.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.5.log 2023-10-18T09:39:13.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.2.log 2023-10-18T09:39:13.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.11.log 2023-10-18T09:39:13.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.5.log: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.z.log 2023-10-18T09:39:13.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.8.log 2023-10-18T09:39:13.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.f.log 2023-10-18T09:39:13.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.c.log 2023-10-18T09:39:13.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.i.log 2023-10-18T09:39:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.c.log 2023-10-18T09:39:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T09:39:13.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.i.log: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.l.log 2023-10-18T09:39:13.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.l.log: 93.2% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-18T09:39:13.589 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-18T09:39:13.589 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-18T09:39:13.598 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.10.log 2023-10-18T09:39:13.599 INFO:teuthology.orchestra.run.smithi061.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-18T09:39:13.599 INFO:teuthology.orchestra.run.smithi061.stderr:: No such file or directory 2023-10-18T09:39:13.600 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.h.log 2023-10-18T09:39:13.601 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.e.log 2023-10-18T09:39:13.601 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.10.log: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.4.log 2023-10-18T09:39:13.601 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.1.log 2023-10-18T09:39:13.601 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.k.log 2023-10-18T09:39:13.603 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T09:39:13.603 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.7.log 2023-10-18T09:39:13.603 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.b.log 2023-10-18T09:39:13.603 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.b.log 2023-10-18T09:39:13.603 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.y.log 2023-10-18T09:39:13.605 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-18T09:39:13.606 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.0.log 2023-10-18T09:39:13.607 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.g.log 2023-10-18T09:39:13.608 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.y.log: 93.1% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-18T09:39:13.608 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.a.log 2023-10-18T09:39:13.609 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.g.log: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-18T09:39:13.609 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.6.log 2023-10-18T09:39:13.610 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.3.log 2023-10-18T09:39:13.611 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.d.log 2023-10-18T09:39:13.612 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.x.log 2023-10-18T09:39:13.613 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.9.log 2023-10-18T09:39:13.614 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-18T09:39:13.615 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.j.log 2023-10-18T09:39:13.616 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.a.log 2023-10-18T09:39:13.617 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.j.log: 94.4% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.d.log.gz 2023-10-18T09:39:13.621 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.a.log: 94.4% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.j.log.gz 2023-10-18T09:39:13.698 INFO:teuthology.orchestra.run.smithi061.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T09:39:13.722 INFO:teuthology.orchestra.run.smithi170.stderr: 93.7% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.l.log.gz 2023-10-18T09:39:13.990 INFO:teuthology.orchestra.run.smithi061.stderr: 90.9% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T09:39:14.076 INFO:teuthology.orchestra.run.smithi061.stderr: 89.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-18T09:39:14.467 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T09:39:14.475 INFO:teuthology.orchestra.run.smithi061.stderr: 93.9% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.a.log.gz 2023-10-18T09:39:14.488 INFO:teuthology.orchestra.run.smithi170.stderr: 91.0% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-18T09:39:14.490 INFO:teuthology.orchestra.run.smithi170.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T09:39:14.548 INFO:teuthology.orchestra.run.smithi110.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-18T09:39:14.770 INFO:teuthology.orchestra.run.smithi110.stderr: 91.8% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.b.log.gz 2023-10-18T09:39:14.872 INFO:teuthology.orchestra.run.smithi061.stderr: 94.4% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.g.log.gz 2023-10-18T09:39:15.160 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.c.log.gz 2023-10-18T09:39:15.186 INFO:teuthology.orchestra.run.smithi110.stderr: 93.8% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.k.log.gz 2023-10-18T09:39:15.724 INFO:teuthology.orchestra.run.smithi170.stderr: 94.7% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.c.log.gz 2023-10-18T09:39:16.000 INFO:teuthology.orchestra.run.smithi061.stderr: 90.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mon.a.log.gz 2023-10-18T09:39:16.204 INFO:teuthology.orchestra.run.smithi170.stderr: 93.4% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.f.log.gz 2023-10-18T09:39:18.443 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.i.log.gz 2023-10-18T09:39:18.854 INFO:teuthology.orchestra.run.smithi110.stderr: 93.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.h.log.gz 2023-10-18T09:41:27.320 INFO:teuthology.orchestra.run.smithi170.stderr: 90.5% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.8.log.gz 2023-10-18T09:41:39.859 INFO:teuthology.orchestra.run.smithi110.stderr: 90.7% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.10.log.gz 2023-10-18T09:42:05.612 INFO:teuthology.orchestra.run.smithi110.stderr: 98.0% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.e.log.gz 2023-10-18T09:42:06.323 INFO:teuthology.orchestra.run.smithi170.stderr: 90.8% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.11.log.gz 2023-10-18T09:42:32.618 INFO:teuthology.orchestra.run.smithi061.stderr: 90.5% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.0.log.gz 2023-10-18T09:42:45.307 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.2.log.gz 2023-10-18T09:43:00.787 INFO:teuthology.orchestra.run.smithi061.stderr: 90.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.3.log.gz 2023-10-18T09:43:05.677 INFO:teuthology.orchestra.run.smithi110.stderr: 90.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.1.log.gz 2023-10-18T09:43:06.632 INFO:teuthology.orchestra.run.smithi061.stderr: 90.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.6.log.gz 2023-10-18T09:43:23.886 INFO:teuthology.orchestra.run.smithi170.stderr: 90.7% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.5.log.gz 2023-10-18T09:43:24.236 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-18T09:43:24.237 INFO:teuthology.orchestra.run.smithi170.stderr:real 4m10.702s 2023-10-18T09:43:24.237 INFO:teuthology.orchestra.run.smithi170.stderr:user 6m27.400s 2023-10-18T09:43:24.237 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m14.035s 2023-10-18T09:43:46.708 INFO:teuthology.orchestra.run.smithi110.stderr: 98.3% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-mds.b.log.gz 2023-10-18T09:43:49.075 INFO:teuthology.orchestra.run.smithi061.stderr: 90.7% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.9.log.gz 2023-10-18T09:43:49.079 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-10-18T09:43:49.079 INFO:teuthology.orchestra.run.smithi061.stderr:real 4m35.526s 2023-10-18T09:43:49.080 INFO:teuthology.orchestra.run.smithi061.stderr:user 7m12.422s 2023-10-18T09:43:49.080 INFO:teuthology.orchestra.run.smithi061.stderr:sys 0m49.004s 2023-10-18T09:43:59.887 INFO:teuthology.orchestra.run.smithi110.stderr: 90.6% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.7.log.gz 2023-10-18T09:44:25.050 INFO:teuthology.orchestra.run.smithi110.stderr: 90.9% -- replaced with /var/log/ceph/c578e0ac-6d95-11ee-8db6-212e2dc638e7/ceph-osd.4.log.gz 2023-10-18T09:44:25.052 INFO:teuthology.orchestra.run.smithi110.stderr: 2023-10-18T09:44:25.052 INFO:teuthology.orchestra.run.smithi110.stderr:real 5m11.517s 2023-10-18T09:44:25.052 INFO:teuthology.orchestra.run.smithi110.stderr:user 7m2.536s 2023-10-18T09:44:25.053 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m17.842s 2023-10-18T09:44:25.053 INFO:tasks.cephadm:Archiving logs... 2023-10-18T09:44:25.054 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi061/log 2023-10-18T09:44:25.056 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T09:45:37.437 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi110/log 2023-10-18T09:45:37.439 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T09:46:39.999 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi170/log 2023-10-18T09:46:40.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-18T09:47:50.776 INFO:tasks.cephadm:Removing cluster... 2023-10-18T09:47:50.777 DEBUG:teuthology.orchestra.run.smithi061:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force 2023-10-18T09:47:51.397 INFO:teuthology.orchestra.run.smithi061.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:47:53.609 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force 2023-10-18T09:47:54.275 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:47:55.216 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c578e0ac-6d95-11ee-8db6-212e2dc638e7 --force 2023-10-18T09:47:55.956 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: c578e0ac-6d95-11ee-8db6-212e2dc638e7 2023-10-18T09:47:56.975 INFO:tasks.cephadm:Removing cephadm ... 2023-10-18T09:47:56.976 DEBUG:teuthology.orchestra.run.smithi061:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T09:47:57.026 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T09:47:57.060 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-18T09:47:57.090 INFO:tasks.cephadm:Teardown complete 2023-10-18T09:47:57.090 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-18T09:47:57.101 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-18T09:47:57.102 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T09:47:57.107 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T09:47:57.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-18T09:47:57.261 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-18T09:47:57.272 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-18T09:47:57.272 DEBUG:teuthology.orchestra.run.smithi061:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:47:57.278 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:47:57.281 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-18T09:47:57.296 INFO:teuthology.orchestra.run.smithi110.stderr:bash: ntpq: command not found 2023-10-18T09:47:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-10-18T09:47:57.316 INFO:teuthology.orchestra.run.smithi061.stderr:bash: ntpq: command not found 2023-10-18T09:47:57.317 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:47:57.317 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2023-10-18T09:47:57.318 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +2555us[+2525us] +/- 57ms 2023-10-18T09:47:57.318 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1413us[-1443us] +/- 72ms 2023-10-18T09:47:57.318 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -1737us[-1766us] +/- 72ms 2023-10-18T09:47:57.318 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:47:57.321 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:47:57.321 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-18T09:47:57.322 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +2443us[+2484us] +/- 57ms 2023-10-18T09:47:57.322 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 24 -1529us[-1488us] +/- 72ms 2023-10-18T09:47:57.322 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -1831us[-1790us] +/- 72ms 2023-10-18T09:47:57.322 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:47:57.332 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-18T09:47:57.332 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-10-18T09:47:57.332 INFO:teuthology.orchestra.run.smithi061.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +2539us[+2495us] +/- 57ms 2023-10-18T09:47:57.333 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -1423us[-1423us] +/- 72ms 2023-10-18T09:47:57.333 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 84 -1699us[-1745us] +/- 71ms 2023-10-18T09:47:57.333 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-18T09:47:57.334 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-18T09:47:57.345 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-18T09:47:57.345 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-18T09:47:57.426 DEBUG:teuthology.orchestra.run.smithi061:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T09:47:57.713 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T09:47:57.897 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-18T09:47:58.060 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T09:47:58.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:58.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T09:47:58.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:58.177 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-18T09:47:58.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:58.210 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-18T09:47:58.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-18T09:47:58.235 INFO:teuthology.task.internal:Duration was 3385.751230 seconds 2023-10-18T09:47:58.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-18T09:47:58.246 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-18T09:47:58.247 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T09:47:58.251 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T09:47:58.254 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-18T09:47:58.332 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T09:47:58.336 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T09:47:58.394 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-18T09:47:58.672 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-18T09:47:58.673 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi061.front.sepia.ceph.com 2023-10-18T09:47:58.673 DEBUG:teuthology.orchestra.run.smithi061:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T09:47:58.741 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2023-10-18T09:47:58.741 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T09:47:58.772 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2023-10-18T09:47:58.773 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-18T09:47:58.826 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-18T09:47:58.826 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T09:47:58.831 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T09:47:58.834 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-18T09:47:58.941 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-18T09:47:58.942 DEBUG:teuthology.orchestra.run.smithi061:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T09:47:58.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T09:47:58.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-18T09:47:59.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-18T09:47:59.137 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-18T09:47:59.138 DEBUG:teuthology.orchestra.run.smithi061:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T09:47:59.210 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T09:47:59.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-18T09:47:59.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-18T09:47:59.308 DEBUG:teuthology.orchestra.run.smithi061:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.313 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.316 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.353 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2023-10-18T09:47:59.356 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2023-10-18T09:47:59.385 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = core 2023-10-18T09:47:59.426 DEBUG:teuthology.orchestra.run.smithi061:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:59.469 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:59.487 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-18T09:47:59.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-18T09:47:59.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-18T09:47:59.515 INFO:teuthology.task.internal:Transferring archived files... 2023-10-18T09:47:59.517 DEBUG:teuthology.misc:Transferring archived files from smithi061:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi061 2023-10-18T09:47:59.517 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T09:47:59.606 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi110 2023-10-18T09:47:59.607 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T09:47:59.653 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726/remote/smithi170 2023-10-18T09:47:59.653 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-18T09:47:59.695 INFO:teuthology.task.internal:Removing archive directory... 2023-10-18T09:47:59.695 DEBUG:teuthology.orchestra.run.smithi061:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T09:47:59.700 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T09:47:59.702 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-18T09:47:59.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-18T09:47:59.757 INFO:teuthology.task.internal:Not uploading archives. 2023-10-18T09:47:59.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-18T09:47:59.811 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-18T09:47:59.811 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T09:47:59.816 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T09:47:59.818 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-18T09:47:59.832 INFO:teuthology.orchestra.run.smithi110.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 09:47 /home/ubuntu/cephtest 2023-10-18T09:47:59.834 INFO:teuthology.orchestra.run.smithi170.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 09:47 /home/ubuntu/cephtest 2023-10-18T09:47:59.856 INFO:teuthology.orchestra.run.smithi061.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 18 09:47 /home/ubuntu/cephtest 2023-10-18T09:47:59.859 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-18T09:47:59.871 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-18T09:47:59.894 INFO:teuthology.nuke:Checking targets against current locks 2023-10-18T09:47:59.924 DEBUG:teuthology.nuke:shortname: smithi061 2023-10-18T09:47:59.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T09:47:59.950 DEBUG:teuthology.nuke:shortname: smithi110 2023-10-18T09:47:59.950 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T09:47:59.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.407029', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T09:47:59.981 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.408790', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T09:47:59.982 DEBUG:teuthology.nuke:shortname: smithi170 2023-10-18T09:47:59.983 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-18T09:48:00.013 INFO:teuthology.orchestra.console.smithi061:Power off 2023-10-18T09:48:00.014 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T09:48:00.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-16_23:59:58-fs-wip-batrick-testing-20231016.203825-distro-default-smithi/7430726', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-18 08:42:56.405045', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-18T09:48:00.034 DEBUG:teuthology.orchestra.console.smithi061:power off output: Chassis Power Control: Down/Off 2023-10-18T09:48:00.034 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:00.056 INFO:teuthology.orchestra.console.smithi110:Power off 2023-10-18T09:48:00.056 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T09:48:00.068 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-10-18T09:48:00.077 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off 2023-10-18T09:48:00.077 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:00.093 INFO:teuthology.orchestra.console.smithi170:Power off 2023-10-18T09:48:00.093 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-18T09:48:00.101 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2023-10-18T09:48:00.117 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-10-18T09:48:00.117 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:00.146 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-18T09:48:04.070 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:04.103 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:04.148 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:04.181 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-10-18T09:48:04.213 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2023-10-18T09:48:04.258 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-18T09:48:08.182 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:08.214 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:08.260 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-18T09:48:08.294 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is off 2023-10-18T09:48:08.325 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2023-10-18T09:48:08.371 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-10-18T09:48:08.395 INFO:teuthology.orchestra.console.smithi061:Power off completed 2023-10-18T09:48:08.427 INFO:teuthology.orchestra.console.smithi110:Power off completed 2023-10-18T09:48:08.472 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-10-18T09:48:08.578 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3385.7512304782867 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2023-10-18T09:48:08.579 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-18T09:48:08.684 INFO:teuthology.run:FAIL